From guido at python.org  Sun Oct  1 23:11:41 2006
From: guido at python.org (Guido van Rossum)
Date: Sun, 1 Oct 2006 14:11:41 -0700
Subject: [Python-3000] Removing __var
In-Reply-To: <0d2101c6e012$9f88be90$4bbd2997@bagio>
References: <6a36e7290609221131v55d52401s59a74e55b8575376@mail.gmail.com>
	<E1GRSj8-0006VK-K5@swing.co.at> <0d2101c6e012$9f88be90$4bbd2997@bagio>
Message-ID: <ca471dc20610011411l17772730u528458721982b1a@mail.gmail.com>

I'm on the fence about dropping __var (some people back in the days
*begged* for it, and it had nothing to do with super; I wonder what
they think now).

But I'm all for adding syntactic sugar to Py3k. Can someone please
draft a proto-PEP? I think the compiler could treat super as a
reserved word and turn super.foo(args) into __super__(<thisclass>,
self).foo(args). Or something. Please be sure you understand the
issues (e.g. you can't rely on self.__class__). It would be great if
it worked inside class methods too. For static methods it don't see
how it ca work since there's no instance; or perhaps it should just
use the static hierarchy? Try not to go overboard; KISS etc.

--Guido

On 9/24/06, Giovanni Bajo <rasky at develer.com> wrote:
> Christian Tanzer wrote:
>
> > I don't use __ for `private`, I use it for making cooperative super
> > calls (and `__super` occurs 1397 in my sandbox).
>
> I think you might be confusing the symptom for the disease. To me, your mail
> means that Py3k should grow some syntactic sugar for super calls. I guess if
> that happens, you won't be missing __.
>
> Giovanni Bajo
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rasky at develer.com  Sun Oct  1 23:20:56 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 1 Oct 2006 23:20:56 +0200
Subject: [Python-3000] Removing __var
References: <6a36e7290609221131v55d52401s59a74e55b8575376@mail.gmail.com>
	<E1GRSj8-0006VK-K5@swing.co.at>
	<0d2101c6e012$9f88be90$4bbd2997@bagio>
	<ca471dc20610011411l17772730u528458721982b1a@mail.gmail.com>
Message-ID: <091701c6e59f$7bf8c6e0$9a4c2a97@bagio>

Guido van Rossum wrote:

> But I'm all for adding syntactic sugar to Py3k. Can someone please
> draft a proto-PEP? I think the compiler could treat super as a
> reserved word and turn super.foo(args) into __super__(<thisclass>,
> self).foo(args). Or something. Please be sure you understand the
> issues (e.g. you can't rely on self.__class__). It would be great if
> it worked inside class methods too. For static methods it don't see
> how it ca work since there's no instance; or perhaps it should just
> use the static hierarchy? Try not to go overboard; KISS etc.

Would you want this PEP to address the fact that, in most cases, you just want
to forward all the arguments to super call (aka super.foo(*args, **args)), ala
Dylan's nextmethod? Or is this what you meant with KISS? :)

One idea would be just:

   super     (or super())

which is equivalent to:

   super.<thismethodname>(<all arguments as passed>)

Let me also state the obvious: *args,**kwargs is not always available, so
having a short syntax to "do the most common thing" is going to be very useful.

Giovanni Bajo


From jimjjewett at gmail.com  Mon Oct  2 00:51:38 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 1 Oct 2006 18:51:38 -0400
Subject: [Python-3000] Removing __var
In-Reply-To: <ca471dc20610011411l17772730u528458721982b1a@mail.gmail.com>
References: <6a36e7290609221131v55d52401s59a74e55b8575376@mail.gmail.com>
	<E1GRSj8-0006VK-K5@swing.co.at> <0d2101c6e012$9f88be90$4bbd2997@bagio>
	<ca471dc20610011411l17772730u528458721982b1a@mail.gmail.com>
Message-ID: <fb6fbf560610011551k38cb709escd0672b00f07302f@mail.gmail.com>

On 10/1/06, Guido van Rossum <guido at python.org> wrote:
> I'm on the fence about dropping __var (some people back in the days
> *begged* for it, and it had nothing to do with super; I wonder what
> they think now).

Until Christian Tanzer's post, the only good use case I had seen was
extreme backwards compatibility for subclasses -- no additional names
would be used.  Are you suggesting

    (1)   another way to get a private namespace,
or (2)   the _name convention is sufficient

Either seems OK to me, but I do note that (2) doesn't work for
Christian's case. He isn't just warning people to be careful -- he
fully expects the child class to want the same private name.

-jJ

From jimjjewett at gmail.com  Mon Oct  2 01:07:19 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 1 Oct 2006 19:07:19 -0400
Subject: [Python-3000] super (was: Removing __var)
Message-ID: <fb6fbf560610011607g1ac51d0cu90e45ad30e26d712@mail.gmail.com>

On 10/1/06, Guido van Rossum <guido at python.org> wrote:
> But I'm all for adding syntactic sugar to Py3k. Can someone please
> draft a proto-PEP? I think the compiler could treat super as a
> reserved word and turn super.foo(args) into __super__(<thisclass>,
> self).foo(args).

Are you saying that it would be OK to peek inside the function at
compile-time and capture self.__class__ then, rather than waiting for
run-time (when self might really be a subclass instance)?

On its own that seems reasonable, but the "peek early" proposals are
starting to add up, and they aren't all consistent.

   The stuff inside a function definition doesn't get evaluated until
the function is called.

    Except that if it has "yield", that changes the compiler settings.
 Nothing gets evalated, we just peek.

    And a switch creation does get evaluated early (if that happens).

    And a static/once/final variable gets evaluated early (to line up
with switch)

    And a super gets created early, but only halfway.  It freezes
self.__class__, but not the self.__mro__

-jJ

From fdrake at acm.org  Mon Oct  2 02:22:50 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 1 Oct 2006 20:22:50 -0400
Subject: [Python-3000] Removing __var
In-Reply-To: <091701c6e59f$7bf8c6e0$9a4c2a97@bagio>
References: <6a36e7290609221131v55d52401s59a74e55b8575376@mail.gmail.com>
	<ca471dc20610011411l17772730u528458721982b1a@mail.gmail.com>
	<091701c6e59f$7bf8c6e0$9a4c2a97@bagio>
Message-ID: <200610012022.51120.fdrake@acm.org>

On Sunday 01 October 2006 17:20, Giovanni Bajo wrote:
 > Would you want this PEP to address the fact that, in most cases, you just
 > want to forward all the arguments to super call (aka super.foo(*args,
 > **args)), ala Dylan's nextmethod? Or is this what you meant with KISS? :)

It's not at all clear to me that passing along *args, **kwargs is the most 
common thing.  While I've done that at times, it's been the exception rather 
than the rule.

I expect the most valuable aspect of some new syntax is to replace the 
super(MyClass, self) with something shorter; possibly super(MyClass, 
self).thismethod.  Something like this seems sufficient:

--------------------------------------
class Derived(Base):

    def method(self, *args, **kw):
        supercall(*args, **kw)
        self.did_this = True
--------------------------------------


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From greg.ewing at canterbury.ac.nz  Mon Oct  2 03:03:51 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 02 Oct 2006 14:03:51 +1300
Subject: [Python-3000] Removing __var
In-Reply-To: <ca471dc20610011411l17772730u528458721982b1a@mail.gmail.com>
References: <6a36e7290609221131v55d52401s59a74e55b8575376@mail.gmail.com>
	<E1GRSj8-0006VK-K5@swing.co.at> <0d2101c6e012$9f88be90$4bbd2997@bagio>
	<ca471dc20610011411l17772730u528458721982b1a@mail.gmail.com>
Message-ID: <45206577.9010604@canterbury.ac.nz>

Guido van Rossum wrote:
> I think the compiler could treat super as a
> reserved word and turn super.foo(args) into __super__(<thisclass>,
> self).foo(args). Or something.

That sounds reasonable.

What would you do in the case of multiple base classes?

--
Greg

From greg.ewing at canterbury.ac.nz  Mon Oct  2 03:04:08 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 02 Oct 2006 14:04:08 +1300
Subject: [Python-3000] Removing __var
In-Reply-To: <091701c6e59f$7bf8c6e0$9a4c2a97@bagio>
References: <6a36e7290609221131v55d52401s59a74e55b8575376@mail.gmail.com>
	<E1GRSj8-0006VK-K5@swing.co.at> <0d2101c6e012$9f88be90$4bbd2997@bagio>
	<ca471dc20610011411l17772730u528458721982b1a@mail.gmail.com>
	<091701c6e59f$7bf8c6e0$9a4c2a97@bagio>
Message-ID: <45206588.1@canterbury.ac.nz>

Giovanni Bajo wrote:

> Would you want this PEP to address the fact that, in most cases, you just want
> to forward all the arguments to super call (aka super.foo(*args, **args))

I dispute this. By far the majority of the super calls
that I make are to __init__ methods, and I almost
always do *not* want to do this.

--
Greg

From martin at v.loewis.de  Mon Oct  2 22:20:01 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Oct 2006 22:20:01 +0200
Subject: [Python-3000] Pre-PEP: Easy Text File Decoding
In-Reply-To: <u1vfg2dnld13h9qh9pm62c4k032ivt7kic@4ax.com>
References: <20060911094447.hjabms9i7sgs4g8s@login.werra.lunarpages.com>	<79990c6b0609111449j1ae99757m35037b27c5727eda@mail.gmail.com>	<1cb725390609111530o3f7b0faah7d87a8cdf532cf39@mail.gmail.com>	<87r6yij7ea.fsf@qrnik.zagroda>	<1cb725390609111816k7187a15eu91c4029dad65885d@mail.gmail.com>
	<u1vfg2dnld13h9qh9pm62c4k032ivt7kic@4ax.com>
Message-ID: <45217471.7050901@v.loewis.de>

John S. Yates, Jr. schrieb:
> It is a mistake on Microsoft's part to fail to strip the BOM
> during conversion to UTF-8.  There is no MEANINGFUL definition
> of BOM in a UTF-8 string.

That's not true. See

http://unicode.org/faq/utf_bom.html#23
http://unicode.org/faq/utf_bom.html#29

The BOM can also serve as an encoding marker. I refer to the
BOM encoded in UTF-8 as "UTF-8 signature". As such, it is
very meaningful. Usage of the BOM in UTF-8-encoded text
is deliberate.

Regards,
Martin

From martin at v.loewis.de  Mon Oct  2 22:23:35 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Oct 2006 22:23:35 +0200
Subject: [Python-3000] BOM handling
In-Reply-To: <ee9ql8$6e2$1@sea.gmane.org>
References: <1cb725390609111816k7187a15eu91c4029dad65885d@mail.gmail.com>	<u1vfg2dnld13h9qh9pm62c4k032ivt7kic@4ax.com>	<20060913092509.F933.JCARLSON@uci.edu>	<1158179602.4721.24.camel@fsol>
	<ee9ql8$6e2$1@sea.gmane.org>
Message-ID: <45217547.1050808@v.loewis.de>

Georg Brandl schrieb:
>>>>> b = (codecs.BOM_UTF8 + "hello").decode("utf-8")
>>>>> len(a)
>> 5
> 
> This behavior is questionable...

Indeed. Try

py> b = (codecs.BOM_UTF8 + "hello").decode("utf-8-sig")
py> len(b)
5

instead.

Regards,
Martin

From martin at v.loewis.de  Mon Oct  2 22:29:01 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Oct 2006 22:29:01 +0200
Subject: [Python-3000] BOM handling
In-Reply-To: <20060913153900.F936.JCARLSON@uci.edu>
References: <20060913092509.F933.JCARLSON@uci.edu>	<1158179602.4721.24.camel@fsol>
	<20060913153900.F936.JCARLSON@uci.edu>
Message-ID: <4521768D.2090609@v.loewis.de>

Josiah Carlson schrieb:
> I'm unable to find that particular utf-8 codec in the version of Python
> 2.5 I have installed, but I may not be looking in the right places, or
> spelling it the right way.

Try "utf-8-sig" as the encoding name, or encodings/utf_8_sig.py for the
implementation.

Regards,
Martin

From martin at v.loewis.de  Mon Oct  2 22:34:28 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Oct 2006 22:34:28 +0200
Subject: [Python-3000] BOM handling
In-Reply-To: <450997BB.6020703@latte.ca>
References: <45090D11.3060908@acm.org>
	<4509556F.4030508@latte.ca>	<20060914092020.F954.JCARLSON@uci.edu>
	<450997BB.6020703@latte.ca>
Message-ID: <452177D4.2070304@v.loewis.de>

Blake Winton schrieb:
> I don't know if the magic number stuff to determine whether a file is 
> executable or not is bash-specific.  Either way, when I save the file in 
> UTF-8, it's fine, but when I save it in UTF-8 with a BOM, it fails.

It's the operating system that does the interpretation. On Linux, the
magic bytes that are recognized can be extended at run-time, which
make the "utf8script" Debian package possible.

Regards,
Martin


From martin at v.loewis.de  Mon Oct  2 22:32:41 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Oct 2006 22:32:41 +0200
Subject: [Python-3000] BOM handling
In-Reply-To: <4509556F.4030508@latte.ca>
References: <20060913092509.F933.JCARLSON@uci.edu>	<1158179602.4721.24.camel@fsol>	<20060913153900.F936.JCARLSON@uci.edu>	<1158214740.5863.19.camel@fsol>	<45090D11.3060908@acm.org>
	<4509556F.4030508@latte.ca>
Message-ID: <45217769.1010901@v.loewis.de>

Blake Winton schrieb:
> Um, what more data do we need for this use-case?  I'm not going to 
> suggest an API, other than it would be nice if I didn't have to manually 
> figure out/hard code all the encodings.  (It's my belief that I will 
> currently have to do that, or at least special-case XML, to read the 
> encoding attribute.)  Oh, and it would be particularly horrible if I 
> output a shell script in UTF-8, and it included the BOM, since I believe 
> that would break the "magic number" of "#!".
> 
> (To test it in vim, set the following options:
> :set encoding=utf-8
> :set bomb
> )
> 
> Jennifer:~ bwinton$ xxd test
> 0000000: efbb bf23 2120 2f62 696e 2f62 6173 680a  ...#! /bin/bash.
> 0000010: 6563 686f 204a 7573 7420 7465 7374 696e  echo Just testin
> 0000020: 672e 2e2e 0a                             g....
> Jennifer:~ bwinton$ ./test
> -bash: ./test: cannot execute binary file

This works fine in Debian if you have the "utf8script" package
installed.

Regards,
Martin


From martin at v.loewis.de  Tue Oct  3 11:39:59 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Oct 2006 11:39:59 +0200
Subject: [Python-3000] Help on text editors
In-Reply-To: <4500CF8B.6040003@blueyonder.co.uk>
References: <1cb725390609071221x7ffc9d3epc59ea0381369d3c4@mail.gmail.com>
	<4500CF8B.6040003@blueyonder.co.uk>
Message-ID: <45222FEF.3020002@v.loewis.de>

David Hopwood schrieb:
>> If you have access to "German Windows XP", "Japanese Windows XP",
> 
> Since Win2K there is actually no such thing, from a technical point of view --
> just Win2K or WinXP with a German or Japanese "language group" installed,
> and a corresponding locale selected as the interface locale for a given user
> account. The links below should make this clearer.

That's not true. Even though you can change the system code page and the
user locale, other aspects of the installation won't change (such as the
language used in the menus, the language of the help files, and so on).

With W2k, Microsoft introduced MUI (multi-lingual user interfaces), so
you could change the language of the menus and help files at run-time,
in a per-user fashion (this was only available to selected customers
for W2k, and is generally available for WXP).

However, there still are separate products for "English Windows XP",
"German Windows XP", and so on. You can install a MUI pack only on
the English version, and an English version + German MUI is different
from the German version: the program files folder is called
"Program Files" in the English version (and doesn't get renamed when
a MUI package is installed), and is called "Programme" in the German
version.

Regards,
Martin


From martin at v.loewis.de  Tue Oct  3 11:47:42 2006
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Oct 2006 11:47:42 +0200
Subject: [Python-3000] Help on text editors
In-Reply-To: <1157660036.8533.18.camel@fsol>
References: <1cb725390609071221x7ffc9d3epc59ea0381369d3c4@mail.gmail.com>
	<1157660036.8533.18.camel@fsol>
Message-ID: <452231BE.3000604@v.loewis.de>

Antoine Pitrou schrieb:
> Ok, I hexdump'ed a few .mo files (the gettext-compatible files which
> contain translation strings) and the result is a bit funny:
> Gnome/KDE .mo files use utf-8, while .mo files for various command-line
> tools (e.g. aspell) use iso-8859-15.

This is a gettext feature: gettext .mo files contain their encoding,
so any file may use any encoding independent of the user's locale.
At run-time, gettext converts the .mo encoding into the user's encoding.

Typically, the selection of the encoding is the choice of the translator
performing the translation. In Germany, some translators now use UTF-8
for translations, as this supports the typographically correct
?quotation?. gettext will automatically transliterate these if the
locale does not support them.

So you shouldn't use the encoding of .mo files to infer some sort
of "default".

Regards,
Martin

From martin at v.loewis.de  Tue Oct  3 12:29:14 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Oct 2006 12:29:14 +0200
Subject: [Python-3000] string C API
In-Reply-To: <fb6fbf560609151437l714541a9t933e4ef865aaf12b@mail.gmail.com>
References: <450AAAD6.5030901@gmail.com>	<fb6fbf560609150725y79c7b4eaw5300fb256502e482@mail.gmail.com>	<20060915102433.F980.JCARLSON@uci.edu>
	<fb6fbf560609151437l714541a9t933e4ef865aaf12b@mail.gmail.com>
Message-ID: <45223B7A.40003@v.loewis.de>

Jim Jewett schrieb:
>>> Interning may get awkward if multiple encodings are allowed within a
>>> program, regardless of whether they're allowed for single strings.  It
>>> might make sense to intern only strings that are in the same encoding
>>> as the source code.  (Or whose values are limited to ASCII?)
> 
>> Why?  If the text hash function is defined on *code points*, then
>> interning, or really any arbitrary dictionary lookup is the same as it
>> has always been.
> 
> The problem isn't the hash; it is the equality.  Which encoding do you
> keep interned?

Are you using the verb "to intern" here in the sense of the intern()
builtin()? If so: intern the representation of the string that gets
interned first. Python currently interns the entire string object
(not just the character data); I see no reason to change that.

>> What about never recoding?  The benefit of the latin-1/ucs-2/ucs-4
>> method I previously described is that each of the encodings offer a
>> minimal representation of the code points that the text object contains.
> 
> There may be some thrashing as
> 
>     s+= (larger char)

This creates a new string, in any case (remember, strings are not
mutable). Assume the old value A was ucs-2, and (larger char) B is
ucs-4. The code to perform the addition would be

   C = PyString_New(A->ob_size+b->ob_size, UCS4);
   UCS2 *a_data = PyString_AsUCS2(A);
   UCS4 *b_data = PyString_AsUCS4(B);
   UCS4 *c_data = PyString_AsUCS4();
   for(int k=0; k < A->ob_size; k++)
      *c_data++ = *a_data++;
   for(int k = 0; k < B->ob_size; k++)
      *c_data++ = *b_data;
   *c_data = 0;

Notice that this code is independent from whether A and B have
different representations or not.

>     s[:6]

This would require two iterations over the string: one to find
the maximum character, and the second to perform the actual
copying.

> The three options might well be a sensible choice, but I think it
> would already have much of the disadvantage of multiple internal
> encodings, and we might eventually regret any specific limits.  (Why
> not the local 8-bit?  Why not UTF-8, if that is the system encoding?)

Take a look at above code. Here, I never invoke a codec routine (which
would be quite expensive). Instead, I rely on the fact that the
characters have the same numeric values in all three representations.

> It is easy enough to answer why not for each specific case, but I'm
> not *certain* that it is the right answer -- so why not leave it up to
> implementors if they want to do more than the basic three?

Not sure what implementors you are talking about: anybody who wants
to clone Python is free to do whatever they want. We *are* the
implementors of CPython, and if we don't want to do more, then
we just don't want it.

Regards,
Martin

From martin at v.loewis.de  Tue Oct  3 12:34:29 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Oct 2006 12:34:29 +0200
Subject: [Python-3000] sys.stdin and sys.stdout with textfile
In-Reply-To: <45062AD2.1090207@canterbury.ac.nz>
References: <1157898432.4246.161.camel@fsol>	<ca471dc20609101004t2d55b686x4908c39981467106@mail.gmail.com>
	<45062AD2.1090207@canterbury.ac.nz>
Message-ID: <45223CB5.7080102@v.loewis.de>

Greg Ewing schrieb:
>> All sorts of things are different when reading stdin vs. opening a
>> filename. e.g. stdin may be a pipe.
> 
> Which suggests that if anything is going to try
> to guess the encoding, it would be better for it
> to start reading from the actual stream you're
> going to use and buffer the result, rather than
> rely on being able to open it separately.

As Guido says: *all* sorts of things are different. When
stdin is a terminal, there are separate API functions which
we should use to determine the terminal's encoding. This
does neither require to read data from a stream, nor
to open a file. If we read data from a pipe, it is probably
most natural to assume that the system code page/locale
encoding is used for these data.

Regards,
Martin


From jimjjewett at gmail.com  Tue Oct  3 17:13:47 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 3 Oct 2006 11:13:47 -0400
Subject: [Python-3000] string C API
In-Reply-To: <45223B7A.40003@v.loewis.de>
References: <450AAAD6.5030901@gmail.com>
	<fb6fbf560609150725y79c7b4eaw5300fb256502e482@mail.gmail.com>
	<20060915102433.F980.JCARLSON@uci.edu>
	<fb6fbf560609151437l714541a9t933e4ef865aaf12b@mail.gmail.com>
	<45223B7A.40003@v.loewis.de>
Message-ID: <fb6fbf560610030813j10967b59sb9bba5d09b78085d@mail.gmail.com>

On 10/3/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Jim Jewett schrieb:

> > The problem isn't the hash; it is the equality.  Which encoding do you
> > keep interned?

When I wrote this, I had been assuming that UCS4(string) and
UCS2(string) would be completely unrelated objects.  With more
thought, I realized that might not make sense.  Today, we special-case
the (compile-time) internal python encoding and the default encoding.

In python 3, a string object might look like

#define PyObject_str_HEAD   \
    PyObject_VAR_HEAD   \
    long ob_shash;   \
    PyObject *cache;

with a typical concrete implementation looking like

typedef struct {
    PyObject_str_HEAD
    PyObject *encoding   /* concrete method implementation, not just codecs */
    data
} PyAbstractUnicodeObject;

cache is a (weak?) mapping encoding -> this string in that encoding,
so two strings with the same cache pointer are equal.

This particular implementation would

  +  take the same overhead memory as today's unicode (adding cache
and encoding, taking out pointers to str and defenc)
  +  co-locate the header and data (which Guido has indicated is important)

  -  Be slightly worse (by object headers) when exactly two encodings
are needed, and those two are the compile-time internal format and the
default encoding.

  +  Be much better if several encodings are needed, or if recoding is
never needed.

A different concrete implementation for UTF-8 (which admitted it was a
subclass of unicode/str) could save the *encoding field.


> >> What about never recoding?  The benefit of the latin-1/ucs-2/ucs-4
> >> method I previously described is that each of the encodings offer a
> >> minimal representation of the code points that the text object contains.

> > There may be some thrashing as

> >     s+= (larger char)

> [ Martin's size-independent code ]

oops ... I had been thinking about surrogates, but Josiah's proposal
chooses the size explicitly to avoid them.

> > It is easy enough to answer why not for each specific case, but I'm
> > not *certain* that it is the right answer -- so why not leave it up to
> > implementors if they want to do more than the basic three?

> Not sure what implementors you are talking about: anybody who wants
> to clone Python is free to do whatever they want. We *are* the
> implementors of CPython, and if we don't want to do more, then
> we just don't want it.

implementors of concrete string types.

Python is normally pretty good about duck typing, but str is a
notorious exception.

One of the major objections to the Path class was that it had to
inherit from string in order to actually work.  If people used (the
equivalent of) PyUnicode_AsUnicode instead of ->str, this wouldn't
have been so hard.

I also expect that the number of concrete types in the core itself may
increase if it is easy to do that.  I don't think any single person
would care enough to maintain all of UCS4, UCS2, Latin-1, Latin-2,
UTF-8, and NSString versions; it wouldn't surprise me if there were
someone who cared enough to maintain each of those.

-jJ

From martin at v.loewis.de  Tue Oct  3 17:53:15 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Oct 2006 17:53:15 +0200
Subject: [Python-3000] string C API
In-Reply-To: <fb6fbf560610030813j10967b59sb9bba5d09b78085d@mail.gmail.com>
References: <450AAAD6.5030901@gmail.com>	
	<fb6fbf560609150725y79c7b4eaw5300fb256502e482@mail.gmail.com>	
	<20060915102433.F980.JCARLSON@uci.edu>	
	<fb6fbf560609151437l714541a9t933e4ef865aaf12b@mail.gmail.com>	
	<45223B7A.40003@v.loewis.de>
	<fb6fbf560610030813j10967b59sb9bba5d09b78085d@mail.gmail.com>
Message-ID: <4522876B.6030707@v.loewis.de>

Jim Jewett schrieb:
> In python 3, a string object might look like
> 
> #define PyObject_str_HEAD   \
>    PyObject_VAR_HEAD   \
>    long ob_shash;   \
>    PyObject *cache;
> 
> with a typical concrete implementation looking like
> 
> typedef struct {
>    PyObject_str_HEAD
>    PyObject *encoding   /* concrete method implementation, not just
> codecs */
>    data
> } PyAbstractUnicodeObject;

I think Josiah is proposing a different implementation:

typedef struct{
  PyObject_VAR_HEAD
  long ob_shash;
  enum{L1,L2,L4} ob_elemsize;
  ucs4 ob_sval[1]; /* could be interpreted as char* or ucs2* as well */
} PyUnicodeObject;

> implementors of concrete string types.

Why should they care what the Python string type is implemented like?
Few people implement their own string types, and those can just
implement their own type, with no concern whatsoever for the builtin
string type.

> Python is normally pretty good about duck typing, but str is a
> notorious exception.

Non-sense. Python is good about duck typing, period. Just look
at UserString for an example on how to implement a new string
type.

You seem to be talking about polymorphism through inheritance.
Python does not support that well for any of the builtin types;
I do think that all these types should be final (in the Java
sense).

> I also expect that the number of concrete types in the core itself may
> increase if it is easy to do that.  I don't think any single person
> would care enough to maintain all of UCS4, UCS2, Latin-1, Latin-2,
> UTF-8, and NSString versions; it wouldn't surprise me if there were
> someone who cared enough to maintain each of those.

I doubt any kind of "pluggable" representation could work in a
reasonable way. With that generality, you lose any information
as to what the internal representation is, and then code becomes
tedious to write and slow to run.

Regards,
Martin

From jimjjewett at gmail.com  Tue Oct  3 19:41:27 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 3 Oct 2006 13:41:27 -0400
Subject: [Python-3000] string C API
In-Reply-To: <4522876B.6030707@v.loewis.de>
References: <450AAAD6.5030901@gmail.com>
	<fb6fbf560609150725y79c7b4eaw5300fb256502e482@mail.gmail.com>
	<20060915102433.F980.JCARLSON@uci.edu>
	<fb6fbf560609151437l714541a9t933e4ef865aaf12b@mail.gmail.com>
	<45223B7A.40003@v.loewis.de>
	<fb6fbf560610030813j10967b59sb9bba5d09b78085d@mail.gmail.com>
	<4522876B.6030707@v.loewis.de>
Message-ID: <fb6fbf560610031041p61fe5971j9b07281bd55a4019@mail.gmail.com>

On 10/3/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Jim Jewett schrieb:
> > In python 3, a string object might look like

> > #define PyObject_str_HEAD   \
> >    PyObject_VAR_HEAD   \
> >    long ob_shash;   \
> >    PyObject *cache;

> > with a typical concrete implementation looking like

> > typedef struct {
> >    PyObject_str_HEAD
> >    PyObject *encoding   /* concrete method implementation, not just
> > codecs */
> >    data
> > } PyAbstractUnicodeObject;

> I think Josiah is proposing a different implementation:

> typedef struct{
>   PyObject_VAR_HEAD
>   long ob_shash;
>   enum{L1,L2,L4} ob_elemsize;
>   ucs4 ob_sval[1]; /* could be interpreted as char* or ucs2* as well */
> } PyUnicodeObject;

Yes.

By knowing that there are only three possible encodings, he reduces
the encoding pointer into an enum.

By knowing that there is only one possible representation for a given
string, he skips the equivalency cache.  On the other hand, he also
loses the equivalency cache.  When python 2.x chooses the unicode
width, it tries to match tcl; under a "minimal size possible" scheme,
strings that fit in ASCII will have to be recoded twice on every round
trip.  The same problem pops up with other extension modules, and with
system encodings.

By exposing the full object insted of the abstract interface,
compilers can do pointer addition instead of calling a get_data
function.  But they still don't know (until run time) how wide the
data at that pointer will be, and we're locked into binary
compatibility.

> > Python is normally pretty good about duck typing, but str is a
> > notorious exception.

> You seem to be talking about polymorphism through inheritance.

Partly.  But for various reasons, strings seem to cause more problems.
 These would go away if people used the API instead of isinstance,
PyString_Check, and PyString_CheckExact (or just assuming the
results), but they don't -- because the layout is public.

> I doubt any kind of "pluggable" representation could work in a
> reasonable way. With that generality, you lose any information
> as to what the internal representation is, and then code becomes
> tedious to write and slow to run.

Instead of working with ((string)obj).data directly, you work with
string.recode(object, desired)

I see nothing wrong with adding (err ... keeping) convenience methods
to the API for common or recommended encodings, so that obj.data
becomes UCS4(obj)

If you're saying this will be slow because it is a C function call,
then I can't really argue; I just think it will be a good trade for
all the times we don't recode at all (or recode only once/encoding).
I'll admit that I'm not sure what sort of data would make a real-world
(as opposed to contrived) benchmark.

-jJ

From martin at v.loewis.de  Tue Oct  3 21:33:33 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Oct 2006 21:33:33 +0200
Subject: [Python-3000] string C API
In-Reply-To: <fb6fbf560610031041p61fe5971j9b07281bd55a4019@mail.gmail.com>
References: <450AAAD6.5030901@gmail.com>	
	<fb6fbf560609150725y79c7b4eaw5300fb256502e482@mail.gmail.com>	
	<20060915102433.F980.JCARLSON@uci.edu>	
	<fb6fbf560609151437l714541a9t933e4ef865aaf12b@mail.gmail.com>	
	<45223B7A.40003@v.loewis.de>	
	<fb6fbf560610030813j10967b59sb9bba5d09b78085d@mail.gmail.com>	
	<4522876B.6030707@v.loewis.de>
	<fb6fbf560610031041p61fe5971j9b07281bd55a4019@mail.gmail.com>
Message-ID: <4522BB0D.1010006@v.loewis.de>

Jim Jewett schrieb:
> By knowing that there is only one possible representation for a given
> string, he skips the equivalency cache.  On the other hand, he also
> loses the equivalency cache.

What is an equivalency cache, and why would one like to have one?

> When python 2.x chooses the unicode
> width, it tries to match tcl; under a "minimal size possible" scheme,
> strings that fit in ASCII will have to be recoded twice on every round
> trip.  The same problem pops up with other extension modules, and with
> system encodings.

In _tkinter, strings have to be copied *always*, whether they use the
same representation or a different one. Tcl requires strings to be
represented in a TclObj; you cannot pass a Python string object directly
into Tcl. As you have to copy, anyway, it doesn't matter if you do
size conversions in the process.

> By exposing the full object insted of the abstract interface,
> compilers can do pointer addition instead of calling a get_data
> function.  But they still don't know (until run time) how wide the
> data at that pointer will be, and we're locked into binary
> compatibility.

That's not true. The internal representation of objects can and did
change across releases. People have to and will recompile their
extension modules for a new feature release.

>> I doubt any kind of "pluggable" representation could work in a
>> reasonable way. With that generality, you lose any information
>> as to what the internal representation is, and then code becomes
>> tedious to write and slow to run.
> 
> Instead of working with ((string)obj).data directly, you work with
> string.recode(object, desired)

... causing a copy of the data, right? This is expensive.

> If you're saying this will be slow because it is a C function call,
> then I can't really argue; I just think it will be a good trade for
> all the times we don't recode at all (or recode only once/encoding).

It's not the function call that makes it slow. It's the copying of
potentially large string data that a recoding requires. In addition,
for some encodings, the algorithm to do the transformation is
fairly slow.

> I'll admit that I'm not sure what sort of data would make a real-world
> (as opposed to contrived) benchmark.

Any kind of text application will suffer if strings get constantly
recoded.

Regards,
Martin

From jimjjewett at gmail.com  Tue Oct  3 22:29:13 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 3 Oct 2006 16:29:13 -0400
Subject: [Python-3000] string C API
In-Reply-To: <4522BB0D.1010006@v.loewis.de>
References: <450AAAD6.5030901@gmail.com>
	<fb6fbf560609150725y79c7b4eaw5300fb256502e482@mail.gmail.com>
	<20060915102433.F980.JCARLSON@uci.edu>
	<fb6fbf560609151437l714541a9t933e4ef865aaf12b@mail.gmail.com>
	<45223B7A.40003@v.loewis.de>
	<fb6fbf560610030813j10967b59sb9bba5d09b78085d@mail.gmail.com>
	<4522876B.6030707@v.loewis.de>
	<fb6fbf560610031041p61fe5971j9b07281bd55a4019@mail.gmail.com>
	<4522BB0D.1010006@v.loewis.de>
Message-ID: <fb6fbf560610031329q3d0ee560q9d66fc11d3cd197d@mail.gmail.com>

On 10/3/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Jim Jewett schrieb:
> > By knowing that there is only one possible representation for a given
> > string, he skips the equivalency cache.  On the other hand, he also
> > loses the equivalency cache.

> What is an equivalency cache, and why would one like to have one?

Same string, different encoding.

The Py 2.x unicode implementation saves a cached copy of the string
encoded in the default coding, but

    (1) it always creates the UCS4 (or UCS2) encoding, even though it
isn't always needed.
    (2) any 3rd encoding -- not matter how frequent -- requires either
a fresh copy every time, or manual caching.

An equivalency cache would save all input/output encodings that the
string was recoded to/from.  (Possibly only with weak references --
the mapping itself might benefit from tuning based on various
applications.)

Today

    Get a string in Latin-1 (or UTF-8, or ...)
    recode to UCS4 (and throw out the latin-1?)
    process...

    Get another string in Latin-1
    recode it to UC4 (and throw out the latin-1?

    compare them (in UCS4)

    convert first string to UTF-8 for output.
    ...
    reconvert first string to UTF-8 again, because it wasn't saved

With my proposal

    Get a string in Latin-1 (or UTF-8, or ...)
    XXX nope, delay (or skip) recoding
    process...

    Get another string in Latin-1
    XXX delay or skip recoding

    compare them (in original Latin-1).

    convert first string to UTF-8 for output.  (and save this in the
encodings cache)
    ...
    reconvert first string to UTF-8 again -- this time just resend the
earlier copy.



> > By exposing the full object insted of the abstract interface,
> > compilers can do pointer addition instead of calling a get_data
> > function.  But they still don't know (until run time) how wide the
> > data at that pointer will be, and we're locked into binary
> > compatibility.

> That's not true. The internal representation of objects can and did
> change across releases. People have to and will recompile their
> extension modules for a new feature release.

recompile is different from rewrite; if you have said "the data will
be at this location" instead of "you can get a pointer to the data
from this method", you can't really change that later.

> >> I doubt any kind of "pluggable" representation could work in a
> >> reasonable way. With that generality, you lose any information
> >> as to what the internal representation is, and then code becomes
> >> tedious to write and slow to run.

> > Instead of working with ((string)obj).data directly, you work with
> > string.recode(object, desired)

> ... causing a copy of the data, right? This is expensive.

Only if
    (1)  You insist on a specific encoding
    (2)  That encoding is not already available, either as the way it
started, or through the equivalency cache.

> > If you're saying this will be slow because it is a C function call,
> > then I can't really argue; I just think it will be a good trade for
> > all the times we don't recode at all (or recode only once/encoding).

> It's not the function call that makes it slow. It's the copying of
> potentially large string data that a recoding requires. In addition,
> for some encodings, the algorithm to do the transformation is
> fairly slow.

Which is why I would like the equivalency cache to save each of the
UCS4, Latin-1, and UTF-8 byte patterns once they're created, but not
to create any of them until needed.

-jJ

From david.nospam.hopwood at blueyonder.co.uk  Tue Oct  3 23:13:40 2006
From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood)
Date: Tue, 03 Oct 2006 22:13:40 +0100
Subject: [Python-3000] Help on text editors
In-Reply-To: <45222FEF.3020002@v.loewis.de>
References: <1cb725390609071221x7ffc9d3epc59ea0381369d3c4@mail.gmail.com>
	<4500CF8B.6040003@blueyonder.co.uk> <45222FEF.3020002@v.loewis.de>
Message-ID: <4522D284.6050105@blueyonder.co.uk>

Martin v. L?wis wrote:
> David Hopwood schrieb:
> 
>>>If you have access to "German Windows XP", "Japanese Windows XP",
>>
>>Since Win2K there is actually no such thing, from a technical point of view --
>>just Win2K or WinXP with a German or Japanese "language group" installed,
>>and a corresponding locale selected as the interface locale for a given user
>>account. The links below should make this clearer.
> 
> That's not true. Even though you can change the system code page and the
> user locale, other aspects of the installation won't change (such as the
> language used in the menus, the language of the help files, and so on).
> 
> With W2k, Microsoft introduced MUI (multi-lingual user interfaces), so
> you could change the language of the menus and help files at run-time,
> in a per-user fashion (this was only available to selected customers
> for W2k, and is generally available for WXP).

I stand corrected. The MS web site is quite misleading on this point.

> However, there still are separate products for "English Windows XP",
> "German Windows XP", and so on. You can install a MUI pack only on
> the English version, and an English version + German MUI is different
> from the German version: the program files folder is called
> "Program Files" in the English version (and doesn't get renamed when
> a MUI package is installed), and is called "Programme" in the German
> version.

Right, but it is just a registry entry that controls where each user's
program files directory (in the sense of the default location to install
*new* programs) is.

-- 
David Hopwood <david.nospam.hopwood at blueyonder.co.uk>




From jcarlson at uci.edu  Wed Oct  4 00:31:15 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 03 Oct 2006 15:31:15 -0700
Subject: [Python-3000] string C API
In-Reply-To: <fb6fbf560610031329q3d0ee560q9d66fc11d3cd197d@mail.gmail.com>
References: <4522BB0D.1010006@v.loewis.de>
	<fb6fbf560610031329q3d0ee560q9d66fc11d3cd197d@mail.gmail.com>
Message-ID: <20061003151408.0921.JCARLSON@uci.edu>


"Jim Jewett" <jimjjewett at gmail.com> wrote:
> On 10/3/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > Jim Jewett schrieb:
> > > By knowing that there is only one possible representation for a given
> > > string, he skips the equivalency cache.  On the other hand, he also
> > > loses the equivalency cache.
> 
> > What is an equivalency cache, and why would one like to have one?
> 
> Same string, different encoding.
> 
> The Py 2.x unicode implementation saves a cached copy of the string
> encoded in the default coding, but
> 
>     (1) it always creates the UCS4 (or UCS2) encoding, even though it
> isn't always needed.
>     (2) any 3rd encoding -- not matter how frequent -- requires either
> a fresh copy every time, or manual caching.
> 
> An equivalency cache would save all input/output encodings that the
> string was recoded to/from.  (Possibly only with weak references --
> the mapping itself might benefit from tuning based on various
> applications.)

If users don't want to recode, they should save the resulting encoding
to a local or global variable.

I'm personally not terribly concerned about needing to recode text every
time one needs to access Tcl, win32, GTK, or QT APIs.  For a large
portion of the cases, Python does that now, and so far I've not heard
any substantial complaints of "Python is slow when accessing API X".

Whether we choose internal encoding based on content (Latin-1, UCS-2,
UCS-4), or choose a single internal encoding based on a tradeoff of
representation size and access time, I don't think it matters.  Why? The
odds are poor that any encoding we choose will really be the right
internal encoding for more than a handful of cases, so users are going
to need to recode, or write to handle the one (or more) internal
encoding(s) available; which they already do.


 - Josiah


From ironfroggy at gmail.com  Wed Oct  4 11:48:33 2006
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Wed, 4 Oct 2006 05:48:33 -0400
Subject: [Python-3000] Nested Iteration?
Message-ID: <76fd5acf0610040248g6851d54l95617dd7dc397158@mail.gmail.com>

I'm sure this has been brought up before, either for Py3K or in
previous development, so just let me know if its already been
shotdown. Maybe even give it a second thought.

for i in some_list in some_list_of_lists:
    do_something_with(i)

Currently, this is passed iterating over the results of (some_list in
some_list_of_lists), which shouldn't ever reveal an iterable result,
so is there any danger in making the for loops, listcomps, and genexps
take special meaning to this chaining of iterators?

This might even remove a great number of the ever-repeating "Why isn't
there a standard flatten function?" debates, because it would provide
a simple, intuitive, understandable base for most any way you would
want to flatten things.

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From ncoghlan at gmail.com  Wed Oct  4 13:04:44 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Oct 2006 21:04:44 +1000
Subject: [Python-3000] Nested Iteration?
In-Reply-To: <76fd5acf0610040248g6851d54l95617dd7dc397158@mail.gmail.com>
References: <76fd5acf0610040248g6851d54l95617dd7dc397158@mail.gmail.com>
Message-ID: <4523954C.1010705@gmail.com>

Calvin Spealman wrote:
> I'm sure this has been brought up before, either for Py3K or in
> previous development, so just let me know if its already been
> shotdown. Maybe even give it a second thought.
> 
> for i in some_list in some_list_of_lists:
>     do_something_with(i)
> 
> Currently, this is passed iterating over the results of (some_list in
> some_list_of_lists), which shouldn't ever reveal an iterable result,
> so is there any danger in making the for loops, listcomps, and genexps
> take special meaning to this chaining of iterators?
> 
> This might even remove a great number of the ever-repeating "Why isn't
> there a standard flatten function?" debates, because it would provide
> a simple, intuitive, understandable base for most any way you would
> want to flatten things.

Unfortunately this "simple, intuitive, understandable base" is wrong for the 
same reason most naive flattening functions are wrong - flattening operations 
typically want to treat strings as atomic data rather than as a container 
(although a previous experiment with making strings not behave like a 
container at all for Py3k showed that to be even *more* inconvenient).

If you really want something like this, you can already use a genexp to 
similar effect:

  for i in (some_list for some_list in some_list_of_lists):
      do_something_with(i)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Wed Oct  4 16:29:41 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Oct 2006 07:29:41 -0700
Subject: [Python-3000] Nested Iteration?
In-Reply-To: <76fd5acf0610040248g6851d54l95617dd7dc397158@mail.gmail.com>
References: <76fd5acf0610040248g6851d54l95617dd7dc397158@mail.gmail.com>
Message-ID: <ca471dc20610040729n305387c2sb97040f9f5f28fb3@mail.gmail.com>

Let me nip this in the bud with a healthy -1000. It would forever
destroy clarity.

On 10/4/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
> I'm sure this has been brought up before, either for Py3K or in
> previous development, so just let me know if its already been
> shotdown. Maybe even give it a second thought.
>
> for i in some_list in some_list_of_lists:
>     do_something_with(i)
>
> Currently, this is passed iterating over the results of (some_list in
> some_list_of_lists), which shouldn't ever reveal an iterable result,
> so is there any danger in making the for loops, listcomps, and genexps
> take special meaning to this chaining of iterators?
>
> This might even remove a great number of the ever-repeating "Why isn't
> there a standard flatten function?" debates, because it would provide
> a simple, intuitive, understandable base for most any way you would
> want to flatten things.
>
> --
> Read my blog! I depend on your acceptance of my opinion! I am interesting!
> http://ironfroggy-code.blogspot.com/
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Thu Oct  5 07:53:01 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Oct 2006 07:53:01 +0200
Subject: [Python-3000] Help on text editors
In-Reply-To: <4522D284.6050105@blueyonder.co.uk>
References: <1cb725390609071221x7ffc9d3epc59ea0381369d3c4@mail.gmail.com>	<4500CF8B.6040003@blueyonder.co.uk>
	<45222FEF.3020002@v.loewis.de> <4522D284.6050105@blueyonder.co.uk>
Message-ID: <45249DBD.5010901@v.loewis.de>

David Hopwood schrieb:
>> However, there still are separate products for "English Windows XP",
>> "German Windows XP", and so on. You can install a MUI pack only on
>> the English version, and an English version + German MUI is different
>> from the German version: the program files folder is called
>> "Program Files" in the English version (and doesn't get renamed when
>> a MUI package is installed), and is called "Programme" in the German
>> version.
> 
> Right, but it is just a registry entry that controls where each user's
> program files directory (in the sense of the default location to install
> *new* programs) is.

Unfortunately, you can't really change it after the installation. You
can change it, but that will affect only new installations. Things
that went into Program Files during initial system setup will stay
there, and can't be moved easily elsewhere.

In any case, my point is that this is hard-coded into the installation
disk in a manner that you can't localize at all, with any supported
procedure. OSX got a step beyond that: While the folder on disk is
always called "Applications", it is rendered with a directory name
of "Programme" if you use the German localization, in the OSX Finder.
While it solves the problem, it is very confusing if you use shell-level
interaction (the shell won't accept "Programme" as a name for that
folder).

Regards,
Martin


From greg.ewing at canterbury.ac.nz  Fri Oct  6 02:54:21 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Oct 2006 13:54:21 +1300
Subject: [Python-3000] Help on text editors
In-Reply-To: <45249DBD.5010901@v.loewis.de>
References: <1cb725390609071221x7ffc9d3epc59ea0381369d3c4@mail.gmail.com>
	<4500CF8B.6040003@blueyonder.co.uk> <45222FEF.3020002@v.loewis.de>
	<4522D284.6050105@blueyonder.co.uk> <45249DBD.5010901@v.loewis.de>
Message-ID: <4525A93D.50609@canterbury.ac.nz>

Martin v. L?wis wrote:
> OSX got a step beyond that: While the folder on disk is
> always called "Applications", it is rendered with a directory name
> of "Programme" if you use the German localization, in the OSX Finder.
> While it solves the problem, it is very confusing if you use shell-level
> interaction

I think this was actually handled better in classic MacOS.
Special folders like System, Applications etc. weren't
located by name at all, but by a different mechanism.
So you could name them anything you wanted.

--
Greg

From ntoronto at cs.byu.edu  Mon Oct  9 10:04:11 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 09 Oct 2006 02:04:11 -0600
Subject: [Python-3000] Sky pie: a "var" keyword
Message-ID: <452A027B.8060009@cs.byu.edu>

As long as patches are flying willy-nilly and we're just kicking around 
ideas like puppies, I thought I'd offer one of my own.

No, not a puppy. An idea. Of course, if it ends up beaten to death, I 
might rather it were a puppy. My apologies if it's been discussed in 
some other form.

It occurs to me that some of Python's gotchas are due to the compiler 
not being able to determine whether the user intends to *create* a 
variable, or *modify* one.

The "global" gotcha:

x = 0
def f():
    print x   # 'local variable 'x' referenced before assignment'
    x = 3


The "read-only lexical scoping" gotcha:

def f():
    x = 0
    def g():
        x = 3   # doesn't alter outer 'x'


And the "changing a writable property name is a very bad plan" gotcha. 
This bit me severely when I was first learning Python. I changed an 
attribute name from "x" to "value" in some Bayesian network code, and 
spent hours trying to determine why it wasn't working anymore. Turns out 
I missed a "node.x = " somewhere. Nowadays, I'd write a dummy property 
named "x" that throws an exception, but that seems an extraordinarily 
hackish thing to do to support an operation that happens a lot.

A "var" keyword fixes them all. The "global" gotcha:

x = 0
def f():
    print x   # no exception - prints "0"
    x = 3

def g():
    print x   # exception
    var x = 3


The read-only lexical scoping gotcha:

def f():
    var x = 0
    def g():
        x = 3   # alters outer 'x'


And when the runtime encountered "node.x = " it would have complained 
that the variable didn't exist. The __init__ method would look like this:

def __init__(self, stuff):
   var self.stuff = stuff
   var self.x = 0
   # ...


If you don't like how it looks, we could call it "eric" instead:

   eric self.x = 0


Of course, "var" isn't a swear word in any language (as far as I know), 
but "eric" is heinously rude in Swahili.

Essentially, "var" means "make a new entry in the dictionary."

Some Zen arguments:

Explicit is better than implicit (are you creating? are you modifying?)
Complex is worse than simple (it's slightly more complex), but complex 
is better than complicated (which the scoping semantics are now)

It also has nice duality with the "del" keyword. Same number of letters 
and everything:

var x = 0
del x


If type annotations ever got into Py3k, they'd go in very naturally. 
This might also enable some compiler optimizations.

The biggest downside I can see is that it would break every scrap of 
Python code ever written.

Another puppy to kick around: If that's totally unacceptable, how about 
making the default for assignment act on a variable in the outer scope 
if one exists, and have "var" override? This would allow read/write 
lexical scoping and put the "this is meant to be new in this scope" 
decoration at the point of declaration. (I also think "this is meant to 
be new" is much cleaner than "this is meant to be old," but that's just me.)

Grovelingly,
Neil


From garys at ihug.com.au  Mon Oct  9 11:34:26 2006
From: garys at ihug.com.au (Gary Stephenson)
Date: Mon, 9 Oct 2006 19:34:26 +1000
Subject: [Python-3000] Sky pie: a "var" keyword
References: <452A027B.8060009@cs.byu.edu>
Message-ID: <001b01c6eb86$1e47e110$0301010a@VIMES>


----- Original Message ----- 
From: "Neil Toronto" <ntoronto at cs.byu.edu>
To: <python-3000 at python.org>
Sent: Monday, October 09, 2006 6:04 PM
Subject: [Python-3000] Sky pie: a "var" keyword


> As long as patches are flying willy-nilly and we're just kicking around
> ideas like puppies, I thought I'd offer one of my own.
>
> No, not a puppy. An idea. Of course, if it ends up beaten to death, I
> might rather it were a puppy. My apologies if it's been discussed in
> some other form.
>
> It occurs to me that some of Python's gotchas are due to the compiler
> not being able to determine whether the user intends to *create* a
> variable, or *modify* one.
>
> The "global" gotcha:
>
> x = 0
> def f():
>    print x   # 'local variable 'x' referenced before assignment'
>    x = 3
>
>
> The "read-only lexical scoping" gotcha:
>
> def f():
>    x = 0
>    def g():
>        x = 3   # doesn't alter outer 'x'
>
>
> And the "changing a writable property name is a very bad plan" gotcha.
> This bit me severely when I was first learning Python. I changed an
> attribute name from "x" to "value" in some Bayesian network code, and
> spent hours trying to determine why it wasn't working anymore. Turns out
> I missed a "node.x = " somewhere. Nowadays, I'd write a dummy property
> named "x" that throws an exception, but that seems an extraordinarily
> hackish thing to do to support an operation that happens a lot.
>
> A "var" keyword fixes them all. The "global" gotcha:
>
> x = 0
> def f():
>    print x   # no exception - prints "0"
>    x = 3
>
> def g():
>    print x   # exception
>    var x = 3
>
>
> The read-only lexical scoping gotcha:
>
> def f():
>    var x = 0
>    def g():
>        x = 3   # alters outer 'x'
>
>
> And when the runtime encountered "node.x = " it would have complained
> that the variable didn't exist. The __init__ method would look like this:
>
> def __init__(self, stuff):
>   var self.stuff = stuff
>   var self.x = 0
>   # ...
>
>
> If you don't like how it looks, we could call it "eric" instead:
>
>   eric self.x = 0
>
>
> Of course, "var" isn't a swear word in any language (as far as I know),
> but "eric" is heinously rude in Swahili.
>
> Essentially, "var" means "make a new entry in the dictionary."
>
> Some Zen arguments:
>
> Explicit is better than implicit (are you creating? are you modifying?)
> Complex is worse than simple (it's slightly more complex), but complex
> is better than complicated (which the scoping semantics are now)
>
> It also has nice duality with the "del" keyword. Same number of letters
> and everything:
>
> var x = 0
> del x
>
>
> If type annotations ever got into Py3k, they'd go in very naturally.
> This might also enable some compiler optimizations.
>
> The biggest downside I can see is that it would break every scrap of
> Python code ever written.
>
> Another puppy to kick around: If that's totally unacceptable, how about
> making the default for assignment act on a variable in the outer scope
> if one exists, and have "var" override? This would allow read/write
> lexical scoping and put the "this is meant to be new in this scope"
> decoration at the point of declaration. (I also think "this is meant to
> be new" is much cleaner than "this is meant to be old," but that's just 
> me.)
>
> Grovelingly,
> Neil

Fwiw, (which is not very much, given that I've never been an active 
Pythonista), I strongly support this idea - most particularly for the 
"read-only lexical scoping gotcha", which is to my eye the ugliest by far of 
Python's warts.  I'd prefer it be called "local" though (not "var" and 
definitely not "eric"), and only apply to local variables of functions - and 
not to instance variables of objects (which are already (sort of) covered 
by the __slots__  thingy).

However, I'm sure the second last paragraph tells the story of why it ain't 
never gonna happen. I imagine it would need  a fork to a completely new 
language.   And I don't think the idea outlined in the last paragraph is 
worthy of support at all.

A pity though - it would greatly improve the Python Language imho,

gary



From qrczak at knm.org.pl  Mon Oct  9 11:56:08 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Mon, 09 Oct 2006 11:56:08 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452A027B.8060009@cs.byu.edu> (Neil Toronto's message of "Mon,
	09 Oct 2006 02:04:11 -0600")
References: <452A027B.8060009@cs.byu.edu>
Message-ID: <87bqol7qyv.fsf@qrnik.zagroda>

Neil Toronto <ntoronto at cs.byu.edu> writes:

> It occurs to me that some of Python's gotchas are due to the compiler 
> not being able to determine whether the user intends to *create* a 
> variable, or *modify* one.

I agree. I've always known this.

I'm only afraid that some people are too accustomed to the current
design.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From fredrik at pythonware.com  Mon Oct  9 12:05:01 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 9 Oct 2006 12:05:01 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
References: <452A027B.8060009@cs.byu.edu>
Message-ID: <egd6sd$tq9$1@sea.gmane.org>

Neil Toronto wrote:

> A "var" keyword fixes them all. The "global" gotcha:
>
> x = 0
> def f():
>    print x   # no exception - prints "0"
>    x = 3
>
> def g():
>    print x   # exception
>    var x = 3

what are the exact semantics of "var" ?  are you saying that local variables
should be global by default?

</F> 




From g.brandl at gmx.net  Mon Oct  9 12:49:21 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 09 Oct 2006 12:49:21 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <egd6sd$tq9$1@sea.gmane.org>
References: <452A027B.8060009@cs.byu.edu> <egd6sd$tq9$1@sea.gmane.org>
Message-ID: <egd9fh$689$1@sea.gmane.org>

Fredrik Lundh wrote:
> Neil Toronto wrote:
> 
>> A "var" keyword fixes them all. The "global" gotcha:
>>
>> x = 0
>> def f():
>>    print x   # no exception - prints "0"
>>    x = 3
>>
>> def g():
>>    print x   # exception
>>    var x = 3
> 
> what are the exact semantics of "var" ?  are you saying that local variables
> should be global by default?

They wouldn't be "local" unless declared with "var", I assume.

Georg


From jimjjewett at gmail.com  Mon Oct  9 15:46:51 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 9 Oct 2006 09:46:51 -0400
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <001b01c6eb86$1e47e110$0301010a@VIMES>
References: <452A027B.8060009@cs.byu.edu>
	<001b01c6eb86$1e47e110$0301010a@VIMES>
Message-ID: <fb6fbf560610090646id0d418ak1c8ab6d28dbdb228@mail.gmail.com>

"Neil Toronto" <ntoronto at cs.byu.edu> wrote:

> It occurs to me that some of Python's gotchas are due to the compiler
> not being able to determine whether the user intends to *create* a
> variable, or *modify* one.

Yes, but there may not be any solution.  One of python's greatest
strengths is the lack of boilerplate.  I doubt there is any way to
indicate this intent without an explicit notation that either looks
magic or becomes boilerplate.

> The "global" gotcha:

> x = 0
> def f():
>    print x   # 'local variable 'x' referenced before assignment'
>    x = 3

Because it is ambiguous.  The print indicates that you want (read)
access to an existing variable, but the assignment indicates that you
want (write) access to that variable.  Did you mean

    x = 0
    def f():
        global x  # modify the existing variable
        print x
        x = 3
or
    x = 0
    def f():
        x = x  # shadow the existing variable
        print x
        x = 3


> A "var" keyword fixes them all. The "global" gotcha:

> x = 0
> def f():
>    print x   # no exception - prints "0"
>    x = 3

But which of the above would this mean?

Below, it sounds like you're adding an implicit global.  In Lisp, this
is somewhat controlled by first declaring the variable special (at the
top level) and then using a conventionally strange name on top of it.
If you could create top-level variables easily (as it looks like from
the x=0 line), people would get nasty suprises about what turned out
to be top-level -- and what changed from local to toplevel after later
code ran.

> The read-only lexical scoping gotcha:

> def f():
>    var x = 0
>    def g():
>        x = 3   # alters outer 'x'

For people coming from another language, "var" isn't likely to mean
"by the way, this gets modified elsewhere".  The existing idiom of

    def f():
        x = [0]
        def g():
            x[0] = 3

is at least strange enough to warn people that they need to read carefully.

-jJ

From qrczak at knm.org.pl  Mon Oct  9 16:27:45 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Mon, 09 Oct 2006 16:27:45 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <fb6fbf560610090646id0d418ak1c8ab6d28dbdb228@mail.gmail.com> (Jim
	Jewett's message of "Mon, 9 Oct 2006 09:46:51 -0400")
References: <452A027B.8060009@cs.byu.edu>
	<001b01c6eb86$1e47e110$0301010a@VIMES>
	<fb6fbf560610090646id0d418ak1c8ab6d28dbdb228@mail.gmail.com>
Message-ID: <87vemtva1q.fsf@qrnik.zagroda>

"Jim Jewett" <jimjjewett at gmail.com> writes:

>> A "var" keyword fixes them all. The "global" gotcha:
>
>> x = 0
>> def f():
>>    print x   # no exception - prints "0"
>>    x = 3
>
> But which of the above would this mean?

It should be:

var x = 0
def f():
   print x
   x = 3

for the global variable, and:

var x = 0
def f():
   var x
   print x
   x = 3

for the local variable.

There are a few sane choices for the semantics of 'var':

1. The variable is visible from 'var' to the end of the scope.
   If 'var' doesn't specify an initial value, accessing the variable
   before it gets assigned to is an error.

2. The variable is visible from 'var' to the end of the scope.
   If 'var' doesn't specify an initial value, 'None' is used.

3. The variable is visible from the beginning to the end of the scope.
   Accessing the variable before the 'var' is executed is an error.
   If 'var' doesn't specify an initial value, 'None' is used.

> For people coming from another language, "var" isn't likely to mean
> "by the way, this gets modified elsewhere".

This aspect is perfectly clear for me. Why would it be disallowed
to be modified elsewhere?

> The existing idiom of
>
>     def f():
>         x = [0]
>         def g():
>             x[0] = 3
>
> is at least strange enough to warn people that they need to read carefully.

This idiom is a silly workaround for the lack of explicit variable
definitions, and thus for the inability of modifying variables defined
in outer scopes.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From p.f.moore at gmail.com  Mon Oct  9 16:47:55 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 9 Oct 2006 15:47:55 +0100
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <87vemtva1q.fsf@qrnik.zagroda>
References: <452A027B.8060009@cs.byu.edu>
	<001b01c6eb86$1e47e110$0301010a@VIMES>
	<fb6fbf560610090646id0d418ak1c8ab6d28dbdb228@mail.gmail.com>
	<87vemtva1q.fsf@qrnik.zagroda>
Message-ID: <79990c6b0610090747i242793d0ia25b3a898329bbc7@mail.gmail.com>

On 10/9/06, Marcin 'Qrczak' Kowalczyk <qrczak at knm.org.pl> wrote:
> There are a few sane choices for the semantics of 'var':
>
> 1. The variable is visible from 'var' to the end of the scope.
>    If 'var' doesn't specify an initial value, accessing the variable
>    before it gets assigned to is an error.
>
> 2. The variable is visible from 'var' to the end of the scope.
>    If 'var' doesn't specify an initial value, 'None' is used.
>
> 3. The variable is visible from the beginning to the end of the scope.
>    Accessing the variable before the 'var' is executed is an error.
>    If 'var' doesn't specify an initial value, 'None' is used.

Of these, only (3) specifies what happens if the variable is
referenced before the 'var' statement. And that one makes 'var x' mean
exactly the same as 'x = None'.

None of them make it clear what the effect of 'var' is in a context
where an outer (or global) definition of the variable is available.

I don't see any way in which the proposed statement is any less
confusing than the current semantics - it's just confusing in a
different way.

So, on the basis that Py3K shouldn't be breaking code unless there's a
good justification[1], this proposal (in any form) gets a solid -1
from me.

Paul.

[1] My interpretation of PEP 3002 is that Py3K, although it is allowed
to break backward compatibility, should not do so arbitrarily, or
without a clear and relatively strong justification.

From qrczak at knm.org.pl  Mon Oct  9 17:01:16 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Mon, 09 Oct 2006 17:01:16 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <79990c6b0610090747i242793d0ia25b3a898329bbc7@mail.gmail.com>
	(Paul Moore's message of "Mon, 9 Oct 2006 15:47:55 +0100")
References: <452A027B.8060009@cs.byu.edu>
	<001b01c6eb86$1e47e110$0301010a@VIMES>
	<fb6fbf560610090646id0d418ak1c8ab6d28dbdb228@mail.gmail.com>
	<87vemtva1q.fsf@qrnik.zagroda>
	<79990c6b0610090747i242793d0ia25b3a898329bbc7@mail.gmail.com>
Message-ID: <87y7rp1qkj.fsf@qrnik.zagroda>

"Paul Moore" <p.f.moore at gmail.com> writes:

>> 1. The variable is visible from 'var' to the end of the scope.
>>    If 'var' doesn't specify an initial value, accessing the variable
>>    before it gets assigned to is an error.
>>
>> 2. The variable is visible from 'var' to the end of the scope.
>>    If 'var' doesn't specify an initial value, 'None' is used.
>>
>> 3. The variable is visible from the beginning to the end of the scope.
>>    Accessing the variable before the 'var' is executed is an error.
>>    If 'var' doesn't specify an initial value, 'None' is used.
>
> Of these, only (3) specifies what happens if the variable is
> referenced before the 'var' statement.

In (1) and (2) an outer meaning of the variable is used.

> None of them make it clear what the effect of 'var' is in a context
> where an outer (or global) definition of the variable is available.

In all variants a new variable is created, unrelated to the outer
variable.

> I don't see any way in which the proposed statement is any less
> confusing than the current semantics - it's just confusing in a
> different way.

Creating a new variable with the given initial value, and modifying
the value of an existing variable, are different concepts. The
proposal distinguishes them, and doesn't need a 'global' declaration.
A variable is created when a 'var' statement is executed.

Current Python doesn't make very clear when a variable is created.
A global variable is created the first time it is assigned to outside
any function without a 'global' declaration. A local variable is
created at the beginning of the innermost scope which contains an
assignment statement to that variable name. This is implicit and
more complex.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From fredrik at pythonware.com  Mon Oct  9 17:14:02 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 09 Oct 2006 17:14:02 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <87vemtva1q.fsf@qrnik.zagroda>
References: <452A027B.8060009@cs.byu.edu>	<001b01c6eb86$1e47e110$0301010a@VIMES>	<fb6fbf560610090646id0d418ak1c8ab6d28dbdb228@mail.gmail.com>
	<87vemtva1q.fsf@qrnik.zagroda>
Message-ID: <egdovq$vep$1@sea.gmane.org>

Marcin 'Qrczak' Kowalczyk wrote:

> It should be:
> 
> var x = 0
> def f():
>    print x
>    x = 3
> 
> for the global variable, and:
> 
> var x = 0
> def f():
>    var x
>    print x
>    x = 3
> 
> for the local variable.

so the goal of this exercise is to make it easier to introduce 
hard-to-spot errors by accident?

</F>


From ntoronto at cs.byu.edu  Mon Oct  9 19:58:51 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 09 Oct 2006 11:58:51 -0600
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <egd6sd$tq9$1@sea.gmane.org>
References: <452A027B.8060009@cs.byu.edu> <egd6sd$tq9$1@sea.gmane.org>
Message-ID: <452A8DDB.8010407@cs.byu.edu>

Fredrik Lundh wrote:
> Neil Toronto wrote:
>
>   
>> A "var" keyword fixes them all. The "global" gotcha:
>>
>> x = 0
>> def f():
>>    print x   # no exception - prints "0"
>>    x = 3
>>
>> def g():
>>    print x   # exception
>>    var x = 3
>>     
>
> what are the exact semantics of "var" ?  are you saying that local variables
> should be global by default?
>   

Heck no. An assignment would modify an existing variable only. A "var" 
in front of a variable name would put the variable in the local dictionary.

Also, the example needs "var x = 0" in the global scope - that was a 
mistake on my part.

Neil


From ntoronto at cs.byu.edu  Mon Oct  9 20:06:07 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 09 Oct 2006 12:06:07 -0600
Subject: [Python-3000] Sky pie: a "var" keyword
Message-ID: <452A8F8F.2010204@cs.byu.edu>

Marcin 'Qrczak' Kowalczyk wrote:
>   It should be:
>
> var x = 0
> def f():
>    print x
>    x = 3
>
> for the global variable, and:
>
> var x = 0
> def f():
>    var x
>    print x
>    x = 3
>
> for the local variable.
>
> There are a few sane choices for the semantics of 'var':
>
> 1. The variable is visible from 'var' to the end of the scope.
>    If 'var' doesn't specify an initial value, accessing the variable
>    before it gets assigned to is an error.
>
> 2. The variable is visible from 'var' to the end of the scope.
>    If 'var' doesn't specify an initial value, 'None' is used.
>
> 3. The variable is visible from the beginning to the end of the scope.
>    Accessing the variable before the 'var' is executed is an error.
>    If 'var' doesn't specify an initial value, 'None' is used.
>   

For the record, though I didn't present it this way in my initial 
proposal, I like #1 and #2 better than #3. Otherwise, you'd get this 
silliness:

def f():
   x = 3   # fine, because the magic 'var' down below creates it
   var x = 0


Neil

From fredrik at pythonware.com  Mon Oct  9 20:02:58 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 09 Oct 2006 20:02:58 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452A8DDB.8010407@cs.byu.edu>
References: <452A027B.8060009@cs.byu.edu> <egd6sd$tq9$1@sea.gmane.org>
	<452A8DDB.8010407@cs.byu.edu>
Message-ID: <ege2si$bej$1@sea.gmane.org>

Neil Toronto wrote:

> Heck no. An assignment would modify an existing variable only.

what's an "existing variable"?  how would STORE_GLOBAL work under your 
proposal?

</F>


From ntoronto at cs.byu.edu  Mon Oct  9 20:12:50 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 09 Oct 2006 12:12:50 -0600
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <ege2si$bej$1@sea.gmane.org>
References: <452A027B.8060009@cs.byu.edu>
	<egd6sd$tq9$1@sea.gmane.org>	<452A8DDB.8010407@cs.byu.edu>
	<ege2si$bej$1@sea.gmane.org>
Message-ID: <452A9122.6070904@cs.byu.edu>

Fredrik Lundh wrote:
> Neil Toronto wrote:
>
>   
>> Heck no. An assignment would modify an existing variable only.
>>     
>
> what's an "existing variable"?  how would STORE_GLOBAL work under your 
> proposal?
>   

I haven't got a clue what STORE_GLOBAL is, apart from the fact that it 
looks like it has something to do with Python's bytecode.

If it's what I think it is (so don't hold me to this), it'd raise an 
exception if you used on something that hadn't been created yet. 
Probably need a MAKE_GLOBAL in addition to it, or keep STORE_GLOBAL as 
it is (for "var x = 0") and add MODIFY_GLOBAL that raises an exception 
on uncreated variables.

Or something. Again, I'm guessing, and you have me at a disadvantage. 
Can we keep the discussion above the bytecode level for now?

Neil


From ntoronto at cs.byu.edu  Mon Oct  9 20:15:30 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 09 Oct 2006 12:15:30 -0600
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452A8F8F.2010204@cs.byu.edu>
References: <452A8F8F.2010204@cs.byu.edu>
Message-ID: <452A91C2.2080907@cs.byu.edu>

Neil Toronto wrote:
> Marcin 'Qrczak' Kowalczyk wrote:
>   
>>   It should be:
>>
>> var x = 0
>> def f():
>>    print x
>>    x = 3
>>
>> for the global variable, and:
>>
>> var x = 0
>> def f():
>>    var x
>>    print x
>>    x = 3
>>
>> for the local variable.
>>
>> There are a few sane choices for the semantics of 'var':
>>
>> 1. The variable is visible from 'var' to the end of the scope.
>>    If 'var' doesn't specify an initial value, accessing the variable
>>    before it gets assigned to is an error.
>>
>> 2. The variable is visible from 'var' to the end of the scope.
>>    If 'var' doesn't specify an initial value, 'None' is used.
>>
>> 3. The variable is visible from the beginning to the end of the scope.
>>    Accessing the variable before the 'var' is executed is an error.
>>    If 'var' doesn't specify an initial value, 'None' is used.
>>   
>>     
>
> For the record, though I didn't present it this way in my initial 
> proposal, I like #1 and #2 better than #3. Otherwise, you'd get this 
> silliness:
>
> def f():
>    x = 3   # fine, because the magic 'var' down below creates it
>    var x = 0
>
>   

On the other hand...

var x = 0
def f():
    if sometest:
        var x = 3

    x = 2

which is seriously evil, because it's not clear which 'x' the last 
assignment refers to. Maybe merely seeing "var" in a scope makes it a 
local, but using it before the "var" is an error.

The other option is making nested blocks have their own scope, but I 
don't think Python wants to go there.

Neil


From solipsis at pitrou.net  Mon Oct  9 20:40:24 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 09 Oct 2006 20:40:24 +0200
Subject: [Python-3000] ugly pie: a "var" keyword
In-Reply-To: <egd9fh$689$1@sea.gmane.org>
References: <452A027B.8060009@cs.byu.edu> <egd6sd$tq9$1@sea.gmane.org>
	<egd9fh$689$1@sea.gmane.org>
Message-ID: <1160419224.7838.13.camel@fsol>


Le lundi 09 octobre 2006 ? 12:49 +0200, Georg Brandl a ?crit :
> They wouldn't be "local" unless declared with "var", I assume.

Which is a sure way to cause all kind of frigging bugs if you forget to
put "var", because then your variable is global, which is fine when it's
a plain int or string, not when it's a complex object whose mere
existence changes lots of things.

Javascript and Perl both suffer from this stupid design (Perl even
invented "use strict" to circumvent it), please don't bring it to
Python :-(

(believe me, there are lots of reasons to hate Perl, but strange bugs
that arise silently when you forgot "my" and thus retain some old value
in a global variable is really one of the most horrible ones I
encountered)


Besides, in a well-written program, you will mostly access local
variables (and attributes and methods), so let's make the clean common
case easy to type and unobstrusive, and the rare dirty case easy to
spot. Lots of "global" in a Python program immediately hint at dirty
code.

Regards

Antoine.



From qrczak at knm.org.pl  Mon Oct  9 20:43:19 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Mon, 09 Oct 2006 20:43:19 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452A8F8F.2010204@cs.byu.edu> (Neil Toronto's message of "Mon,
	09 Oct 2006 12:06:07 -0600")
References: <452A8F8F.2010204@cs.byu.edu>
Message-ID: <87slhxib3s.fsf@qrnik.zagroda>

Neil Toronto <ntoronto at cs.byu.edu> writes:

> For the record, though I didn't present it this way in my initial 
> proposal, I like #1 and #2 better than #3. Otherwise, you'd get this 
> silliness:
>
> def f():
>    x = 3   # fine, because the magic 'var' down below creates it
>    var x = 0

In my semantics #3 this would be a runtime error at the assignment
(the compiler could catch it statically in this case, but not always).


Neil Toronto <ntoronto at cs.byu.edu> writes:

> var x = 0
> def f():
>     if sometest:
>         var x = 3
>
>     x = 2
>
> which is seriously evil, because it's not clear which 'x' the last 
> assignment refers to.

Should be:

var x = 0
def f():
   var x
   if sometest:
      x = 3
   x = 2

For 'for' loops there are two choices:

1. 'for x in coll' creates x which is local to the loop.

2. 'for var x in coll' creates a local x, 'for x in coll'
   modifies an existing x.

I admit that Python choices of expressing certain constructs
(conditionals, loops) as statements instead of expressions
make a sane semantics of variable declarations less convenient.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From g.brandl at gmx.net  Mon Oct  9 20:45:12 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 09 Oct 2006 20:45:12 +0200
Subject: [Python-3000] ugly pie: a "var" keyword
In-Reply-To: <1160419224.7838.13.camel@fsol>
References: <452A027B.8060009@cs.byu.edu>
	<egd6sd$tq9$1@sea.gmane.org>	<egd9fh$689$1@sea.gmane.org>
	<1160419224.7838.13.camel@fsol>
Message-ID: <ege5bo$l9b$1@sea.gmane.org>

Antoine Pitrou wrote:
> Le lundi 09 octobre 2006 ? 12:49 +0200, Georg Brandl a ?crit :
>> They wouldn't be "local" unless declared with "var", I assume.
> 
> Which is a sure way to cause all kind of frigging bugs if you forget to
> put "var", because then your variable is global, which is fine when it's
> a plain int or string, not when it's a complex object whose mere
> existence changes lots of things.
> 
> Javascript and Perl both suffer from this stupid design (Perl even
> invented "use strict" to circumvent it), please don't bring it to
> Python :-(

No contradiction from my side.

> (believe me, there are lots of reasons to hate Perl, but strange bugs
> that arise silently when you forgot "my" and thus retain some old value
> in a global variable is really one of the most horrible ones I
> encountered)
> 
> 
> Besides, in a well-written program, you will mostly access local
> variables (and attributes and methods), so let's make the clean common
> case easy to type and unobstrusive, and the rare dirty case easy to
> spot. Lots of "global" in a Python program immediately hint at dirty
> code.

Full ACK.

Georg


From qrczak at knm.org.pl  Mon Oct  9 20:49:38 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Mon, 09 Oct 2006 20:49:38 +0200
Subject: [Python-3000] ugly pie: a "var" keyword
In-Reply-To: <1160419224.7838.13.camel@fsol> (Antoine Pitrou's message of
	"Mon, 09 Oct 2006 20:40:24 +0200")
References: <452A027B.8060009@cs.byu.edu> <egd6sd$tq9$1@sea.gmane.org>
	<egd9fh$689$1@sea.gmane.org> <1160419224.7838.13.camel@fsol>
Message-ID: <87wt79z5ml.fsf@qrnik.zagroda>

Antoine Pitrou <solipsis at pitrou.net> writes:

> Which is a sure way to cause all kind of frigging bugs if you forget to
> put "var", because then your variable is global, which is fine when it's
> a plain int or string, not when it's a complex object whose mere
> existence changes lots of things.

When it's not specified in which scope a variable is created,
it should be an error.

"When in doubt, refuse the temptation to guess".

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From jcarlson at uci.edu  Mon Oct  9 21:18:12 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 09 Oct 2006 12:18:12 -0700
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452A9122.6070904@cs.byu.edu>
References: <ege2si$bej$1@sea.gmane.org> <452A9122.6070904@cs.byu.edu>
Message-ID: <20061009120645.098E.JCARLSON@uci.edu>


Neil Toronto <ntoronto at cs.byu.edu> wrote:
> Fredrik Lundh wrote:
> > Neil Toronto wrote:
> >> Heck no. An assignment would modify an existing variable only.
> >>     
> >
> > what's an "existing variable"?  how would STORE_GLOBAL work under your 
> > proposal?
> 
> I haven't got a clue what STORE_GLOBAL is, apart from the fact that it 
> looks like it has something to do with Python's bytecode.
> 
> If it's what I think it is (so don't hold me to this), it'd raise an 
> exception if you used on something that hadn't been created yet. 
> Probably need a MAKE_GLOBAL in addition to it, or keep STORE_GLOBAL as 
> it is (for "var x = 0") and add MODIFY_GLOBAL that raises an exception 
> on uncreated variables.
> 
> Or something. Again, I'm guessing, and you have me at a disadvantage. 
> Can we keep the discussion above the bytecode level for now?

Ok, but you need to understand a few things about how Python handles
things "under the covers".

Function-level scopes are not implemented as dictionaries.  They are
implemented as arrays, and the compiler maps local names to positions in
the in the array that stores locals.  The module-level namespace (which
can be accessed as a dictionary via globals()) is a dictionary, but this
is because anyone can insert or remove names arbitrarily, which is not
the case for function locals.  Also, Python's use of an array for locals
allows for a significant improvement in execution speed.


When Fredrik brought up the STORE_GLOBAL opcode, it was applicable
because the compiler needs to know at compile time whether a variable is
global (for the various *_GLOBAL opcodes) or local (for the various
*_FAST opcodes).

From what I can tell, the only thing that your 'var' keyword does is
ambiguate the global vs. local case, nevermind breaking 99% of currently
working Python code.  I'd save my time and get used to the way scopes
are working now, because I doubt they are going to change (none of the
core developers (I'm not one) who have chimed in have been positive).

 - Josiah


From gerrit at nl.linux.org  Mon Oct  9 17:24:17 2006
From: gerrit at nl.linux.org (Gerrit Holl)
Date: Mon, 9 Oct 2006 17:24:17 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <fb6fbf560610090646id0d418ak1c8ab6d28dbdb228@mail.gmail.com>
References: <452A027B.8060009@cs.byu.edu>
	<001b01c6eb86$1e47e110$0301010a@VIMES>
	<fb6fbf560610090646id0d418ak1c8ab6d28dbdb228@mail.gmail.com>
Message-ID: <20061009152416.GA18661@topjaklont.student.utwente.nl>

On 2006-10-09 15:47:09 +0200, Jim Jewett wrote:
> Because it is ambiguous.  The print indicates that you want (read)
> access to an existing variable, but the assignment indicates that you
> want (write) access to that variable.  Did you mean
> 
>     x = 0
>     def f():
>         global x  # modify the existing variable
>         print x
>         x = 3
> or
>     x = 0
>     def f():
>         x = x  # shadow the existing variable
>         print x
>         x = 3

I would expect the latter.

However, why global?
What's the use case for having global variables in the first place?
If I want to alter a variable that is not in the current scope, I use a
class:

class Foo:
    x = 0
    def f(self):
        self.x = 3

According to Google codesearch, it is hardly used in the Python library:
http://tinyurl.com/lkhlg
Is global evil?

Just asking.

regards,
Gerrit.

From qrczak at knm.org.pl  Mon Oct  9 21:24:25 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Mon, 09 Oct 2006 21:24:25 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <20061009120645.098E.JCARLSON@uci.edu> (Josiah Carlson's
	message of "Mon, 09 Oct 2006 12:18:12 -0700")
References: <ege2si$bej$1@sea.gmane.org> <452A9122.6070904@cs.byu.edu>
	<20061009120645.098E.JCARLSON@uci.edu>
Message-ID: <8764ete1hy.fsf@qrnik.zagroda>

Josiah Carlson <jcarlson at uci.edu> writes:

> From what I can tell, the only thing that your 'var' keyword does is
> ambiguate the global vs. local case,

No, it can select between any of outer scopes.

It also allows to put globals in an array instead of a dictionary,
making them as fast as locals.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From fredrik at pythonware.com  Mon Oct  9 21:41:23 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 09 Oct 2006 21:41:23 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <8764ete1hy.fsf@qrnik.zagroda>
References: <ege2si$bej$1@sea.gmane.org>
	<452A9122.6070904@cs.byu.edu>	<20061009120645.098E.JCARLSON@uci.edu>
	<8764ete1hy.fsf@qrnik.zagroda>
Message-ID: <ege8l4$3br$1@sea.gmane.org>

Marcin 'Qrczak' Kowalczyk wrote:

> It also allows to put globals in an array instead of a dictionary,
> making them as fast as locals.

so module namespaces will no longer be populated by *executing* the 
module contents?

what would happen if you tried to execute the following code?

    def foo():
       bar()

    def bar():
       pass

</F>


From qrczak at knm.org.pl  Mon Oct  9 21:49:54 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Mon, 09 Oct 2006 21:49:54 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <ege8l4$3br$1@sea.gmane.org> (Fredrik Lundh's message of "Mon,
	09 Oct 2006 21:41:23 +0200")
References: <ege2si$bej$1@sea.gmane.org> <452A9122.6070904@cs.byu.edu>
	<20061009120645.098E.JCARLSON@uci.edu> <8764ete1hy.fsf@qrnik.zagroda>
	<ege8l4$3br$1@sea.gmane.org>
Message-ID: <871wphe0bh.fsf@qrnik.zagroda>

Fredrik Lundh <fredrik at pythonware.com> writes:

>> It also allows to put globals in an array instead of a dictionary,
>> making them as fast as locals.
>
> so module namespaces will no longer be populated by *executing* the 
> module contents?

They would, but the set of global names except 'from module import *'
(which would need a slower lookup mechanism) could be determined
without executing it.

> what would happen if you tried to execute the following code?
>
>     def foo():
>        bar()
>
>     def bar():
>        pass

Depends on the semantics of var. I presented 3 choices.

My favorite choice is #3, for all bindings: the variable is visible in
the whole scope in which it is defined, but accessing it before its
definition has executed is an error. (Other choices are not practical
for functions.)

This means that the behavior of this code is the same as in current
Python.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From brian at sweetapp.com  Mon Oct  9 22:04:11 2006
From: brian at sweetapp.com (Brian Quinlan)
Date: Mon, 09 Oct 2006 22:04:11 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <871wphe0bh.fsf@qrnik.zagroda>
References: <ege2si$bej$1@sea.gmane.org>
	<452A9122.6070904@cs.byu.edu>	<20061009120645.098E.JCARLSON@uci.edu>
	<8764ete1hy.fsf@qrnik.zagroda>	<ege8l4$3br$1@sea.gmane.org>
	<871wphe0bh.fsf@qrnik.zagroda>
Message-ID: <452AAB3B.7070806@sweetapp.com>

Marcin 'Qrczak' Kowalczyk wrote:
> Fredrik Lundh <fredrik at pythonware.com> writes:
> 
>>> It also allows to put globals in an array instead of a dictionary,
>>> making them as fast as locals.
>> so module namespaces will no longer be populated by *executing* the 
>> module contents?
> 
> They would, but the set of global names except 'from module import *'
> (which would need a slower lookup mechanism) could be determined
> without executing it.

That would be a pretty big semantic change from Python <3000. e.g.

if sys.platform.startswith('win32'):
    import win32api
    def get_filesystem_roots():
        return win32api.GetLogicalDrives()
else:
    def get_filesystem_roots():
        return ['/']

How do I statically determine if win32api is a global or not? Do I have 
to declare it? Also, what happens in the presence of "exec" statements?

Cheers,
Brian


From ntoronto at cs.byu.edu  Mon Oct  9 22:07:15 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 09 Oct 2006 14:07:15 -0600
Subject: [Python-3000] ugly pie: a "var" keyword
In-Reply-To: <1160419224.7838.13.camel@fsol>
References: <452A027B.8060009@cs.byu.edu>
	<egd6sd$tq9$1@sea.gmane.org>	<egd9fh$689$1@sea.gmane.org>
	<1160419224.7838.13.camel@fsol>
Message-ID: <452AABF3.4040109@cs.byu.edu>

Antoine Pitrou wrote:
> Le lundi 09 octobre 2006 ? 12:49 +0200, Georg Brandl a ?crit :
>   
>> They wouldn't be "local" unless declared with "var", I assume.
>>     
>
> Besides, in a well-written program, you will mostly access local
> variables (and attributes and methods), so let's make the clean common
> case easy to type and unobstrusive, and the rare dirty case easy to
> spot. Lots of "global" in a Python program immediately hint at dirty
> code.
>   

I totally buy this argument. -1 on my original proposal, now that I see 
how messy it can become.

How about having "outer" replace "global?" Of course, it's been 
discussed to death - I believe it was called "lexical" or something. No 
reason we couldn't beat on that dead puppy some more, though.

I may come up with a new property for my own use called "deprecated" or 
something:

    x = deprecated()

when you want to throw errors at people (or yourself!) for not keeping 
up-to-date with changes to your class. Something like that ought to be 
standard.

Neil


From jcarlson at uci.edu  Mon Oct  9 22:31:09 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 09 Oct 2006 13:31:09 -0700
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <8764ete1hy.fsf@qrnik.zagroda>
References: <20061009120645.098E.JCARLSON@uci.edu>
	<8764ete1hy.fsf@qrnik.zagroda>
Message-ID: <20061009131359.0994.JCARLSON@uci.edu>


"Marcin 'Qrczak' Kowalczyk" <qrczak at knm.org.pl> wrote:
> 
> Josiah Carlson <jcarlson at uci.edu> writes:
> 
> > From what I can tell, the only thing that your 'var' keyword does is
> > ambiguate the global vs. local case,
> 
> No, it can select between any of outer scopes.

No.  It can't select between *any* of the outer scopes, only those
scopes that have previously existing *names* that one wants to modify,
and really, only the deepest nested scope with a 'var' declaration.

It is no better, semantically, then just using a list.  In fact, by
virtue of breaking currently working code, and making it visually
ambiguous, it is worse.  As Jim Jewett stated, using a list "is at least
strange enough to warn people that they need to read carefully."


> It also allows to put globals in an array instead of a dictionary,
> making them as fast as locals.

No, it doesn't.

    import foo
    #foo.a didn't exist before
    foo.a = x
    #foo.a exists now

This kind of thing was stated in the earlier thread about making globals
as fast as locals.


 - Josiah


From ncoghlan at gmail.com  Tue Oct 10 11:24:24 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Oct 2006 19:24:24 +1000
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452A027B.8060009@cs.byu.edu>
References: <452A027B.8060009@cs.byu.edu>
Message-ID: <452B66C8.6020703@gmail.com>

Neil Toronto wrote:
> As long as patches are flying willy-nilly and we're just kicking around 
> ideas like puppies, I thought I'd offer one of my own.
> 
> No, not a puppy. An idea. Of course, if it ends up beaten to death, I 
> might rather it were a puppy. My apologies if it's been discussed in 
> some other form.
> 
> It occurs to me that some of Python's gotchas are due to the compiler 
> not being able to determine whether the user intends to *create* a 
> variable, or *modify* one.

Any proposal such as this also needs to addresses all of the *other* name 
binding statements in Python:

   try/except
   for loop
   with statement
   def statement
   class statement

All of these statements bind a name in the current scope regardless of whether 
or not that name is used elsewhere in the current scope or an outer scope.

The current behaviour of assignment statements is identical - they all bind a 
name in the current scope (with augmented assignment statement expecting that 
the name already be defined).

This proposal introduces a *new* statement (var) to take on the meaning of the 
current assignment statement, and proposes to give assignment statements a 
special "rebind" behaviour that will silently break code like the following 
due to the missing 'var' on the inner x variable:

for x in range(10):
   def some_op(y):
       x = y+2
       real_op(x, y)
   for y in range(x):
       some_op(x, y)

I prefer my non-local effects to be explicit, thank-you-very-much. A solid -1 
here.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From fredrik at pythonware.com  Tue Oct 10 11:36:44 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 10 Oct 2006 11:36:44 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452B66C8.6020703@gmail.com>
References: <452A027B.8060009@cs.byu.edu> <452B66C8.6020703@gmail.com>
Message-ID: <egfpje$t96$1@sea.gmane.org>

Nick Coghlan wrote:

> Any proposal such as this also needs to addresses all of the *other* name 
> binding statements in Python:
> 
>    try/except
>    for loop
>    with statement
>    def statement
>    class statement

+    import statement
+    list comprehension (in 2.X)

and probably some other thing that I'm forgetting right now.

</F>


From python at zesty.ca  Tue Oct 10 15:34:49 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Tue, 10 Oct 2006 08:34:49 -0500 (CDT)
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <egfpje$t96$1@sea.gmane.org>
References: <452A027B.8060009@cs.byu.edu> <452B66C8.6020703@gmail.com>
	<egfpje$t96$1@sea.gmane.org>
Message-ID: <Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>

On Tue, 10 Oct 2006, Fredrik Lundh wrote:
> Nick Coghlan wrote:
> > Any proposal such as this also needs to addresses all of the *other* name
> > binding statements in Python:
> >
> >    try/except
> >    for loop
> >    with statement
> >    def statement
> >    class statement
> +    import statement
> +    list comprehension (in 2.X)

The only sane proposal i've seen that consistently addresses all
these cases, doesn't create new ambiguous situations, and doesn't
break most existing Python code is the "nonlocal" declaration.

    nonlocal x

means

    "Don't make a new binding for x in the local scope."

With this declaration, any binding statements modify the x that
is visible from an outer scope instead of creating a local x.


-- ?!ng

From fredrik at pythonware.com  Tue Oct 10 15:47:55 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 10 Oct 2006 15:47:55 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
References: <452A027B.8060009@cs.byu.edu>
	<452B66C8.6020703@gmail.com><egfpje$t96$1@sea.gmane.org>
	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
Message-ID: <egg8ab$tf9$1@sea.gmane.org>

Ka-Ping Yee wrote:

> The only sane proposal i've seen that consistently addresses all
> these cases, doesn't create new ambiguous situations, and doesn't
> break most existing Python code is the "nonlocal" declaration.
>
>    nonlocal x
>
> means
>
>    "Don't make a new binding for x in the local scope."

wasn't there a PEP on this once upon a time ?

wait a minute...

alright, the first google hit for "nonlocal pep" is a mail from Guido where he's
"+1/can anyone write a PEP" in response to a mail from you:

    http://mail.python.org/pipermail/python-dev/2006-July/067067.html

(and for the curious, the followups to Guido's post includes links to earlier
discussions).

</F> 




From ncoghlan at gmail.com  Tue Oct 10 16:45:05 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Oct 2006 00:45:05 +1000
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
References: <452A027B.8060009@cs.byu.edu>
	<452B66C8.6020703@gmail.com>	<egfpje$t96$1@sea.gmane.org>
	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
Message-ID: <452BB1F1.3000901@gmail.com>

Ka-Ping Yee wrote:
> On Tue, 10 Oct 2006, Fredrik Lundh wrote:
>> Nick Coghlan wrote:
>>> Any proposal such as this also needs to addresses all of the *other* name
>>> binding statements in Python:
>>>
>>>    try/except
>>>    for loop
>>>    with statement
>>>    def statement
>>>    class statement
>> +    import statement
>> +    list comprehension (in 2.X)
> 
> The only sane proposal i've seen that consistently addresses all
> these cases, doesn't create new ambiguous situations, and doesn't
> break most existing Python code is the "nonlocal" declaration.
> 
>     nonlocal x
> 
> means
> 
>     "Don't make a new binding for x in the local scope."
> 
> With this declaration, any binding statements modify the x that
> is visible from an outer scope instead of creating a local x.

Agreed - I believe the only real problem with the idea was that nobody could 
come up with a name for this statement that was clearly the right name.

Re-using 'global' wasn't popular because it would actually be *wrong* for the 
new semantics (the name might not be right), and other contenders like 'outer' 
and 'nonlocal' had their share of critics, too ('outer' isn't quite right 
because the name used is rebound in the innermost lexical scope that defines 
it, while 'nonlocal' had the problem of being a negative definition). Jeremy 
Hylton provided a nice list of the goals for a good choice of keyword [1] 
('global' failed criteria #2, whereas 'nonlocal' passed criteria #2 but failed 
the rest).

As mentioned in that thread, something like 'external' would be a fairly nice 
fit, but the terminology baggage from C/C++ could be a problem (since we would 
mean external to the current scope, but C/C++ uses the term to mean external 
to the module. That said, anyone trying to interpret Python scoping in terms 
of C/C++ scoping is going to get horribly confused in fairly short order, 
particularly if lexically scoped variables are involved).

     def f():
         x = 0
         def g(i=1):
             external x
             x += i
             return x
         return g

Cheers,
Nick.

[1] http://mail.python.org/pipermail/python-dev/2006-July/067205.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mbk.lists at gmail.com  Tue Oct 10 17:12:42 2006
From: mbk.lists at gmail.com (Mike Krell)
Date: Tue, 10 Oct 2006 08:12:42 -0700
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452BB1F1.3000901@gmail.com>
References: <452A027B.8060009@cs.byu.edu> <452B66C8.6020703@gmail.com>
	<egfpje$t96$1@sea.gmane.org>
	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
	<452BB1F1.3000901@gmail.com>
Message-ID: <da7032ce0610100812s4a02f70l4d2e24edfd6b1953@mail.gmail.com>

> > The only sane proposal i've seen that consistently addresses all
> > these cases, doesn't create new ambiguous situations, and doesn't
> > break most existing Python code is the "nonlocal" declaration.

[snip]

> Agreed - I believe the only real problem with the idea was that nobody could
> come up with a name for this statement that was clearly the right name.

[snip of arguments against previously proposed keywords]

The fact that this issue gets raised over and over again tells me that
I'm not alone in thinking this is a big language wart.  Please,
please, please add a "nonlocal" keyword under some appropriate
spelling.  Even if we can't all agree on the best name, I'd rather
have the small wart of an oddly spelled keyword than the IMHO big mess
we have today.

Then again, I'm the kind of guy that likes the new conditional
expression syntax, so what do I know? :-)

   Mike

From fredrik at pythonware.com  Tue Oct 10 17:42:57 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 10 Oct 2006 17:42:57 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <da7032ce0610100812s4a02f70l4d2e24edfd6b1953@mail.gmail.com>
References: <452A027B.8060009@cs.byu.edu>
	<452B66C8.6020703@gmail.com>	<egfpje$t96$1@sea.gmane.org>	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>	<452BB1F1.3000901@gmail.com>
	<da7032ce0610100812s4a02f70l4d2e24edfd6b1953@mail.gmail.com>
Message-ID: <eggf22$tfe$1@sea.gmane.org>

Mike Krell wrote:

> The fact that this issue gets raised over and over again tells me that
> I'm not alone in thinking this is a big language wart.

there's no connection whatsoever between things that are raised now and 
then in various forums, and things that are real "seriously hurts people
trying to get things done in Python" warts.

the vast majority of all Python programmers never uses nested functions 
at all.

</F>


From mbk.lists at gmail.com  Tue Oct 10 18:54:24 2006
From: mbk.lists at gmail.com (Mike Krell)
Date: Tue, 10 Oct 2006 09:54:24 -0700
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <eggf22$tfe$1@sea.gmane.org>
References: <452A027B.8060009@cs.byu.edu> <452B66C8.6020703@gmail.com>
	<egfpje$t96$1@sea.gmane.org>
	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
	<452BB1F1.3000901@gmail.com>
	<da7032ce0610100812s4a02f70l4d2e24edfd6b1953@mail.gmail.com>
	<eggf22$tfe$1@sea.gmane.org>
Message-ID: <da7032ce0610100954i74d8b913k42b4b0b6d0920f6f@mail.gmail.com>

> there's no connection whatsoever between things that are raised now and
> then in various forums, and things that are real "seriously hurts people
> trying to get things done in Python" warts.

Fair enough.

> the vast majority of all Python programmers never uses nested functions
> at all.

Yes, but part of the reason for this may be this very wart.  I know
I'm campaigning for this as a fix for what the OP calls the "read only
lexical scoping gotcha".  A fix for that makes it much more convienent
to write closures that modify closed-over values, which in turn makes
a much more compelling use case for using nested functions.

Of couse, you may think that's a bad thing.  Personally, I'd really like it.

In any event, as you pointed out yourself, at last check Guido is +1
on this general idea.  Maybe you should ask him why he's in favor of
it :-)

  Mike

From talin at acm.org  Tue Oct 10 20:30:23 2006
From: talin at acm.org (Talin)
Date: Tue, 10 Oct 2006 11:30:23 -0700
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452A027B.8060009@cs.byu.edu>
References: <452A027B.8060009@cs.byu.edu>
Message-ID: <452BE6BF.5060305@acm.org>

You might want to check out this thread in which I proposed something 
similar (and got similar objections):

http://mail.python.org/pipermail/python-dev/2006-July/066978.html

Neil Toronto wrote:
> As long as patches are flying willy-nilly and we're just kicking around 
> ideas like puppies, I thought I'd offer one of my own.
> 
> No, not a puppy. An idea. Of course, if it ends up beaten to death, I 
> might rather it were a puppy. My apologies if it's been discussed in 
> some other form.
> 
> It occurs to me that some of Python's gotchas are due to the compiler 
> not being able to determine whether the user intends to *create* a 
> variable, or *modify* one.
> 
> The "global" gotcha:
> 
> x = 0
> def f():
>     print x   # 'local variable 'x' referenced before assignment'
>     x = 3
> 
> 
> The "read-only lexical scoping" gotcha:
> 
> def f():
>     x = 0
>     def g():
>         x = 3   # doesn't alter outer 'x'
> 
> 
> And the "changing a writable property name is a very bad plan" gotcha. 
> This bit me severely when I was first learning Python. I changed an 
> attribute name from "x" to "value" in some Bayesian network code, and 
> spent hours trying to determine why it wasn't working anymore. Turns out 
> I missed a "node.x = " somewhere. Nowadays, I'd write a dummy property 
> named "x" that throws an exception, but that seems an extraordinarily 
> hackish thing to do to support an operation that happens a lot.
> 
> A "var" keyword fixes them all. The "global" gotcha:
> 
> x = 0
> def f():
>     print x   # no exception - prints "0"
>     x = 3
> 
> def g():
>     print x   # exception
>     var x = 3
> 
> 
> The read-only lexical scoping gotcha:
> 
> def f():
>     var x = 0
>     def g():
>         x = 3   # alters outer 'x'
> 
> 
> And when the runtime encountered "node.x = " it would have complained 
> that the variable didn't exist. The __init__ method would look like this:
> 
> def __init__(self, stuff):
>    var self.stuff = stuff
>    var self.x = 0
>    # ...
> 
> 
> If you don't like how it looks, we could call it "eric" instead:
> 
>    eric self.x = 0
> 
> 
> Of course, "var" isn't a swear word in any language (as far as I know), 
> but "eric" is heinously rude in Swahili.
> 
> Essentially, "var" means "make a new entry in the dictionary."
> 
> Some Zen arguments:
> 
> Explicit is better than implicit (are you creating? are you modifying?)
> Complex is worse than simple (it's slightly more complex), but complex 
> is better than complicated (which the scoping semantics are now)
> 
> It also has nice duality with the "del" keyword. Same number of letters 
> and everything:
> 
> var x = 0
> del x
> 
> 
> If type annotations ever got into Py3k, they'd go in very naturally. 
> This might also enable some compiler optimizations.
> 
> The biggest downside I can see is that it would break every scrap of 
> Python code ever written.
> 
> Another puppy to kick around: If that's totally unacceptable, how about 
> making the default for assignment act on a variable in the outer scope 
> if one exists, and have "var" override? This would allow read/write 
> lexical scoping and put the "this is meant to be new in this scope" 
> decoration at the point of declaration. (I also think "this is meant to 
> be new" is much cleaner than "this is meant to be old," but that's just me.)
> 
> Grovelingly,
> Neil
> 
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/talin%40acm.org
> 
> 
> 

From jcarlson at uci.edu  Tue Oct 10 23:17:10 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 10 Oct 2006 14:17:10 -0700
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <da7032ce0610100954i74d8b913k42b4b0b6d0920f6f@mail.gmail.com>
References: <eggf22$tfe$1@sea.gmane.org>
	<da7032ce0610100954i74d8b913k42b4b0b6d0920f6f@mail.gmail.com>
Message-ID: <20061010134725.09B7.JCARLSON@uci.edu>


"Mike Krell" <mbk.lists at gmail.com> wrote:
> 
> > there's no connection whatsoever between things that are raised now and
> > then in various forums, and things that are real "seriously hurts people
> > trying to get things done in Python" warts.
> 
> Fair enough.
> 
> > the vast majority of all Python programmers never uses nested functions
> > at all.
> 
> Yes, but part of the reason for this may be this very wart.  I know
> I'm campaigning for this as a fix for what the OP calls the "read only
> lexical scoping gotcha".  A fix for that makes it much more convienent
> to write closures that modify closed-over values, which in turn makes
> a much more compelling use case for using nested functions.

Every use that does not involve shadowing variables can be handled with
a list indexing variant.  Is it ugly and a bit slower?  Maybe.  But no
one who really wants to use nested functions is (in my opinion) hurt by
Python's lack of the equivalent of 'outer' or the variant 'my'.

That's one of the reasons I've been -1 for a while, but also because
nested functions can essentially be seen as shared namespaces, which can
also be implemented as methods on an instance, or even just attributes
off of a shared 'namespace' object.  The fact that a 'namespace object'
(not to be confused with local and lexical scopes) has slower access
times is an implementation detail that shouldn't concern users as much
as clarity and perhaps modularity, both of which can (and usually do)
become second-class citizens to speed.

For example, a user had been using the following type of thing in a
sudoku solver...

    def solve(...):
        state = set(...)
        def update_state(...):
            ...
            state ^= ...
            ...
        ...

Of course the ^= didn't work because state was a variable in a parent
scope.  The 'make it work' solution used:

    def solve(...):
        state = [set(...)]
        def update_state(...):
            ...
            state[0] ^= ...
            ...
        ...

Either way, the following would be more clear (at least in my opinion)...

    class namespace: pass

    def update_state(ns, ...):
        ...
        ns.state ^= ...
        ...

    def solve(...):
        ns = namespace()
        ns.state = set(...)
        ...

Why?  Because the shared namespace is explicit, both in the solve() and
update_state() definitions.  One could also then use the same
update_state() function anywhere else, thus increasing modularity and
potential for code reuse.  There is also the fact that it follows the
"flat is better than nested" zen, but the "explicit is better than
implicit" zen could be applied to either the explicit namespace, or even
the 'outer' or 'my' variants.


> In any event, as you pointed out yourself, at last check Guido is +1
> on this general idea.  Maybe you should ask him why he's in favor of
> it :-)

I'm only guessing here, but I would say it is because it removes a
(small) wart with regards to nested scopes, and it gets people to stop
complaining about this little thing (nevermind the functional
programming people who have been crying that Python ruins functional
programming for not having this included with the language).

 - Josiah


From mbk.lists at gmail.com  Wed Oct 11 00:31:08 2006
From: mbk.lists at gmail.com (Mike Krell)
Date: Tue, 10 Oct 2006 15:31:08 -0700
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <20061010134725.09B7.JCARLSON@uci.edu>
References: <eggf22$tfe$1@sea.gmane.org>
	<da7032ce0610100954i74d8b913k42b4b0b6d0920f6f@mail.gmail.com>
	<20061010134725.09B7.JCARLSON@uci.edu>
Message-ID: <da7032ce0610101531g4ac05e28jc747a6138da4c737@mail.gmail.com>

> > Yes, but part of the reason for this may be this very wart.  I know
> > I'm campaigning for this as a fix for what the OP calls the "read only
> > lexical scoping gotcha".  A fix for that makes it much more convienent
> > to write closures that modify closed-over values, which in turn makes
> > a much more compelling use case for using nested functions.
>
> Every use that does not involve shadowing variables can be handled with
> a list indexing variant.

Yes, I know.  Which is exactly why I described this change as a
convenience.  Just like programming in Python instead of machine
language is a convenience.  It's merely a matter of degree.

>  Is it ugly and a bit slower?  Maybe.  But no
> one who really wants to use nested functions is (in my opinion) hurt by
> Python's lack of the equivalent of 'outer' or the variant 'my'.

All conveniences come at a cost; the operative question is whether the
benefit outweighs the cost.  For both the reason I cited above and the
reasons you mention below, I think the answer is yes.  Reasonable
people can obviously disagree on this one.

> > In any event, as you pointed out yourself, at last check Guido is +1
> > on this general idea.  Maybe you should ask him why he's in favor of
> > it :-)
>
> I'm only guessing here, but I would say it is because it removes a
> (small) wart with regards to nested scopes, and it gets people to stop
> complaining about this little thing (nevermind the functional
> programming people who have been crying that Python ruins functional
> programming for not having this included with the language).

Yeah, I'm tired of people spreading the FUD that Python doesn't have
proper closures.  It does; they're just inconvenient to use.
Hopefully Python 3000 will address this :-)

   Mike

From greg.ewing at canterbury.ac.nz  Wed Oct 11 02:15:31 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 11 Oct 2006 13:15:31 +1300
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452BB1F1.3000901@gmail.com>
References: <452A027B.8060009@cs.byu.edu> <452B66C8.6020703@gmail.com>
	<egfpje$t96$1@sea.gmane.org>
	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
	<452BB1F1.3000901@gmail.com>
Message-ID: <452C37A3.8000007@canterbury.ac.nz>

Nick Coghlan wrote:

> Re-using 'global' wasn't popular because it would actually be *wrong* for the 
> new semantics

All things considered, re-using "global" is
what I'd be most in favour of at the moment.

I don't agree that it's wrong -- it's perfectly
legitimate to regard "local" and "global" as
relative terms, i.e. anything not local to you
is global from your point of view.

--
Greg

From ntoronto at cs.byu.edu  Wed Oct 11 04:51:12 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Tue, 10 Oct 2006 20:51:12 -0600
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452BB1F1.3000901@gmail.com>
References: <452A027B.8060009@cs.byu.edu>	<452B66C8.6020703@gmail.com>	<egfpje$t96$1@sea.gmane.org>	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
	<452BB1F1.3000901@gmail.com>
Message-ID: <452C5C20.10602@cs.byu.edu>

Nick Coghlan wrote:
>> The only sane proposal i've seen that consistently addresses all
>> these cases, doesn't create new ambiguous situations, and doesn't
>> break most existing Python code is the "nonlocal" declaration.
>>
>>     nonlocal x
>>
>> means
>>
>>     "Don't make a new binding for x in the local scope."
>>
>> With this declaration, any binding statements modify the x that
>> is visible from an outer scope instead of creating a local x.
>>     
>
> Agreed - I believe the only real problem with the idea was that nobody could 
> come up with a name for this statement that was clearly the right name.
>
> Re-using 'global' wasn't popular because it would actually be *wrong* for the 
> new semantics (the name might not be right), and other contenders like 'outer' 
> and 'nonlocal' had their share of critics, too ('outer' isn't quite right 
> because the name used is rebound in the innermost lexical scope that defines 
> it, while 'nonlocal' had the problem of being a negative definition). Jeremy 
> Hylton provided a nice list of the goals for a good choice of keyword [1] 
> ('global' failed criteria #2, whereas 'nonlocal' passed criteria #2 but failed 
> the rest).
>   

 From what I can see, you could object to *every* keyword on the grounds 
that the name is rebound in the innermost lexical scope: nonlocal (it's 
not, it has a local binding), outer (it's really inner), and external 
(nope, it's internal) all have this problem.

For me, +1 on "outer" because it has the fewest syllables and 
characters, and accurately describes the semantics (minus the rebinding, 
which every keyword lies about) without baggage from any other language 
I know of.

I also like "outie," but I'm weird like that.

So is the reason a PEP was never drafted because nobody could agree on 
the word? It seems like the name should be subject to the standard 
Pythonic way of doing this: draft a PEP with candidates, vote on them, 
and then Guido will disregard the voting results and pick the one he 
likes. :D

Neil


From ncoghlan at gmail.com  Wed Oct 11 10:27:13 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Oct 2006 18:27:13 +1000
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452C37A3.8000007@canterbury.ac.nz>
References: <452A027B.8060009@cs.byu.edu> <452B66C8.6020703@gmail.com>
	<egfpje$t96$1@sea.gmane.org>
	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
	<452BB1F1.3000901@gmail.com> <452C37A3.8000007@canterbury.ac.nz>
Message-ID: <452CAAE1.4010808@gmail.com>

Greg Ewing wrote:
> Nick Coghlan wrote:
> 
>> Re-using 'global' wasn't popular because it would actually be *wrong* 
>> for the new semantics
> 
> All things considered, re-using "global" is
> what I'd be most in favour of at the moment.
> 
> I don't agree that it's wrong -- it's perfectly
> legitimate to regard "local" and "global" as
> relative terms, i.e. anything not local to you
> is global from your point of view.

Except that the term currently refers specifically to module globals. Names in 
nested scopes are typically referred to as closure variables, not as any 
flavour of global.

You *could* try to change the meaning of global as you describe, but I believe 
it would be horribly confusing ("wait, do you mean a real module global, or do 
you mean a non-local variable?").

Cheers,
Nick.

P.S. In case it wasn't clear, I'm personally fine with 'nonlocal'. The only 
objections I've seen to the term are that it's a negative definition (but the 
concept itself is a negative one) and the word looks ugly to some folks. 
Neither of those objections bothers me overly much :)


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From g.brandl at gmx.net  Wed Oct 11 10:31:33 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 11 Oct 2006 10:31:33 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452CAAE1.4010808@gmail.com>
References: <452A027B.8060009@cs.byu.edu>
	<452B66C8.6020703@gmail.com>	<egfpje$t96$1@sea.gmane.org>	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>	<452BB1F1.3000901@gmail.com>
	<452C37A3.8000007@canterbury.ac.nz> <452CAAE1.4010808@gmail.com>
Message-ID: <egia56$b5h$1@sea.gmane.org>

Nick Coghlan wrote:

> P.S. In case it wasn't clear, I'm personally fine with 'nonlocal'. The only 
> objections I've seen to the term are that it's a negative definition (but the 
> concept itself is a negative one) and the word looks ugly to some folks. 
> Neither of those objections bothers me overly much :)

Plus, it's not used in Python code as an identifier (judging from the stdlib and
what's indexed on Google's codesearch).

Georg


From fredrik at pythonware.com  Wed Oct 11 10:34:35 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 11 Oct 2006 10:34:35 +0200
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452CAAE1.4010808@gmail.com>
References: <452A027B.8060009@cs.byu.edu>
	<452B66C8.6020703@gmail.com>	<egfpje$t96$1@sea.gmane.org>	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>	<452BB1F1.3000901@gmail.com>
	<452C37A3.8000007@canterbury.ac.nz> <452CAAE1.4010808@gmail.com>
Message-ID: <egiaas$btk$1@sea.gmane.org>

Nick Coghlan wrote:

> Except that the term currently refers specifically to module globals. Names in 
> nested scopes are typically referred to as closure variables

they're called "free variables" in Python.

"closure" is such a boring name.  when did you last see a function that 
wasn't "closed"?

</F>


From ncoghlan at gmail.com  Wed Oct 11 10:47:20 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Oct 2006 18:47:20 +1000
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452C5C20.10602@cs.byu.edu>
References: <452A027B.8060009@cs.byu.edu>	<452B66C8.6020703@gmail.com>	<egfpje$t96$1@sea.gmane.org>	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>	<452BB1F1.3000901@gmail.com>
	<452C5C20.10602@cs.byu.edu>
Message-ID: <452CAF98.7050403@gmail.com>

Neil Toronto wrote:
>  From what I can see, you could object to *every* keyword on the grounds 
> that the name is rebound in the innermost lexical scope: nonlocal (it's 
> not, it has a local binding), outer (it's really inner), and external 
> (nope, it's internal) all have this problem.

The specific problem with 'outer' is the potential confusion between 'an outer 
scope' and 'the outermost scope'.

The other two terms don't have the same problem - it is fairly clear with 
either term that the author is merely saying something about where the 
variable *isn't* (i.e. not local to this function), rather than something 
about where it actually is.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From python at zesty.ca  Wed Oct 11 11:18:05 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Wed, 11 Oct 2006 04:18:05 -0500 (CDT)
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452C37A3.8000007@canterbury.ac.nz>
References: <452A027B.8060009@cs.byu.edu> <452B66C8.6020703@gmail.com>
	<egfpje$t96$1@sea.gmane.org>
	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
	<452BB1F1.3000901@gmail.com> <452C37A3.8000007@canterbury.ac.nz>
Message-ID: <Pine.LNX.4.58.0610110407520.13594@server1.LFW.org>

Nick Coghlan wrote:
> Re-using 'global' wasn't popular because it would actually be
> *wrong* for the new semantics

On Wed, 11 Oct 2006, Greg Ewing wrote:
> I don't agree that it's wrong -- it's perfectly
> legitimate to regard "local" and "global" as
> relative terms, i.e. anything not local to you
> is global from your point of view.

It is possible to think of those terms in that way, but in this
context, i disagree.  I don't believe this is what any programmer
means when they say "global variable".

Consider the following example:

    def spam(x):
        def eggs():
            global x
            x = x + 1
        eggs()
        print x

In today's Python, eggs() affects the global variable x; with the
new meaning for "global", it instead affects the x local to spam().

Here are two arguments against redefining the "global" keyword:

1.  Notice that changing "global" to mean "don't create a new binding"
    changes the meaning of the above code incompatibly.

2.  With the new meaning for "global", what is x?  Is x a "global
    variable"?  I just can't imagine calling it that.  It doesn't
    make sense to use the keyword "global" to declare something that
    is clearly not a global variable.


-- ?!ng

From krstic at solarsail.hcs.harvard.edu  Wed Oct 11 11:21:33 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Wed, 11 Oct 2006 05:21:33 -0400
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <egg8ab$tf9$1@sea.gmane.org>
References: <452A027B.8060009@cs.byu.edu>	<452B66C8.6020703@gmail.com><egfpje$t96$1@sea.gmane.org>	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
	<egg8ab$tf9$1@sea.gmane.org>
Message-ID: <452CB79D.60603@solarsail.hcs.harvard.edu>

Fredrik Lundh wrote:
> alright, the first google hit for "nonlocal pep" is a mail from Guido
> where he's "+1/can anyone write a PEP" in response to a mail from
> you:

I'm +1 on `nonlocal` as well, having long been bothered by the absence
of something like it. Ka-Ping, will you write a PEP? If not, I'll take
it on.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From krstic at solarsail.hcs.harvard.edu  Wed Oct 11 11:26:38 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Wed, 11 Oct 2006 05:26:38 -0400
Subject: [Python-3000] Kill GIL? - to PEP 3099?
In-Reply-To: <eerhns$v1h$3@sea.gmane.org>
References: <fb6fbf560609180856s1b63ded2o66061d453368a9@mail.gmail.com>	<451142FF.20203@solarsail.hcs.harvard.edu>	<eerg9j$pj8$1@sea.gmane.org>
	<eerhns$v1h$3@sea.gmane.org>
Message-ID: <452CB8CE.5060209@solarsail.hcs.harvard.edu>

Fredrik Lundh wrote:
> and you don't need to wait for Python 3000 either, of course -- if 
> done right, [shared memory] would certainly fit into some future 2.X
> release.

Working on this. Unixoids should be doable nicely; Windows is a bit of a
challenge, but I'm looking to see what's the best I can do there.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From mbk.lists at gmail.com  Wed Oct 11 16:47:49 2006
From: mbk.lists at gmail.com (Mike Krell)
Date: Wed, 11 Oct 2006 07:47:49 -0700
Subject: [Python-3000] Sky pie: a "var" keyword
In-Reply-To: <452CB79D.60603@solarsail.hcs.harvard.edu>
References: <452A027B.8060009@cs.byu.edu> <452B66C8.6020703@gmail.com>
	<egfpje$t96$1@sea.gmane.org>
	<Pine.LNX.4.58.0610100831450.13594@server1.LFW.org>
	<egg8ab$tf9$1@sea.gmane.org>
	<452CB79D.60603@solarsail.hcs.harvard.edu>
Message-ID: <da7032ce0610110747s6a831b1eg9f666c493ee7cece@mail.gmail.com>

> I'm +1 on `nonlocal` as well, having long been bothered by the absence
> of something like it. Ka-Ping, will you write a PEP? If not, I'll take
> it on.

Since a PEP seems to be on the horizon, I just wanted to give a quick
mention of what eventually became my favorite spelling of this keyword
-- "use".  To me, it's short, not awkward, and reasonably suggestive
of the semantics.

As I mentioned before, I'm +1 regardless of the spelling.

   Mike

From solipsis at pitrou.net  Thu Oct 12 10:25:40 2006
From: solipsis at pitrou.net (Antoine)
Date: Thu, 12 Oct 2006 10:25:40 +0200 (CEST)
Subject: [Python-3000] try ... then ... then ... except / finally
Message-ID: <26478.62.39.9.251.1160641540.squirrel@webmail.nerim.net>


Well I'm sorry for polluting the list with another wild idea.

It often happens that you have to do some cleanup of multiple resources,
but those cleanup functions may raise an exception. Yet you wanna clean
all the resources. So you end up writing something like this :

def finish(self):
    try:
        self.db_transaction.commit()
    finally:
        try:
            self.tmpoutfile.close()
        finally:
            (etc.)

Needless to say that as soon as you have two or more resources, it becomes
quite ugly.

I thought it would be nice to have an idiom where several blocks of code
are tried in order, regardless of whether they throw an exception or not:

try:
    self.db_transaction.commit()
then:
    self.tmpoutfile.close()
then:
    self.destroy_shared_memory_handle()
except Exception, e:
    logging.error("an exception occurred in cleanup: %s", e)

Now, there are two questions I'm wondering about:
- is it necessary? have I missed an existing idiom for doing this as nicely?
- if several exceptions are raised (one for each block of code), what is
caught in the "except" statement? The last raised exception (possibly
chained to the previous one, and so on)?

Sorry if it looks really silly.

Regards

Antoine.



From g.brandl at gmx.net  Thu Oct 12 10:31:40 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 12 Oct 2006 10:31:40 +0200
Subject: [Python-3000] try ... then ... then ... except / finally
In-Reply-To: <26478.62.39.9.251.1160641540.squirrel@webmail.nerim.net>
References: <26478.62.39.9.251.1160641540.squirrel@webmail.nerim.net>
Message-ID: <egkuhc$4b1$1@sea.gmane.org>

Antoine wrote:

> try:
>     self.db_transaction.commit()
> then:
>     self.tmpoutfile.close()
> then:
>     self.destroy_shared_memory_handle()
> except Exception, e:
>     logging.error("an exception occurred in cleanup: %s", e)


Can you explain the difference to:

try:
     self.db_transaction.commit()
     self.tmpoutfile.close()
     self.destroy_shared_memory_handle()
except Exception, e:
     logging.error("an exception occurred in cleanup: %s", e)


Geor


From solipsis at pitrou.net  Thu Oct 12 11:01:14 2006
From: solipsis at pitrou.net (Antoine)
Date: Thu, 12 Oct 2006 11:01:14 +0200 (CEST)
Subject: [Python-3000] try ... then ... then ... except / finally
In-Reply-To: <egkuhc$4b1$1@sea.gmane.org>
References: <26478.62.39.9.251.1160641540.squirrel@webmail.nerim.net>
	<egkuhc$4b1$1@sea.gmane.org>
Message-ID: <56946.62.39.9.251.1160643674.squirrel@webmail.nerim.net>


> Can you explain the difference to:
>
> try:
>      self.db_transaction.commit()
>      self.tmpoutfile.close()
>      self.destroy_shared_memory_handle()
> except Exception, e:
>      logging.error("an exception occurred in cleanup: %s", e)

In your construct, if self.db_transaction.commit() raises an exception,
self.tmpoutfile won't get closed and self.destroy_shared_memory_handle()
won't be called.

In my proposal, if an exception is raised in a "try" block, the execution
skips to the next "then" block, and so on.
At the end it executes the "except" block if at least one "try" or "then"
block raised an exception.




From fredrik at pythonware.com  Thu Oct 12 11:18:29 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 12 Oct 2006 11:18:29 +0200
Subject: [Python-3000] try ... then ... then ... except / finally
References: <26478.62.39.9.251.1160641540.squirrel@webmail.nerim.net>
Message-ID: <egl195$ffs$1@sea.gmane.org>

Antoine wrote:

> I thought it would be nice to have an idiom where several blocks of code
> are tried in order, regardless of whether they throw an exception or not:
>
> try:
>    self.db_transaction.commit()
> then:
>    self.tmpoutfile.close()
> then:
>    self.destroy_shared_memory_handle()
> except Exception, e:
>    logging.error("an exception occurred in cleanup: %s", e)

that's spelled "with nested" in Python 2.5.  see:

    http://docs.python.org/whatsnew/pep-343.html

</F> 




From g.brandl at gmx.net  Thu Oct 12 12:39:20 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 12 Oct 2006 12:39:20 +0200
Subject: [Python-3000] try ... then ... then ... except / finally
In-Reply-To: <56946.62.39.9.251.1160643674.squirrel@webmail.nerim.net>
References: <26478.62.39.9.251.1160641540.squirrel@webmail.nerim.net>	<egkuhc$4b1$1@sea.gmane.org>
	<56946.62.39.9.251.1160643674.squirrel@webmail.nerim.net>
Message-ID: <egl60o$1s3$1@sea.gmane.org>

Antoine wrote:
>> Can you explain the difference to:
>>
>> try:
>>      self.db_transaction.commit()
>>      self.tmpoutfile.close()
>>      self.destroy_shared_memory_handle()
>> except Exception, e:
>>      logging.error("an exception occurred in cleanup: %s", e)
> 
> In your construct, if self.db_transaction.commit() raises an exception,
> self.tmpoutfile won't get closed and self.destroy_shared_memory_handle()
> won't be called.
> 
> In my proposal, if an exception is raised in a "try" block, the execution
> skips to the next "then" block, and so on.
> At the end it executes the "except" block if at least one "try" or "then"
> block raised an exception.

Ah. On Error Resume Next, revisited ;)

Like Fredrik, I really think that this is a good application for context
managers.

Georg


From talin at acm.org  Fri Oct 13 07:23:23 2006
From: talin at acm.org (Talin)
Date: Thu, 12 Oct 2006 22:23:23 -0700
Subject: [Python-3000] Alternatives to 'outer'
Message-ID: <452F22CB.2030806@acm.org>

In reference to the discussion of assignment to variables in outer 
scopes: I get the sense that there are a substantial number of folks who 
  are in agreement on the basic semantics, and the only sticking point 
is exactly what keyword to pick.

(I should mention that I'm still off to the side in the 'point of 
definition' camp, but I'm always happy to help out with things even if I 
don't agree with them :)

So let me spit out some possible suggestions in terms of keywords:

	ref
	refer
	share
	sharing
	common
	use
	using
	borrow

Rationale: 'ref' / 'use' / 'using' in the sense we're not declaring a 
variable, but simply referring to one.

'share' / 'sharing' / 'common' - I'm thinking of the word 'share' in the 
dictionary sense of common ownership, or a 'nonrival good'.)

(BTW, Guido, who I talked to at BayPiggies tonight, tells me that 
'share' meant the same as 'global' in ABC.)

To get a sense of how 'refer' might look in context:

   def myfunc():
     x = 10
     def inner():
        refer x
        x = 20
     inner()


-- Talin

From talin at acm.org  Fri Oct 13 07:39:47 2006
From: talin at acm.org (Talin)
Date: Thu, 12 Oct 2006 22:39:47 -0700
Subject: [Python-3000] Proposal: No more standard library additions
Message-ID: <452F26A3.7060506@acm.org>

Now that I've gotten your attention :)

Seriously, though, I am not proposing that there *never* be additions to 
the standard library -- instead, I simply want 'easy_install' to work 
100% of the time, so that there's much less reason to add something to 
the standard library.

In fact, I would go a step further and say that I'd like to see the 
standard library cut in half. For backwards compatibility with existing 
Python programs, you would simply type 'easy_install legacy', and all of 
the old "batteries included" modules would be installed for you. For 
writing *new* programs however, you could use a much lighter, slimmer 
distribution.

How's that for Py3000 cruft removal? :)

Unfortunately, the main obstacle at the moment is that about 50% of the 
packages out there aren't compatible with setuptools. About 50% of the 
time when I say 'easy_install <bleargh>' it downloads the package and 
then says 'setup.py' not found.

Moreover, I want 'easy_uninstall' to work just as easily, and I want it 
to clean up the installed package without a trace. I want to be able to 
download X, check it out, say "blech, I don't like X", uninstall, and 
have no guilt or loose files cluttering up my site-packages.

-- Talin


From ronaldoussoren at mac.com  Fri Oct 13 07:56:06 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri, 13 Oct 2006 07:56:06 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <452F26A3.7060506@acm.org>
References: <452F26A3.7060506@acm.org>
Message-ID: <3AE5B28E-97D4-422C-AFBB-5F42893851DB@mac.com>


On Oct 13, 2006, at 7:39 AM, Talin wrote:

> Now that I've gotten your attention :)
>
> Seriously, though, I am not proposing that there *never* be  
> additions to
> the standard library -- instead, I simply want 'easy_install' to work
> 100% of the time, so that there's much less reason to add something to
> the standard library.
>
> In fact, I would go a step further and say that I'd like to see the
> standard library cut in half. For backwards compatibility with  
> existing
> Python programs, you would simply type 'easy_install legacy', and  
> all of
> the old "batteries included" modules would be installed for you. For
> writing *new* programs however, you could use a much lighter, slimmer
> distribution.
>
> How's that for Py3000 cruft removal? :)

I'm +1 on the general idea.

>
> Unfortunately, the main obstacle at the moment is that about 50% of  
> the
> packages out there aren't compatible with setuptools. About 50% of the
> time when I say 'easy_install <bleargh>' it downloads the package and
> then says 'setup.py' not found.
>
> Moreover, I want 'easy_uninstall' to work just as easily, and I  
> want it
> to clean up the installed package without a trace. I want to be  
> able to
> download X, check it out, say "blech, I don't like X", uninstall, and
> have no guilt or loose files cluttering up my site-packages.

There's no need to wait for py3k for this, you can start on this  
right now :-). The first part is probably mostly evangelizing and/or  
patching packages.

The second part is extending setuptools with better package  
management, PJE has some ideas about that, hop over to distutils-sig  
to ask where to start ;-)

Ronald

P.S. I already have a lightweight version of easy_uninstall, it's  
called rm :-)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061013/8fc0f771/attachment.bin 

From rasky at develer.com  Fri Oct 13 08:25:55 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Fri, 13 Oct 2006 08:25:55 +0200
Subject: [Python-3000] Proposal: No more standard library additions
References: <452F26A3.7060506@acm.org>
Message-ID: <2ab101c6ee90$83796470$9a4c2a97@bagio>

Talin wrote:

> Seriously, though, I am not proposing that there never be additions
> to the standard library -- instead, I simply want 'easy_install' to
> work 100% of the time, so that there's much less reason to add
> something to the standard library.

-10000000.

The standard library is not about easeness of installation. It is about having
a consistent fixed codebase to work with. I don't want to go Perl/CPAN,
where you have 3-4 alternatives to do thing A which will never interoperate
with whatever you chose among the 3-4 alternatives to do thing B. I don't want
to stop being able to read and fix programs because they use 15 libraries I
never heard of, with obscure APIs and terrible naming conventions.

Let us keep our batteries, please.
--
Giovanni Bajo

From ronaldoussoren at mac.com  Fri Oct 13 09:30:24 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri, 13 Oct 2006 09:30:24 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <2ab101c6ee90$83796470$9a4c2a97@bagio>
References: <452F26A3.7060506@acm.org> <2ab101c6ee90$83796470$9a4c2a97@bagio>
Message-ID: <5848559.1160724624593.JavaMail.ronaldoussoren@mac.com>

 
On Friday, October 13, 2006, at 08:26AM, Giovanni Bajo <rasky at develer.com> wrote:

>Talin wrote:
>
>> Seriously, though, I am not proposing that there never be additions
>> to the standard library -- instead, I simply want 'easy_install' to
>> work 100% of the time, so that there's much less reason to add
>> something to the standard library.
>
>-10000000.
>
>The standard library is not about easeness of installation. It is about having
>a consistent fixed codebase to work with. I don't want to go Perl/CPAN,
>where you have 3-4 alternatives to do thing A which will never interoperate
>with whatever you chose among the 3-4 alternatives to do thing B. I don't want
>to stop being able to read and fix programs because they use 15 libraries I
>never heard of, with obscure APIs and terrible naming conventions.
>
>Let us keep our batteries, please.

An alternative to a large standard library is a smaller one plus a set of 3th-party libraries that are advertised as being good libraries. That way "we" can point people that look for a good cross-platform GUI library to wxPython, without needing to ship it as part of the standard library (to use one example).

Ronald


From rasky at develer.com  Fri Oct 13 10:04:05 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Fri, 13 Oct 2006 10:04:05 +0200
Subject: [Python-3000] Proposal: No more standard library additions
References: <452F26A3.7060506@acm.org> <2ab101c6ee90$83796470$9a4c2a97@bagio>
	<5848559.1160724624593.JavaMail.ronaldoussoren@mac.com>
Message-ID: <2bd601c6ee9e$27471090$9a4c2a97@bagio>

Ronald Oussoren wrote:

> An alternative to a large standard library is a smaller one plus a
> set of 3th-party libraries that are advertised as being good
> libraries. That way "we" can point people that look for a good
> cross-platform GUI library to wxPython, without needing to ship it as
> part of the standard library (to use one example).

I agree in the case of large toolkits like GUIs, but the 99% rest of the
current standard library belongs to a standard Python distribution.

Giovanni Bajo


From solipsis at pitrou.net  Fri Oct 13 10:19:04 2006
From: solipsis at pitrou.net (Antoine)
Date: Fri, 13 Oct 2006 10:19:04 +0200 (CEST)
Subject: [Python-3000] Alternatives to 'outer'
In-Reply-To: <452F22CB.2030806@acm.org>
References: <452F22CB.2030806@acm.org>
Message-ID: <17700.62.39.9.251.1160727544.squirrel@webmail.nerim.net>

> So let me spit out some possible suggestions in terms of keywords:
>
> 	ref
> 	refer
> 	share
> 	sharing
> 	common
> 	use
> 	using
> 	borrow

I think you are trying too hard.
Those proposals are all semantically ambiguous.
"ref" / "refer" seems to hint at the notion of reference or pointer (and
the "weakref" module already uses those words).
"share" / "sharing" seems to hint at a shared library thing.
"use" and "using" are used in other languages to import namespaces, or
define options.
"common" and "borrow" looks like it might be some kind of totally global
symbol (a global constant, whatever).

IMHO "nonlocal" is fine, it tells just what it does, not more, not less;
it is part of the same kind of vocabulary as "global", which is the right
thing as they are really similar notions.

Also, "nonlocal" is probably quite rare as an identifier in Python
programs (because of the negation: you hardly ever name a variable
"nonsomething"), which won't be the case with "ref", "share", "common",
etc.




From solipsis at pitrou.net  Fri Oct 13 10:26:53 2006
From: solipsis at pitrou.net (Antoine)
Date: Fri, 13 Oct 2006 10:26:53 +0200 (CEST)
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <2ab101c6ee90$83796470$9a4c2a97@bagio>
References: <452F26A3.7060506@acm.org> <2ab101c6ee90$83796470$9a4c2a97@bagio>
Message-ID: <52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>

> The standard library is not about easeness of installation. It is about
> having
> a consistent fixed codebase to work with. I don't want to go Perl/CPAN,
> where you have 3-4 alternatives to do thing A which will never
> interoperate
> with whatever you chose among the 3-4 alternatives to do thing B.

Currently in Python:
http://docs.python.org/lib/module-xml.dom.html
http://docs.python.org/lib/module-xml.dom.minidom.html
http://docs.python.org/lib/module-xml.sax.html
http://docs.python.org/lib/module-xml.parsers.expat.html
http://docs.python.org/lib/module-xml.etree.ElementTree.html

The problem of "consistent fixed codebase" is that standards get higher,
so eventually those old stable modules lose popularity in favor of newer,
better modules.

Therefore, you have to obsolete old stuff if you want there to be only One
Obvious Way To Do It.



From fredrik at pythonware.com  Fri Oct 13 10:37:15 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 13 Oct 2006 10:37:15 +0200
Subject: [Python-3000] Proposal: No more standard library additions
References: <452F26A3.7060506@acm.org> <2ab101c6ee90$83796470$9a4c2a97@bagio>
	<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>
Message-ID: <egnj7r$7mk$1@sea.gmane.org>

Antoine wrote:

>> The standard library is not about easeness of installation. It is about
>> having
>> a consistent fixed codebase to work with. I don't want to go Perl/CPAN,
>> where you have 3-4 alternatives to do thing A which will never
>> interoperate
>> with whatever you chose among the 3-4 alternatives to do thing B.
>
> Currently in Python:
> http://docs.python.org/lib/module-xml.dom.html
> http://docs.python.org/lib/module-xml.dom.minidom.html
> http://docs.python.org/lib/module-xml.sax.html
> http://docs.python.org/lib/module-xml.parsers.expat.html
> http://docs.python.org/lib/module-xml.etree.ElementTree.html

those five are in no way mutually exclusive, though.

</F> 




From rasky at develer.com  Fri Oct 13 11:49:15 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Fri, 13 Oct 2006 11:49:15 +0200
Subject: [Python-3000] Proposal: No more standard library additions
References: <452F26A3.7060506@acm.org> <2ab101c6ee90$83796470$9a4c2a97@bagio>
	<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>
Message-ID: <2f7601c6eeac$d85fcb20$9a4c2a97@bagio>

Antoine wrote:

>> The standard library is not about easeness of installation. It is
>> about having
>> a consistent fixed codebase to work with. I don't want to go
>> Perl/CPAN, where you have 3-4 alternatives to do thing A which will
>> never interoperate
>> with whatever you chose among the 3-4 alternatives to do thing B.
>
> Currently in Python:
> http://docs.python.org/lib/module-xml.dom.html
> http://docs.python.org/lib/module-xml.dom.minidom.html
> http://docs.python.org/lib/module-xml.sax.html
> http://docs.python.org/lib/module-xml.parsers.expat.html
> http://docs.python.org/lib/module-xml.etree.ElementTree.html
>
> The problem of "consistent fixed codebase" is that standards get
> higher, so eventually those old stable modules lose popularity in
> favor of newer, better modules.

Those are different paradigms of "doing XML". For instance, the standard
library was missing a "pythonic" library to do XML processing, and several
arose. ElementTree (fortunately) won and joined the standard distribution. This
should allievate the need for other libraries in future.

Instead of looking what we have inside, look outside. There are dozens of
different XML "pythonic" libraries. I have fought in the past with programs
that required large XML frameworks, that in turn required to be downloaded,
built, installed, and understood to make the required modifictions to the
programs themselves. This slowed down my own development, and caused infinite
headaches before of version compatibilities (A requires the XML library B, but
only versions < 1.2, otherwise you can use A 2.0, which needs Python 2.4+, and
then you can use latest B; etc. etc. repeat and complicate ad-libitum). A
single version number (that of Python) and a large fixed set of libraries
anybody can use is a strong PLUS.

Then, there is the opposite phenomenom, which is interesting as well. I met
many perl programmers which simply re-invented their little wheel everytime.
They were mostly system administrators, so they knew very well what hell the
dependency chains are for both programmers and users. Thus, since perl does not
have a standard library, they simply did not import any module. This way, the
program is "easier" to ship, distribute and use, but it's harder to code, read,
fix, and contain unnecessary duplications with everybody's else script. Need to
send an e-mail? Why using a library, just paste chunks of cut&pasted mail
headers (with MIME, etc.) and do some basic string substitution; and the SMTP
protocol is easy, just open a socket a dump some strings to it; or you can use
'sendmail' which is available on any UNIX (and there it goes portability, just
because they did not want to evaluate and choose one of the 6 Perl SMTP
libraries... and rightfully so!).

> Therefore, you have to obsolete old stuff if you want there to be
> only One Obvious Way To Do It.

I'm totally in favor of obsoletion and removal of old cruft from the standard
library.
I'm totally against not having a standard library.

Giovanni Bajo


From talin at acm.org  Fri Oct 13 19:01:51 2006
From: talin at acm.org (Talin)
Date: Fri, 13 Oct 2006 10:01:51 -0700
Subject: [Python-3000] Alternatives to 'outer'
In-Reply-To: <17700.62.39.9.251.1160727544.squirrel@webmail.nerim.net>
References: <452F22CB.2030806@acm.org>
	<17700.62.39.9.251.1160727544.squirrel@webmail.nerim.net>
Message-ID: <452FC67F.2050207@acm.org>

Antoine wrote:
> IMHO "nonlocal" is fine, it tells just what it does, not more, not less;
> it is part of the same kind of vocabulary as "global", which is the right
> thing as they are really similar notions.
> 
> Also, "nonlocal" is probably quite rare as an identifier in Python
> programs (because of the negation: you hardly ever name a variable
> "nonsomething"), which won't be the case with "ref", "share", "common",
> etc.

The problem with 'nonlocal' is not that it's confusing, but that it's 
ugly. And my guess is that there will be enough others that share the 
same sentiment that adopting the word will be difficult.

-- Talin

From talin at acm.org  Fri Oct 13 19:14:48 2006
From: talin at acm.org (Talin)
Date: Fri, 13 Oct 2006 10:14:48 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <2f7601c6eeac$d85fcb20$9a4c2a97@bagio>
References: <452F26A3.7060506@acm.org>
	<2ab101c6ee90$83796470$9a4c2a97@bagio>	<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>
	<2f7601c6eeac$d85fcb20$9a4c2a97@bagio>
Message-ID: <452FC988.5000608@acm.org>

Giovanni Bajo wrote:
>> Therefore, you have to obsolete old stuff if you want there to be
>> only One Obvious Way To Do It.
> 
> I'm totally in favor of obsoletion and removal of old cruft from the standard
> library.
> I'm totally against not having a standard library.

Who said anything about not having a standard library?

Right now, Python, as a community, has essentially no means for library 
cruft removal. In order to keep the large body of existing Python 
programs out there still working, we have to keep every module that's 
ever enjoyed even a modicum of popularity, even when something better 
comes along.

Yes, with Py3K we may be able to remove some things, but even there our 
hands are tied - we can't go around breaking things gratuitously, and a 
"Py3K" opportunity only comes around once every 5-10 years or so, if we 
go by our current (small) sample size of release history.

The idea of "easy_install legacy" essentially gives us the library 
equivalent of "from __past__ import ...". It means that any given user 
can easily reconstruct a version of the library that has all of the 
backwards-compatible modules that they would need to run old code, 
without cluttering the library forevermore.

It also does so in a way that doesn't require those old programs to be 
modified, nor does it require any special command-line options to the 
Python executable. Because it's a modification to the Python environment 
instead, it means that even batch files that invoke Python scripts will 
continue to run as intended. All that's required is that when a user 
upgrades to a new version of Python, they also install (or not, if they 
choose) the legacy packages.

Right now, there's just enough duplication in the library to make even 
experienced programmers like me confused. I often find myself using a 
particularly library function, only to later discover that there's a 
better, more recent version that I didn't know about. For example, I 
wanted to parse command-line options, and I opened up the library index, 
did a search on 'opt', and the first thing I came to was 'getopt', and I 
thought "Oh right, getopt, just like in glibc" and proceed to write my 
code using that. A couple days later, I stumble across "optparse", and 
realize I should have been using that instead - if only I had pressed 
the "find next" button a few more times! The same is true for 'popen' 
vs. 'subprocess' and others. So much for OOWTDI.

-- Talin

From talin at acm.org  Fri Oct 13 19:24:48 2006
From: talin at acm.org (Talin)
Date: Fri, 13 Oct 2006 10:24:48 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <3AE5B28E-97D4-422C-AFBB-5F42893851DB@mac.com>
References: <452F26A3.7060506@acm.org>
	<3AE5B28E-97D4-422C-AFBB-5F42893851DB@mac.com>
Message-ID: <452FCBE0.5050302@acm.org>

Ronald Oussoren wrote:
> P.S. I already have a lightweight version of easy_uninstall, it's called 
> rm :-)

You are kidding right? So in other words, if I do:

    easy_install mercurial

instead of just typing:

    easy_uninstall mercurial

I have to type:

    "rm -rf 
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/mercurial-0.9.1-py2.5-macosx-10.3-fat.egg/"

How is an ordinary user supposed to know how to do this?

Worse yet:

   easy_install turbogears

Now in order to install that, I need to rm 15-20 separate packages. How 
is a user supposed to know how to do this?

-- Talin

From steven.bethard at gmail.com  Fri Oct 13 22:11:55 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 13 Oct 2006 14:11:55 -0600
Subject: [Python-3000] Alternatives to 'outer'
In-Reply-To: <452FC67F.2050207@acm.org>
References: <452F22CB.2030806@acm.org>
	<17700.62.39.9.251.1160727544.squirrel@webmail.nerim.net>
	<452FC67F.2050207@acm.org>
Message-ID: <d11dcfba0610131311s5da7368cvbc5ed1e89102fea@mail.gmail.com>

On 10/13/06, Talin <talin at acm.org> wrote:
> The problem with 'nonlocal' is not that it's confusing, but that it's
> ugly. And my guess is that there will be enough others that share the
> same sentiment that adopting the word will be difficult.

Not at all.  Community sentiment about syntax is just not that
important -- in the end, Guido will pick his favorite version and the
rest of us will go along with it.  (I personally think this is a
wonderful thing because we'd never come to a consensus otherwise.)

What this discussion needs now is not further keyword suggestions, but
a PEP and an implementation.

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From aahz at pythoncraft.com  Fri Oct 13 22:52:28 2006
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 13 Oct 2006 13:52:28 -0700
Subject: [Python-3000] Alternatives to 'outer'
In-Reply-To: <452F22CB.2030806@acm.org>
References: <452F22CB.2030806@acm.org>
Message-ID: <20061013205228.GA12949@panix.com>

Did anyone else think "outies" and "innies" when reading the Subject:
line?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"If you don't know what your program is supposed to do, you'd better not
start writing it."  --Dijkstra

From ntoronto at cs.byu.edu  Fri Oct 13 23:23:29 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Fri, 13 Oct 2006 15:23:29 -0600
Subject: [Python-3000] Alternatives to 'outer'
In-Reply-To: <20061013205228.GA12949@panix.com>
References: <452F22CB.2030806@acm.org> <20061013205228.GA12949@panix.com>
Message-ID: <453003D1.7040200@cs.byu.edu>

Aahz wrote:
> Did anyone else think "outies" and "innies" when reading the Subject:
> line?
>   

The real question is who didn't?

Um, to put this back on track, by analogy, uh... outies are less 
frequent in the general population, just like they should be in code. Or 
something.

Steven Bethard wrote:
> On 10/13/06, Talin <talin at acm.org> wrote:
>   
>> The problem with 'nonlocal' is not that it's confusing, but that it's
>> ugly. And my guess is that there will be enough others that share the
>> same sentiment that adopting the word will be difficult.
>>     
>
> Not at all.  Community sentiment about syntax is just not that
> important -- in the end, Guido will pick his favorite version and the
> rest of us will go along with it.  (I personally think this is a
> wonderful thing because we'd never come to a consensus otherwise.)
>
> What this discussion needs now is not further keyword suggestions, but
> a PEP and an implementation.
>   

+10.

Neil


From brett at python.org  Sat Oct 14 01:04:15 2006
From: brett at python.org (Brett Cannon)
Date: Fri, 13 Oct 2006 16:04:15 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <452FC988.5000608@acm.org>
References: <452F26A3.7060506@acm.org> <2ab101c6ee90$83796470$9a4c2a97@bagio>
	<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>
	<2f7601c6eeac$d85fcb20$9a4c2a97@bagio> <452FC988.5000608@acm.org>
Message-ID: <bbaeab100610131604k2d948b9bw7c5efbe4150c1e87@mail.gmail.com>

On 10/13/06, Talin <talin at acm.org> wrote:
>
> Giovanni Bajo wrote:
> >> Therefore, you have to obsolete old stuff if you want there to be
> >> only One Obvious Way To Do It.
> >
> > I'm totally in favor of obsoletion and removal of old cruft from the
> standard
> > library.
> > I'm totally against not having a standard library.
>
> Who said anything about not having a standard library?
>
> Right now, Python, as a community, has essentially no means for library
> cruft removal. In order to keep the large body of existing Python
> programs out there still working, we have to keep every module that's
> ever enjoyed even a modicum of popularity, even when something better
> comes along.
>
> Yes, with Py3K we may be able to remove some things, but even there our
> hands are tied - we can't go around breaking things gratuitously, and a
> "Py3K" opportunity only comes around once every 5-10 years or so, if we
> go by our current (small) sample size of release history.


Actually, I think we can to a certain extent.  What really needs to happen
is the truly important, used modules stay, and of those that are in need of
a rewrite (e.g., urllib and friends), they get cleaned up appropriately.
Everything else we ditch.  This might require a huge voting on the part of
python-dev through a Py3K PEP listing things to remove compared to keep, but
I think we can definitely prune it down.

The idea of "easy_install legacy" essentially gives us the library
> equivalent of "from __past__ import ...". It means that any given user
> can easily reconstruct a version of the library that has all of the
> backwards-compatible modules that they would need to run old code,
> without cluttering the library forevermore.


That's fine.  If there is a desire to toss up the ripped out modules online
for easy re-installation I don't think you will have major arguments.

It also does so in a way that doesn't require those old programs to be
> modified, nor does it require any special command-line options to the
> Python executable. Because it's a modification to the Python environment
> instead, it means that even batch files that invoke Python scripts will
> continue to run as intended. All that's required is that when a user
> upgrades to a new version of Python, they also install (or not, if they
> choose) the legacy packages.
>
> Right now, there's just enough duplication in the library to make even
> experienced programmers like me confused. I often find myself using a
> particularly library function, only to later discover that there's a
> better, more recent version that I didn't know about. For example, I
> wanted to parse command-line options, and I opened up the library index,
> did a search on 'opt', and the first thing I came to was 'getopt', and I
> thought "Oh right, getopt, just like in glibc" and proceed to write my
> code using that. A couple days later, I stumble across "optparse", and
> realize I should have been using that instead - if only I had pressed
> the "find next" button a few more times! The same is true for 'popen'vs.
> 'subprocess' and others. So much for OOWTDI.


This is a documentation problem of not having the popen docs point you to
subprocess as a better solution.

I think the negative response has been from the feeling that you want to
strip the stdlib lib heavily.  I personally just want to ditch modules that
have very little value to a large portion of our usebase (who uses
sunaudiodev?).  But I do not want important modules to require some
setuptools install to get at.  For marginalized stuff, fine.  And if someone
steps forward to have a "best of breed" listing that people pull from, then
that's fine as well (but that won't be python-dev; see how long it took to
get sqlite added).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061013/c215de72/attachment.htm 

From talin at acm.org  Sat Oct 14 04:25:02 2006
From: talin at acm.org (Talin)
Date: Fri, 13 Oct 2006 19:25:02 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <bbaeab100610131604k2d948b9bw7c5efbe4150c1e87@mail.gmail.com>
References: <452F26A3.7060506@acm.org> <2ab101c6ee90$83796470$9a4c2a97@bagio>	
	<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>	
	<2f7601c6eeac$d85fcb20$9a4c2a97@bagio> <452FC988.5000608@acm.org>
	<bbaeab100610131604k2d948b9bw7c5efbe4150c1e87@mail.gmail.com>
Message-ID: <45304A7E.2050209@acm.org>

Brett Cannon wrote:
> On 10/13/06, Talin <talin at acm.org> wrote:
> I think the negative response has been from the feeling that you want to
> strip the stdlib lib heavily.  I personally just want to ditch modules that
> have very little value to a large portion of our usebase (who uses
> sunaudiodev?).  But I do not want important modules to require some
> setuptools install to get at.  For marginalized stuff, fine.  And if 
> someone
> steps forward to have a "best of breed" listing that people pull from, then
> that's fine as well (but that won't be python-dev; see how long it took to
> get sqlite added).

'Stripping' the standard library is really only a side issue for me. I 
think it would be nice if it didn't get much *bigger* - but what I 
really want most of all is for easy_install (or something like it) to 
always work, with every package. Right now, its actually easier for me 
to hunt down a Windows installer or Mac .dmg for about 50% of the 
packages than it is to install them from the command line. When I 
download something, the last thing I want to spend time doing is 
debugging its setup.py.

-- Talin


From krstic at solarsail.hcs.harvard.edu  Sat Oct 14 05:10:22 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Fri, 13 Oct 2006 23:10:22 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45304A7E.2050209@acm.org>
References: <452F26A3.7060506@acm.org>
	<2ab101c6ee90$83796470$9a4c2a97@bagio>		<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>		<2f7601c6eeac$d85fcb20$9a4c2a97@bagio>
	<452FC988.5000608@acm.org>	<bbaeab100610131604k2d948b9bw7c5efbe4150c1e87@mail.gmail.com>
	<45304A7E.2050209@acm.org>
Message-ID: <4530551E.3000400@solarsail.hcs.harvard.edu>

Talin wrote:
> 'Stripping' the standard library is really only a side issue for me. I 
> think it would be nice if it didn't get much *bigger* - but what I 
> really want most of all is for easy_install (or something like it) to 
> always work, with every package. 

That'd be nice. And on that note, for the love of confused masses
everywhere, can we please change the name of easy_install to something
Python-specific, such as 'egg' or 'py_install'?

easy_install is completely ambiguous (is it a package manager frontend
for the OS? for something else?), and feels the need to remind me just
how easy it is every time I use it -- when I go to type the command name. :)

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From pje at telecommunity.com  Sat Oct 14 05:34:00 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 13 Oct 2006 23:34:00 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <mailman.1100.1160792708.11738.python-3000@python.org>
Message-ID: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>

At 04:25 AM 10/14/2006 +0200, Talin <talin at acm.org> wrote:
>'Stripping' the standard library is really only a side issue for me. I
>think it would be nice if it didn't get much *bigger* - but what I
>really want most of all is for easy_install (or something like it) to
>always work, with every package. Right now, its actually easier for me
>to hunt down a Windows installer or Mac .dmg for about 50% of the
>packages than it is to install them from the command line. When I
>download something, the last thing I want to spend time doing is
>debugging its setup.py.

FYI, easy_install automatically uses distutils-built win32.exe installers 
and converts them to eggs on-the-fly when run on Windows.

With regard to how many packages can work with easy_install right now, the 
answer is tricky.  Of packages added to the Cheeseshop in the last year or 
so, probably more than 50% are easy_install-able or have eggs already 
uploaded.  However, of the extremely popular packages that have existed for 
a very long time and have their own distutils modifications or hacks (e.g. 
Twisted, PIL, pywin32, etc.) probably *more* than 50% *don't* work with 
setuptools, although many have it on their list to investigate and/or 
migrate.  It's just not a big priority for them, as they already have 
solved whatever build-and-install problems they already care about.

For Py3K, of course, there is the question of what happens to distutils and 
setuptools in the first place.  I can't imagine wanting to continue adding 
more layers of cruft and meta-cruft on them, yet can't imagine how they 
could be replaced, either.

I do, however, have my eye on Jim Fulton's zc.buildout tool, as it rather 
seems to me like it could be Python's answer to Ruby's "rake" or Java's 
"Ant".  Most of the other Python-based build tools I have seen (like SCons 
and two or three less-well-known ones whose names escape me) seem to focus 
too much on being build tools for C programs and not enough on being build 
tools.  zc.buildout on the other hand, is almost *too* flexible, but is 
much more modular, and I could see it being a lot easier to maintain e.g. C 
compiler plugins for zc.buildout than trying to keep maintaining the 
distutils' compiler framework indefinitely.

The tricky part of zc.buildout is that it depends on setuptools, at least 
to do egg downloads and installation.  However, there is some possibility 
that the dependency could be inverted, so that setuptools was built on 
zc.buildout instead of the other way around.  But these are pie-in-the-sky 
thoughts for me right now, as there's a lot still to do for Python 2.x, let 
alone thinking about Py3K.


From rasky at develer.com  Sat Oct 14 11:37:33 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 14 Oct 2006 11:37:33 +0200
Subject: [Python-3000] Proposal: No more standard library additions
References: <452F26A3.7060506@acm.org><2ab101c6ee90$83796470$9a4c2a97@bagio>		<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>		<2f7601c6eeac$d85fcb20$9a4c2a97@bagio><452FC988.5000608@acm.org>	<bbaeab100610131604k2d948b9bw7c5efbe4150c1e87@mail.gmail.com><45304A7E.2050209@acm.org>
	<4530551E.3000400@solarsail.hcs.harvard.edu>
Message-ID: <328301c6ef74$60566f80$9a4c2a97@bagio>

Ivan Krsti? wrote:
> Talin wrote:

> That'd be nice. And on that note, for the love of confused masses
> everywhere, can we please change the name of easy_install to something
> Python-specific, such as 'egg' or 'py_install'?

+1. I think at some point I heard mentioned the name 'nest' for a program that
manages eggs. I am not sure whether Philip's wanted it to have the features of
easy_install or not, though.

Giovanni Bajo


From martin at v.loewis.de  Sat Oct 14 16:43:35 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Oct 2006 16:43:35 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <452F26A3.7060506@acm.org>
References: <452F26A3.7060506@acm.org>
Message-ID: <4530F797.1020104@v.loewis.de>

Talin schrieb:
> Unfortunately, the main obstacle at the moment is that about 50% of the 
> packages out there aren't compatible with setuptools. About 50% of the 
> time when I say 'easy_install <bleargh>' it downloads the package and 
> then says 'setup.py' not found.

I disagree. The main obstacle is that distutils is unmaintained. I would
not like to trust deployment of the standard library to an unmaintained
library.

So if you want to do something really useful, start maintaining
distutils. Look into the most popular extensions (probably starting
with setuptools, then going on to the extensions that high-ranking
packages make), and see what features are missing in distutils, and
try to integrate them. Do so in cooperation with the authors of these
extensions, but don't incorporate blindly, instead, apply a design
that likely is useful for more than one package.

Define distutils features that are obsolete, and come up with
a strategy of removing or replacing them.

When you are done, you can come back proposing that the library
should be made out of distutils packages.

Regards,
Martin

From martin at v.loewis.de  Sat Oct 14 18:41:25 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Oct 2006 18:41:25 +0200
Subject: [Python-3000] Pre-PEP: Easy Text File Decoding
In-Reply-To: <4505F149.8030509@blueyonder.co.uk>
References: <1cb725390609092029v6dae753ajf8bf85e4974bb4d0@mail.gmail.com>		<1157884285.4246.41.camel@fsol>
	<4503FDC8.2030608@gmail.com>		<1157892435.4246.107.camel@fsol>	<450418AC.2010400@blueyonder.co.uk>		<1cb725390609101202x644fe40fm1e234b564890c8d5@mail.gmail.com>		<45047FC2.40904@blueyonder.co.uk>	<1cb725390609102111u44287761i8b509729aa6f5ce1@mail.gmail.com>
	<4505F149.8030509@blueyonder.co.uk>
Message-ID: <45311335.2010704@v.loewis.de>

David Hopwood schrieb:
>> The PEP doesn't deal with streams. It is about files.
> 
> An important part of the Unix design philosophy (partially adopted by Windows)
> is to make streams and files behave as similarly as possible. It is quite
> feasible to make *some* detection algorithms work for streams, and this is
> an advantage over algorithms that don't work for streams.

There is nothing wrong with that principle. But it's just a principle.
Depending on the kind of stream, different technologies for detecting
the encoding are necessary. For example, on a Win32 terminal, a special
API is available to determine the encoding. On a Unix interactive
terminal, the locale's encoding should be assumed (and more so than for
regular files: for a file, the user may chose a different encoding;
for the console, the encoding is close to (sic) a hardware setting that
cannot be changed).

The Unix principle is often extended to reach into areas where it really
becomes inadequate, look at Plan 9 for an example. Wrt. byte streams
and encodings, the principle works only if all applications use the
same encoding (which they do on Plan 9).

Regards,
Martin

From pje at telecommunity.com  Sat Oct 14 18:46:15 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 14 Oct 2006 12:46:15 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <mailman.51.1160820014.31100.python-3000@python.org>
Message-ID: <5.1.1.6.0.20061014123310.03c90298@sparrow.telecommunity.com>

At 12:00 PM 10/14/2006 +0200, "Giovanni Bajo" <rasky at develer.com> wrote:
>Ivan Krsti? wrote:
> > Talin wrote:
>
> > That'd be nice. And on that note, for the love of confused masses
> > everywhere, can we please change the name of easy_install to something
> > Python-specific, such as 'egg' or 'py_install'?
>
>+1. I think at some point I heard mentioned the name 'nest' for a program that
>manages eggs. I am not sure whether Philip's wanted it to have the features of
>easy_install or not, though.

Yes.  "nest" will be an overall command similar to "yum" or other package 
managers of its ilk, with subcommands like "nest install" that will work 
like easy_install.

However, "nest install" will not have easy_install's arcane option set, nor 
will it be able to do all the things easy_install does.  Instead, things 
like "easy_install -eb. packagename" (fetch and unpack a project's sources 
to a subdirectory) will become something like "nest source packagename", so 
that there are separate commands for different use cases instead of one 
arcane super-command.

Also, instead of using options to specify installation locations, it's 
likely that nest will have, well, "nests", which are configured locations 
for installing packages, scripts, source, headers, etc.  So at most you 
would specify what nest to use when invoking a command.  But there will 
also be a "nest use nestname" command that will invoke a subshell in which 
the given nest will be the default.

But before anybody asks, I have no clue when I'll even be able to start on 
writing nest, let alone finishing it.  I'm still shoring up leaks in the 
abstractions of setuptools as it sits; for example, last night an OSAF 
developer discovered that the 'ext_package' option to setup() (which I'd 
forgotten even existed) didn't work right with setuptools' "test" 
feature.  Distutils is *very* very big.  :(  If I'd known what I was 
getting into before I started, I'm not sure I would've done it.

Distutils itself is in sore need of a 3K-ing.


From martin at v.loewis.de  Sat Oct 14 18:47:39 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Oct 2006 18:47:39 +0200
Subject: [Python-3000] content-based detection
In-Reply-To: <1157918276.4257.30.camel@fsol>
References: <1cb725390609092029v6dae753ajf8bf85e4974bb4d0@mail.gmail.com>	<1157884285.4246.41.camel@fsol>
	<4503FDC8.2030608@gmail.com>	<1157892435.4246.107.camel@fsol>	<1cb725390609101130w73f7a12bs243d8d6548b7b2d8@mail.gmail.com>
	<1157918276.4257.30.camel@fsol>
Message-ID: <453114AB.1000108@v.loewis.de>

Antoine Pitrou schrieb:
> I doubt many people have to discriminate between UTF-16LE, UCS-4 and
> UTF-8. Are there real cases like that for text files?

Not sure what "that" is, in this question. There are certainly UTF-8
files. There are also UTF-16 files; they typically are LE, and often
have a BOM. I've never seen an UCS-4 file "in the wild" (except for
testing, of course).

Regards,
Martin

From martin at v.loewis.de  Sat Oct 14 19:03:21 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Oct 2006 19:03:21 +0200
Subject: [Python-3000] Pre-PEP: Easy Text File Decoding
In-Reply-To: <8764fu3cxy.fsf@qrnik.zagroda>
References: <1cb725390609092029v6dae753ajf8bf85e4974bb4d0@mail.gmail.com>	<87d5a366xd.fsf@qrnik.zagroda>	<1cb725390609102142m3a1dd33ha8400ec8e2005ba@mail.gmail.com>
	<8764fu3cxy.fsf@qrnik.zagroda>
Message-ID: <45311859.6080102@v.loewis.de>

Marcin 'Qrczak' Kowalczyk schrieb:
> I've implemented a hack which allows simple programs to "just work" in
> case of UTF-8. It's a modified encoder/decoder which escapes malformed
> UTF-8 sequences with '\0' bytes, and thus allows arbitrary byte
> sequences to round-trip UTF-8 decoding and encoding. It's not used by
> default and it's never used when "UTF-8" is specified explicitly,
> because it's not the true UTF-8, but I have an environment variable
> which says "if the locale is UTF-8, use the modified UTF-8 as the
> default encoding".

Actually, I think there is a "better" (i.e. more unicode-like way):
use the private-use area. For "wide" Unicode, chose some "high"
characters, e.g. from plane 16 (say, U+1020xx). For "narrow"
Unicode, chose some from the "middle" (say, U+F4xx). There is
a slight chance of ambiguity here if the actual input also
contains such PUA characters; if you worry about this, you could
escape those.

For Py3k, I would like to propose a standard "binary" codec,
which is an ASCII superset and decodes bytes 00..7F to ASCII,
and bytes 80..FF to U+EFxx. This would allow to round-trip
bytes through text.

Regards,
Martin

From rasky at develer.com  Sat Oct 14 19:10:31 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 14 Oct 2006 19:10:31 +0200
Subject: [Python-3000] Proposal: No more standard library additions
References: <5.1.1.6.0.20061014123310.03c90298@sparrow.telecommunity.com>
Message-ID: <004b01c6efb3$a7629cc0$e303030a@trilan>

Phillip J. Eby wrote:

> Yes.  "nest" will be an overall command similar to "yum" or other
> package managers of its ilk, with subcommands like "nest install"
> that will work like easy_install.

Yes this will be great.

> But before anybody asks, I have no clue when I'll even be able to
> start on writing nest, let alone finishing it.

I think you could setup a wiki page with the design of the nest command.
After having discussed that, other people might be able to help with the
implementation.

> Distutils itself is in sore need of a 3K-ing.

Agreed, and your input as to what parts of distuils/setuptools should be
dropped would be mostly appreciated. Again, a wiki page or a PEP is in
order.
-- 
Giovanni Bajo


From qrczak at knm.org.pl  Sat Oct 14 20:45:54 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Sat, 14 Oct 2006 20:45:54 +0200
Subject: [Python-3000] Pre-PEP: Easy Text File Decoding
In-Reply-To: <45311859.6080102@v.loewis.de> (Martin v.
	=?iso-8859-2?q?L=F6wis's?= message of "Sat, 14 Oct 2006 19:03:21 +0200")
References: <1cb725390609092029v6dae753ajf8bf85e4974bb4d0@mail.gmail.com>
	<87d5a366xd.fsf@qrnik.zagroda>
	<1cb725390609102142m3a1dd33ha8400ec8e2005ba@mail.gmail.com>
	<8764fu3cxy.fsf@qrnik.zagroda> <45311859.6080102@v.loewis.de>
Message-ID: <87odsed9ct.fsf@qrnik.zagroda>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Marcin 'Qrczak' Kowalczyk schrieb:
>> I've implemented a hack which allows simple programs to "just work" in
>> case of UTF-8. It's a modified encoder/decoder which escapes malformed
>> UTF-8 sequences with '\0' bytes, and thus allows arbitrary byte
>> sequences to round-trip UTF-8 decoding and encoding. It's not used by
>> default and it's never used when "UTF-8" is specified explicitly,
>> because it's not the true UTF-8, but I have an environment variable
>> which says "if the locale is UTF-8, use the modified UTF-8 as the
>> default encoding".
>
> Actually, I think there is a "better" (i.e. more unicode-like way):
> use the private-use area.

It changes the interpretation of some filenames which are valid UTF-8
(or generally of texts known to not contain '\0'). My hack is a pure
extension since U+0000 can't be produced by standard UTF-8.

> For Py3k, I would like to propose a standard "binary" codec,
> which is an ASCII superset and decodes bytes 00..7F to ASCII,
> and bytes 80..FF to U+EFxx. This would allow to round-trip
> bytes through text.

It's simpler to use the existing ISO-8859-1 encoding.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From martin at v.loewis.de  Sat Oct 14 20:54:06 2006
From: martin at v.loewis.de (=?ISO-8859-2?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Oct 2006 20:54:06 +0200
Subject: [Python-3000] Pre-PEP: Easy Text File Decoding
In-Reply-To: <87odsed9ct.fsf@qrnik.zagroda>
References: <1cb725390609092029v6dae753ajf8bf85e4974bb4d0@mail.gmail.com>	<87d5a366xd.fsf@qrnik.zagroda>	<1cb725390609102142m3a1dd33ha8400ec8e2005ba@mail.gmail.com>	<8764fu3cxy.fsf@qrnik.zagroda>
	<45311859.6080102@v.loewis.de> <87odsed9ct.fsf@qrnik.zagroda>
Message-ID: <4531324E.4040206@v.loewis.de>

Marcin 'Qrczak' Kowalczyk schrieb:
> It changes the interpretation of some filenames which are valid UTF-8
> (or generally of texts known to not contain '\0'). My hack is a pure
> extension since U+0000 can't be produced by standard UTF-8.

That's not true. See RFC 2279:

# Character values from 0000 0000 to 0000 007F (US-ASCII repertoire)
# correspond to octets 00 to 7F (7 bit US-ASCII values).

So U+0000 is represented by the octet 00.

Regards,
Martin

From pje at telecommunity.com  Sat Oct 14 20:54:58 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 14 Oct 2006 14:54:58 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <004b01c6efb3$a7629cc0$e303030a@trilan>
References: <5.1.1.6.0.20061014123310.03c90298@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061014144459.027f6250@sparrow.telecommunity.com>

At 07:10 PM 10/14/2006 +0200, Giovanni Bajo wrote:
>Phillip J. Eby wrote:
>
> > Yes.  "nest" will be an overall command similar to "yum" or other
> > package managers of its ilk, with subcommands like "nest install"
> > that will work like easy_install.
>
>Yes this will be great.
>
> > But before anybody asks, I have no clue when I'll even be able to
> > start on writing nest, let alone finishing it.
>
>I think you could setup a wiki page with the design of the nest command.
>After having discussed that, other people might be able to help with the
>implementation.
>
> > Distutils itself is in sore need of a 3K-ing.
>
>Agreed, and your input as to what parts of distuils/setuptools should be
>dropped would be mostly appreciated. Again, a wiki page or a PEP is in
>order.

Yes, well, first I'd have to have the time for it, relative to other 
priorities.  See above.  :)

I would note, however, that the thing that's probably most needed is an 
analysis of what features the producers of major packages with distutils 
extensions (Twisted, Numeric, wxPython, etc.) need in order to be able to 
drop their extensions.  I think that maintaining API compatibility for 
setup scripts that don't actually extend the distutils is a reasonable 
possibility.  Stuff that extends or monkeypatches the distutils isn't.  So, 
we would need to get the hooks in to allow people to do the things they 
need without patching and subclassing.

Setuptools has already shown the way forward in this respect with entry 
points.  It's trivial to register new distutils commands, setup() 
arguments, and even revision control handlers for setuptools to use, and I 
can foresee that a distutils 3K could break down highly-coupled tools like 
install and its install_* subcommands to use entry points instead, so that 
you could e.g. add stages to a build step without needing to subclass anything.

So I guess I would say that the thing that most needs to go from the 
distutils, is extension by subclassing -- to be replaced by extension via 
entry points.

Note that this would also mean things like allowing compiler types to be 
registered via entry points, so that support for different C compilers 
wouldn't need to be hacked in.  Thus, if people wanted to implement 
cross-compile support, they could do it by implementing a compiler class 
and registering it as an entry point.  They could then distribute an egg 
containing that cross-compiler support class, and anyone could just install 
it to add a new compiler type to their distutils installation.


From rrr at ronadam.com  Sat Oct 14 21:30:15 2006
From: rrr at ronadam.com (Ron Adam)
Date: Sat, 14 Oct 2006 14:30:15 -0500
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <5.1.1.6.0.20061014123310.03c90298@sparrow.telecommunity.com>
References: <mailman.51.1160820014.31100.python-3000@python.org>
	<5.1.1.6.0.20061014123310.03c90298@sparrow.telecommunity.com>
Message-ID: <egre2s$oq1$1@sea.gmane.org>

Phillip J. Eby wrote:
> At 12:00 PM 10/14/2006 +0200, "Giovanni Bajo" <rasky at develer.com> wrote:
>> Ivan Krsti? wrote:
>>> Talin wrote:
>>> That'd be nice. And on that note, for the love of confused masses
>>> everywhere, can we please change the name of easy_install to something
>>> Python-specific, such as 'egg' or 'py_install'?
>> +1. I think at some point I heard mentioned the name 'nest' for a program that
>> manages eggs. I am not sure whether Philip's wanted it to have the features of
>> easy_install or not, though.
> 
> Yes.  "nest" will be an overall command similar to "yum" or other package 
> managers of its ilk, with subcommands like "nest install" that will work 
> like easy_install.
> 
> However, "nest install" will not have easy_install's arcane option set, nor 
> will it be able to do all the things easy_install does.  Instead, things 
> like "easy_install -eb. packagename" (fetch and unpack a project's sources 
> to a subdirectory) will become something like "nest source packagename", so 
> that there are separate commands for different use cases instead of one 
> arcane super-command.
> 
> Also, instead of using options to specify installation locations, it's 
> likely that nest will have, well, "nests", which are configured locations 
> for installing packages, scripts, source, headers, etc.  So at most you 
> would specify what nest to use when invoking a command.  But there will 
> also be a "nest use nestname" command that will invoke a subshell in which 
> the given nest will be the default.
> 
> But before anybody asks, I have no clue when I'll even be able to start on 
> writing nest, let alone finishing it.  I'm still shoring up leaks in the 
> abstractions of setuptools as it sits; for example, last night an OSAF 
> developer discovered that the 'ext_package' option to setup() (which I'd 
> forgotten even existed) didn't work right with setuptools' "test" 
> feature.  Distutils is *very* very big.  :(  If I'd known what I was 
> getting into before I started, I'm not sure I would've done it.
> 
> Distutils itself is in sore need of a 3K-ing.


Will "nest" (or what ever it's called) be a single module to handle everything? 
or will it be a front-end to distutils?

It would probably be a *lot* easier to write and maintain if it were a simple 
front end to disutils and is used to inspect eggs (via lists) and having 
disutils do the work of installing or un-installing them.

Ideally each egg would know what it is and where it belongs (*) and so it could 
then only be a matter of getting the install info from the eggs:

   description
   author/owner/home-page
   license
   www install-from location  (to use if egg is not here)
   install-to location
   external dependencies (other eggs (or software) needed)

... and then sorting them accordingly... installed, not installed, pythonlib, 
third party package, etc.

If it were a gui app, then I envision having items highlighted in green if all 
the external dependencies (other eggs) are already installed and red if they 
aren't.  Things like that could make life easier for many people I think.


(*) It's install location might be either a default, relative directory, or an 
environment variable name.

_Ron


From qrczak at knm.org.pl  Sat Oct 14 22:15:21 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Sat, 14 Oct 2006 22:15:21 +0200
Subject: [Python-3000] Pre-PEP: Easy Text File Decoding
In-Reply-To: <4531324E.4040206@v.loewis.de> (Martin v.
	=?iso-8859-2?q?L=F6wis's?= message of "Sat, 14 Oct 2006 20:54:06 +0200")
References: <1cb725390609092029v6dae753ajf8bf85e4974bb4d0@mail.gmail.com>
	<87d5a366xd.fsf@qrnik.zagroda>
	<1cb725390609102142m3a1dd33ha8400ec8e2005ba@mail.gmail.com>
	<8764fu3cxy.fsf@qrnik.zagroda> <45311859.6080102@v.loewis.de>
	<87odsed9ct.fsf@qrnik.zagroda> <4531324E.4040206@v.loewis.de>
Message-ID: <87zmbyejs6.fsf@qrnik.zagroda>

"Martin v. L?wis" <martin at v.loewis.de> writes:

>> It changes the interpretation of some filenames which are valid UTF-8
                                         ^^^^^^^^^
>> (or generally of texts known to not contain '\0'). My hack is a pure
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> extension since U+0000 can't be produced by standard UTF-8.
>
> That's not true. See RFC 2279:
>
> # Character values from 0000 0000 to 0000 007F (US-ASCII repertoire)
> # correspond to octets 00 to 7F (7 bit US-ASCII values).
>
> So U+0000 is represented by the octet 00.

'\0' (and thus U+0000) can't appear in Unix filenames, in names or
values of environment variables, in program invocation arguments etc.

It is true that it can change the interpretation of file contents.
This is unavoidable. Unless someone uses unpaired surrogates for this
purpose (or code points above U+10FFFF) - I've seen such proposals,
but IMHO they are abusing rules too far.

Anyway, Unicode people don't like my hack (it was really inspired by
Mono). They dislike any modifications of UTF-8, perhaps because such
modifications invite incompatibilities when different software use
different variants. I understand this, and I don't want software to
start exchanging data in such "encoding", but IMHO it's a useful hack
when the given language runtime translates filenames to UTF-8: it's
often the case that the interpretation of characters in filenames
doesn't matter, only preserving the byte sequences.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From talin at acm.org  Sun Oct 15 00:52:21 2006
From: talin at acm.org (Talin)
Date: Sat, 14 Oct 2006 15:52:21 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <328301c6ef74$60566f80$9a4c2a97@bagio>
References: <452F26A3.7060506@acm.org><2ab101c6ee90$83796470$9a4c2a97@bagio>		<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>		<2f7601c6eeac$d85fcb20$9a4c2a97@bagio><452FC988.5000608@acm.org>	<bbaeab100610131604k2d948b9bw7c5efbe4150c1e87@mail.gmail.com><45304A7E.2050209@acm.org>	<4530551E.3000400@solarsail.hcs.harvard.edu>
	<328301c6ef74$60566f80$9a4c2a97@bagio>
Message-ID: <45316A25.7020708@acm.org>

Giovanni Bajo wrote:
> Ivan Krsti? wrote:
>> Talin wrote:
> 
>> That'd be nice. And on that note, for the love of confused masses
>> everywhere, can we please change the name of easy_install to something
>> Python-specific, such as 'egg' or 'py_install'?
> 
> +1. I think at some point I heard mentioned the name 'nest' for a program that
> manages eggs. I am not sure whether Philip's wanted it to have the features of
> easy_install or not, though.

I think I'd prefer something a little less clever and obscure, like 
maybe "pypackage". (Or pypkg for lazy typers.)

-- Talin

From greg.ewing at canterbury.ac.nz  Sun Oct 15 00:59:45 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 15 Oct 2006 11:59:45 +1300
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4530551E.3000400@solarsail.hcs.harvard.edu>
References: <452F26A3.7060506@acm.org> <2ab101c6ee90$83796470$9a4c2a97@bagio>
	<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>
	<2f7601c6eeac$d85fcb20$9a4c2a97@bagio> <452FC988.5000608@acm.org>
	<bbaeab100610131604k2d948b9bw7c5efbe4150c1e87@mail.gmail.com>
	<45304A7E.2050209@acm.org> <4530551E.3000400@solarsail.hcs.harvard.edu>
Message-ID: <45316BE1.8060209@canterbury.ac.nz>

Ivan Krsti? wrote:

> That'd be nice. And on that note, for the love of confused masses
> everywhere, can we please change the name of easy_install to something
> Python-specific, such as 'egg' or 'py_install'?

I don't think that easy_install support is going to
become universal until its functionality is folded
into distutils and it becomes part of the standard
distribution. Expecting everyone to learn about and
support a particular 3rd party addition is just not
realistic.

--
Greg

From greg.ewing at canterbury.ac.nz  Sun Oct 15 01:08:44 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 15 Oct 2006 12:08:44 +1300
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
Message-ID: <45316DFC.8020006@canterbury.ac.nz>

Phillip J. Eby wrote:
> I could see it being a lot easier to maintain e.g. C 
> compiler plugins for zc.buildout than trying to keep maintaining the 
> distutils' compiler framework indefinitely.

This sounds good. We definitely need *something* with
a more modular and better documented architecture than
distutils.

--
Greg

From martin at v.loewis.de  Sun Oct 15 08:16:42 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Oct 2006 08:16:42 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45316DFC.8020006@canterbury.ac.nz>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
Message-ID: <4531D24A.1060905@v.loewis.de>

Greg Ewing schrieb:
>> I could see it being a lot easier to maintain e.g. C 
>> compiler plugins for zc.buildout than trying to keep maintaining the 
>> distutils' compiler framework indefinitely.
> 
> This sounds good. We definitely need *something* with
> a more modular and better documented architecture than
> distutils.

I definitely disagree. Why are you calling distutils non-modular?
It was designed to be modular and extensible, and, indeed, it is.
If it is not documented well, the solution shouldn't be to replace
it, but instead to document it.

Regards,
Martin

From martin at v.loewis.de  Sun Oct 15 08:26:29 2006
From: martin at v.loewis.de (=?ISO-8859-2?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Oct 2006 08:26:29 +0200
Subject: [Python-3000] Pre-PEP: Easy Text File Decoding
In-Reply-To: <87zmbyejs6.fsf@qrnik.zagroda>
References: <1cb725390609092029v6dae753ajf8bf85e4974bb4d0@mail.gmail.com>	<87d5a366xd.fsf@qrnik.zagroda>	<1cb725390609102142m3a1dd33ha8400ec8e2005ba@mail.gmail.com>	<8764fu3cxy.fsf@qrnik.zagroda>
	<45311859.6080102@v.loewis.de>	<87odsed9ct.fsf@qrnik.zagroda>
	<4531324E.4040206@v.loewis.de> <87zmbyejs6.fsf@qrnik.zagroda>
Message-ID: <4531D495.5030909@v.loewis.de>

Marcin 'Qrczak' Kowalczyk schrieb:
> It is true that it can change the interpretation of file contents.
> This is unavoidable. Unless someone uses unpaired surrogates for this
> purpose (or code points above U+10FFFF) - I've seen such proposals,
> but IMHO they are abusing rules too far.

It's not exactly unavoidable: any escaping mechanism can support the
full range of valid input. In your escaping mechanism, you could
duplicate 0 bytes on decoding, and write a null byte if you have two
subsequent NUL characters on encoding.

I still think that PUA characters would be a better use: in your
encoding, you get two characters of encoded text for one byte of
input; if people need to render the file name, this will be confusing.
With a PUA character, rendering will still produce moji-bake, but
you will likely get one "box" of output for what the user thinks
should be one character.

Refining my last proposal: I think there should be a "pass-through"
error handler for codecs which puts undecodable bytes into PUA
characters, and encodes unencodable characters from the PUA range
into the corresponding bytes. This could lie on top of existing
codecs, and help to decode undecodable file names in a way
that round-trips.

Regards,
Martin

From pje at telecommunity.com  Sun Oct 15 08:46:07 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 15 Oct 2006 02:46:07 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4531D24A.1060905@v.loewis.de>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
Message-ID: <5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>

At 08:16 AM 10/15/2006 +0200, Martin v. L?wis wrote:
>Greg Ewing schrieb:
> >> I could see it being a lot easier to maintain e.g. C
> >> compiler plugins for zc.buildout than trying to keep maintaining the
> >> distutils' compiler framework indefinitely.
> >
> > This sounds good. We definitely need *something* with
> > a more modular and better documented architecture than
> > distutils.
>
>I definitely disagree. Why are you calling distutils non-modular?
>It was designed to be modular and extensible,

No, it's designed to be subclassable and customizable.  That is very 
different from being extensible.  setuptools is somewhat 
extensible.  zc.buildout is very extensible.  But the distutils are not 
extensible.  The documentation or lack thereof has nothing to do with it.

"Modular" is a red herring.  The distutils isn't a bad framework, for its 
time.  But it was not designed with extensibility (vs. customizability) in 
mind.

The difference between the two is that an extensible system is one that 
follows the same rules for extenders as it does for its core 
developers.  For example, in the distutils, there are two ways to register 
commands: one for extenders, and one for core developers.  (Actually, there 
are three, as there is yet another way added in Python 2.4 for third 
parties to add commands as well.)

Setuptools, in contrast, has one preferred way to register commands, and it 
follows that way itself.  Core, extenders, and third parties all use the 
same mechanism, so it only needs to be documented once, and the platform is 
more open.  (For backward compatibility, of course, it has to support all 
the other ways that distutils does it.)

Another example is compiler support.  Using the techniques of setuptools 
and zc.buildout, we could allow anybody to write support for a particular C 
compiler abstraction, to e.g. support cross-compilation.  They could 
distribute that support separately and would not have to be a distutils 
developer to do so.  Other people could use that support for their own 
projects without having to bundle it in, and create the massive distutils 
"extensions" used by many larger projects today.

The lack of ability to do this is a characteristic design limitation of the 
distutils (because only core developers of distutils can add new compilers, 
archive formats, etc.).

That doesn't mean the distutils are "bad", it just means that the 
distutils' architecture is not an "architecture of participation".  When a 
system's architecture is designed for collaboration (as opposed to mere 
customization via subclassing) it will have more collaboration happening 
around it.

Two of the biggest Python "community" development trends in recent years, 
setuptools and WSGI, are good demonstrations of how a design can be used to 
promote collaboration, rather than balkanization.  The distutils' design 
promotes balkanization through project-specific customization, but this can 
be turned around with a better design that allows more people to 
participate in development and extension.


From g.brandl at gmx.net  Sun Oct 15 09:16:59 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 15 Oct 2006 09:16:59 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45316BE1.8060209@canterbury.ac.nz>
References: <452F26A3.7060506@acm.org>
	<2ab101c6ee90$83796470$9a4c2a97@bagio>	<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>	<2f7601c6eeac$d85fcb20$9a4c2a97@bagio>
	<452FC988.5000608@acm.org>	<bbaeab100610131604k2d948b9bw7c5efbe4150c1e87@mail.gmail.com>	<45304A7E.2050209@acm.org>
	<4530551E.3000400@solarsail.hcs.harvard.edu>
	<45316BE1.8060209@canterbury.ac.nz>
Message-ID: <egsn9b$qok$1@sea.gmane.org>

Greg Ewing wrote:
> Ivan Krsti? wrote:
> 
>> That'd be nice. And on that note, for the love of confused masses
>> everywhere, can we please change the name of easy_install to something
>> Python-specific, such as 'egg' or 'py_install'?
> 
> I don't think that easy_install support is going to
> become universal until its functionality is folded
> into distutils and it becomes part of the standard
> distribution. Expecting everyone to learn about and
> support a particular 3rd party addition is just not
> realistic.

The Python 3 timeframe should be reasonably wide to allow setuptools and
distutils becoming one and being integrated in the stdlib.

Georg


From ronaldoussoren at mac.com  Sun Oct 15 09:27:01 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 15 Oct 2006 09:27:01 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45316BE1.8060209@canterbury.ac.nz>
References: <452F26A3.7060506@acm.org> <2ab101c6ee90$83796470$9a4c2a97@bagio>
	<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>
	<2f7601c6eeac$d85fcb20$9a4c2a97@bagio> <452FC988.5000608@acm.org>
	<bbaeab100610131604k2d948b9bw7c5efbe4150c1e87@mail.gmail.com>
	<45304A7E.2050209@acm.org>
	<4530551E.3000400@solarsail.hcs.harvard.edu>
	<45316BE1.8060209@canterbury.ac.nz>
Message-ID: <2DE9DE61-6D77-4D0D-B623-B513FFF63E96@mac.com>


On Oct 15, 2006, at 12:59 AM, Greg Ewing wrote:

> Ivan Krsti? wrote:
>
>> That'd be nice. And on that note, for the love of confused masses
>> everywhere, can we please change the name of easy_install to  
>> something
>> Python-specific, such as 'egg' or 'py_install'?
>
> I don't think that easy_install support is going to
> become universal until its functionality is folded
> into distutils and it becomes part of the standard
> distribution. Expecting everyone to learn about and
> support a particular 3rd party addition is just not
> realistic.

Even then easy_install support won't be univeral. I still run into  
packages that don't use distutils, mostly stuff that is or was part  
of a larger C code base.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061015/d407045a/attachment.bin 

From rasky at develer.com  Sun Oct 15 12:55:05 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 15 Oct 2006 12:55:05 +0200
Subject: [Python-3000] Proposal: No more standard library additions
References: <45316DFC.8020006@canterbury.ac.nz><5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com><45316DFC.8020006@canterbury.ac.nz>
	<4531D24A.1060905@v.loewis.de>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
Message-ID: <366f01c6f048$5f5b2c40$9a4c2a97@bagio>

Phillip J. Eby wrote:

>>> This sounds good. We definitely need *something* with
>>> a more modular and better documented architecture than
>>> distutils.
>>
>> I definitely disagree. Why are you calling distutils non-modular?
>> It was designed to be modular and extensible,
>
> No, it's designed to be subclassable and customizable.  That is very
> different from being extensible.  setuptools is somewhat
> extensible.  zc.buildout is very extensible.  But the distutils are
> not extensible.  The documentation or lack thereof has nothing to do
> with it.
>
> "Modular" is a red herring.  The distutils isn't a bad framework, for
> its time.  But it was not designed with extensibility (vs.
> customizability) in mind.
>
> The difference between the two is that an extensible system is one
> that follows the same rules for extenders as it does for its core
> developers.

Being among the ones who have fought many times against the lack of a proper
extensible design in distutils, I can't but agree.

Giovanni Bajo


From martin at v.loewis.de  Sun Oct 15 13:41:55 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Oct 2006 13:41:55 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
Message-ID: <45321E83.2030401@v.loewis.de>

Phillip J. Eby schrieb:
> The difference between the two is that an extensible system is one that
> follows the same rules for extenders as it does for its core
> developers.  For example, in the distutils, there are two ways to
> register commands: one for extenders, and one for core developers. 
> (Actually, there are three, as there is yet another way added in Python
> 2.4 for third parties to add commands as well.)

It's only a terminology issue, but I think most people would assume
a different definition for "extensible". I'd call a system extensible
if it supports being extended, period. By my definition, distutils
is extensible; by your definition, it's not.

> Another example is compiler support.  Using the techniques of setuptools
> and zc.buildout, we could allow anybody to write support for a
> particular C compiler abstraction, to e.g. support cross-compilation. 
> They could distribute that support separately and would not have to be a
> distutils developer to do so.  Other people could use that support for
> their own projects without having to bundle it in, and create the
> massive distutils "extensions" used by many larger projects today.

Actually, it is possible to add a new compiler to some Python
installation, so that users can pass it to the --compiler option,
and distutils packages don't have to be modified for that.
You just have to extend the dictionary
distutils.ccompiler.compiler_class.

> Two of the biggest Python "community" development trends in recent
> years, setuptools and WSGI, are good demonstrations of how a design can
> be used to promote collaboration, rather than balkanization.  The
> distutils' design promotes balkanization through project-specific
> customization, but this can be turned around with a better design that
> allows more people to participate in development and extension.

I don't object to improving the design. I object to dropping distutils
and replacing it with something else. If certain functionality is
missing in distutils (e.g. means for more transparent extension), then
distutils should be enhanced to provide this functionality.

Regards,
Martin


From pje at telecommunity.com  Sun Oct 15 18:01:30 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 15 Oct 2006 12:01:30 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45321E83.2030401@v.loewis.de>
References: <5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>

At 01:41 PM 10/15/2006 +0200, Martin v. L?wis wrote:
>Phillip J. Eby schrieb:
> > The difference between the two is that an extensible system is one that
> > follows the same rules for extenders as it does for its core
> > developers.  For example, in the distutils, there are two ways to
> > register commands: one for extenders, and one for core developers.
> > (Actually, there are three, as there is yet another way added in Python
> > 2.4 for third parties to add commands as well.)
>
>It's only a terminology issue, but I think most people would assume
>a different definition for "extensible". I'd call a system extensible
>if it supports being extended, period. By my definition, distutils
>is extensible; by your definition, it's not.

Yes, and the people who complain about the distutils' design are using my 
definition or something like it.  Even by your definition it's sometimes 
very difficult to extend, because there is a tendency in the distutils to 
have page-long methods that encode an entire sequence of operations that 
cannot be customized without copying and modifying the whole block of 
code.  Command methods also often have side effects, which is something 
that the distutils encourages by its architecture.

(The architecture is also poor from a testability perspective, which limits 
our ability to simply refactor and improve the architecture.)

Last, but not least, the distutils suffer from an excess of policy 
flexibility in areas where flexibility is undesirable from a community 
perspective.  There should be "one obvious way" to do a lot of things that 
the distutils provide a ridiculous number of ways to do (e.g. insane 
project directory layouts including package-by-package directory 
mapping).  This is not something that affects the distutils themselves 
much, but it essentially prevents the distutils from improving in certain 
ways because there are so few assumptions one can make about its use.


>Actually, it is possible to add a new compiler to some Python
>installation, so that users can pass it to the --compiler option,
>and distutils packages don't have to be modified for that.
>You just have to extend the dictionary
>distutils.ccompiler.compiler_class.

Yes, and how do you accomplish *that*?  You have to either modify some 
Python code (e.g. sitecustomize) or use a big -c to python when running the 
setup script.

With entry point extension, it is only necessary to have an extension 
installed on sys.path; it is not necessary to run some code to do the 
registration.

In the setuptools case, if you want to add setup commands to your Python 
installation, you simply install distributions containing those commands, 
and they become available automatically for any setup script using 
setuptools.  If you want to add setup() arguments, same thing.  Revision 
control plugins, likewise.  Egg metadata file generator plugins 
also.  Essentially, anything that someone can think of a use case to allow 
plugins for, it is a simple change to add.


>I don't object to improving the design. I object to dropping distutils
>and replacing it with something else. If certain functionality is
>missing in distutils (e.g. means for more transparent extension), then
>distutils should be enhanced to provide this functionality.

Adding setuptools-style plugin ability to the distutils is very simple, and 
doesn't require a wholesale reorganization.  After all, setuptools manages 
to do it!  However, there are areas of build functionality that the 
distutils does not cover or provides incomplete coverage for, such as 
configuration and build of more complex C libraries and programs, shared 
libraries, documentation production, and so on, that the existing 
architecture of the distutils doesn't really encompass.

The current design of the distutils has essentially two layers: one very 
high-level layer dealing with command objects, and a very low-level layer 
that deals with copying files or trees thereof, making archives, and 
various other simple tasks.  The low-level layer is often infested with 
assumptions, however, that are propagated from higher layers.  Meanwhile, 
the higher-level layer is based on the idea of dependency between 
sequentially invoked commands, rather than dependency between things being 
built.  This makes it difficult to deal with concepts like having a bunch 
of reST release note docs combined with a LaTeX documentation build.

So, distutils' extensibility *could* be improved dramatically in the 2.x 
line by adding setuptools-style plugin entry points.  However, adding any 
*new* build functionality to the distutils is probably going to be better 
off being based on something that offers an extensible object-dependency 
layer, ala zc.buildout.

Could that be done while leaving the main distutils structure largely 
intact?  Perhaps.  I'm just not convinced that that's what should happen 
for Py3K.  There's a ton of other obsolete or nearly-so stuff happening in 
the existing distutils, like the specialized option parser, crazy bytecode 
compiling hacks, and so on.

(Which reminds me.  Does anybody have any idea why the distutils compile 
modules in build/?  There seems to be no point to this, since any 
installation of the modules is going to need to recompile them.  Even bdist 
operations end up recompiling them in a different location!)


From martin at v.loewis.de  Sun Oct 15 18:58:02 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Oct 2006 18:58:02 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
Message-ID: <4532689A.5040708@v.loewis.de>

Phillip J. Eby schrieb:
> Last, but not least, the distutils suffer from an excess of policy
> flexibility in areas where flexibility is undesirable from a community
> perspective.  There should be "one obvious way" to do a lot of things
> that the distutils provide a ridiculous number of ways to do (e.g.
> insane project directory layouts including package-by-package directory
> mapping).  This is not something that affects the distutils themselves
> much, but it essentially prevents the distutils from improving in
> certain ways because there are so few assumptions one can make about its
> use.

While each of these points might be true, I also believe they can
be fixed. For example, it would be possible to deprecate-then-remove
a non-standard package mapping. I'm not sure how many packages actually
use such a mapping, but we would learn about them through the
deprecation (or they just fix their package).

>> Actually, it is possible to add a new compiler to some Python
>> installation, so that users can pass it to the --compiler option,
>> and distutils packages don't have to be modified for that.
>> You just have to extend the dictionary
>> distutils.ccompiler.compiler_class.
> 
> Yes, and how do you accomplish *that*?  You have to either modify some
> Python code (e.g. sitecustomize) or use a big -c to python when running
> the setup script.

There is a third option: write a .pth file, and put that on the path.

> With entry point extension, it is only necessary to have an extension
> installed on sys.path; it is not necessary to run some code to do the
> registration.

One thing that bothers me about setuptools is the strange, non-intuitive
terminology. What is "entry point extension"? If you'd call it "ini
files listing command names", I could understand much better.

> Adding setuptools-style plugin ability to the distutils is very simple,
> and doesn't require a wholesale reorganization.  After all, setuptools
> manages to do it!

Ok, if that's the most desired property, I think it should be added
(provided somebody is willing to contribute, of course); one doesn't
have to wait for 3.0 to add it.

> However, there are areas of build functionality that
> the distutils does not cover or provides incomplete coverage for, such
> as configuration and build of more complex C libraries and programs,
> shared libraries, documentation production, and so on, that the existing
> architecture of the distutils doesn't really encompass.

I agree it doesn't cover it. I don't see why it isn't "easy" to add
it (where "easy" really means "as easy doing it stand-alone").

> Meanwhile, the higher-level layer is based on the idea of dependency
> between sequentially invoked commands, rather than dependency between
> things being built.  This makes it difficult to deal with concepts like
> having a bunch of reST release note docs combined with a LaTeX
> documentation build.

This I don't understand. Couldn't there be a subcommand of "build" that
creates the documentation? Of course, that command would have to do
the time-stamp checking, and use whatever build machinery is appropriate
(make, ant, scons, ...).

> So, distutils' extensibility *could* be improved dramatically in the 2.x
> line by adding setuptools-style plugin entry points.  However, adding
> any *new* build functionality to the distutils is probably going to be
> better off being based on something that offers an extensible
> object-dependency layer, ala zc.buildout.

I don't mind adding a time-stamp-or-content-based dependency
infrastructure to the standard library, either, and it might be useful
to reimplement the existing build commands on top of that
(for example, it bothers me that the standard setup.py doesn't take
 header file dependencies into account, for the modules of the standard
 library).
That's still no reason to drop distutils.

> (Which reminds me.  Does anybody have any idea why the distutils compile
> modules in build/?  There seems to be no point to this, since any
> installation of the modules is going to need to recompile them.  Even
> bdist operations end up recompiling them in a different location!)

It was added to build_py.py with this checkin:

r17820 | gward | 2000-10-02 04:19:04 +0200 (Mon, 02 Oct 2000) | 8 lines

Added the ability to do byte-compilation at build time, currently off
by default (since compiling at install time works just fine).  Details:
  - added 'compile' and 'optimize' options
  - added 'byte_compile()' method
  - changed 'get_outputs()' so it includes bytecode files
A lot of the code added is very similar to code in install_lib.py;
would be nice to factor it out further.

Notice that it is meant to be off by default, and, indeed, for me,
it is off. Not sure why you are getting .pyc files in your
build directory.

Regards,
Martin

From pje at telecommunity.com  Sun Oct 15 19:44:12 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 15 Oct 2006 13:44:12 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4532689A.5040708@v.loewis.de>
References: <5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061015130129.02be57c8@sparrow.telecommunity.com>

At 06:58 PM 10/15/2006 +0200, Martin v. L?wis wrote:
>Phillip J. Eby schrieb:
> > Last, but not least, the distutils suffer from an excess of policy
> > flexibility in areas where flexibility is undesirable from a community
> > perspective.  There should be "one obvious way" to do a lot of things
> > that the distutils provide a ridiculous number of ways to do (e.g.
> > insane project directory layouts including package-by-package directory
> > mapping).  This is not something that affects the distutils themselves
> > much, but it essentially prevents the distutils from improving in
> > certain ways because there are so few assumptions one can make about its
> > use.
>
>While each of these points might be true, I also believe they can
>be fixed. For example, it would be possible to deprecate-then-remove
>a non-standard package mapping. I'm not sure how many packages actually
>use such a mapping,

They are not common in published packages (although PIL is a notable 
exception).  But they often are in use in unpublished packages, judging by 
distutils-sig threads inquiring about the use of more esoteric directory 
layouts with setuptools.  Setuptools tries hard to support arbitrary 
package mappings, but there are some things that just won't work, like 
making the in-development source code available on sys.path.


> >> Actually, it is possible to add a new compiler to some Python
> >> installation, so that users can pass it to the --compiler option,
> >> and distutils packages don't have to be modified for that.
> >> You just have to extend the dictionary
> >> distutils.ccompiler.compiler_class.
> >
> > Yes, and how do you accomplish *that*?  You have to either modify some
> > Python code (e.g. sitecustomize) or use a big -c to python when running
> > the setup script.
>
>There is a third option: write a .pth file, and put that on the path.

Which means you are now importing the distutils whenever Python is started, 
just as in the sitecustomize case.


> > With entry point extension, it is only necessary to have an extension
> > installed on sys.path; it is not necessary to run some code to do the
> > registration.
>
>One thing that bothers me about setuptools is the strange, non-intuitive
>terminology. What is "entry point extension"? If you'd call it "ini
>files listing command names", I could understand much better.

When I name a new thing, I give it a new name.  If I called it something 
you merely *thought* you understood, I would have a much harder time 
correcting any misunderstanding.  It is intended that you ask "what is 
that?", because then there is some chance you will listen to my answer, or 
that you will investigate to find the answer yourself.  :)

Terminology is also a group/social engineering tool, as well as an 
educational engineering tool.  In the same way that such documents as the 
Zen of Python create a community vocabulary of concepts such as EIBTI, so 
too do setuptools terms create a community vocabulary by which its users 
can recognize each other's level of understanding.  You can observe this in 
blog discussions, hallway conversations at PyCon, etc., where the idea that 
"entry points are cool" is like a little secret handshake among the 
informed, and the opportunity to *explain* them to outsiders provides an 
opportunity for social reward.

A lot of people leave these kinds of things to chance; I consider them as 
much a part of the design of a total system, as anything in the actual 
code.  (See also WSGI "middleware" for a similar social phenomenon, and 
Kathy Sierra's "Creating Passionate Users" blog for general discussion of 
the concepts involved in creating a passionate user community.)

Certainly it is true that there are people who do not care about these 
things or see them as counterproductive.  However, if I am doing "community 
engineering", I do not allow this to get in the way, as those people are 
simply not a "compatible platform" for the socialware I'm creating.  There 
are enough people who *will* run socialware that there is always a viable 
and even vital community for it.


> > Adding setuptools-style plugin ability to the distutils is very simple,
> > and doesn't require a wholesale reorganization.  After all, setuptools
> > manages to do it!
>
>Ok, if that's the most desired property, I think it should be added
>(provided somebody is willing to contribute, of course); one doesn't
>have to wait for 3.0 to add it.

Of course, and I've already said this could be done in the 2.x series.  It 
doesn't even need the whole of setuptools to be incorporated into the 
stdlib; the pkg_resources module suffices.  (Of course, there would need to 
be additions to support building and installing the entry point data.)


> > However, there are areas of build functionality that
> > the distutils does not cover or provides incomplete coverage for, such
> > as configuration and build of more complex C libraries and programs,
> > shared libraries, documentation production, and so on, that the existing
> > architecture of the distutils doesn't really encompass.
>
>I agree it doesn't cover it. I don't see why it isn't "easy" to add
>it (where "easy" really means "as easy doing it stand-alone").

If you look at how much distutils infrastructure ends up being reinvented 
in specific distutils extension, because the original wasn't customizable 
in a useful way, you would see that there is in many respects very little 
difference between adding something to the distutils, and "doing it 
standalone".  Distutils' lower layers encode a bit too many assumptions 
from its upper layers, and it's rarely customizable in the places you need 
it to be.  But even if you have the ability to change the distutils itself, 
you may be stuck because changing these assumptions means breaking the 
existing distutils, but not having any decent way to *discover* that you 
have done so via tests.

Having done massive extensions to the distutils, I think I'm qualified to 
say that, if I had the choice, I would've built something that was merely 
API-compatible with the distutils, borrowing code and ideas from the 
distutils, rather than layering on top of it.

Of course, I didn't have that choice, because one of my design goals was to 
"embrace and extend" by supporting automated installation of a large number 
of existing packages.  However, with setuptools' current marketshare, an 
API-compatible version of setuptools that wasn't based on the current 
distutils architecture could likely be quite successful.


> > Meanwhile, the higher-level layer is based on the idea of dependency
> > between sequentially invoked commands, rather than dependency between
> > things being built.  This makes it difficult to deal with concepts like
> > having a bunch of reST release note docs combined with a LaTeX
> > documentation build.
>
>This I don't understand. Couldn't there be a subcommand of "build" that
>creates the documentation? Of course, that command would have to do
>the time-stamp checking, and use whatever build machinery is appropriate
>(make, ant, scons, ...).

What I'm saying is that the reason there is no build_docs command is 
because documentation build processes aren't sufficiently uniform to fit 
within the distutils' simple command-oriented framework.  Most use of 
distutils extensions "in the wild" are trying to solve issues related to 
this.  For example, Twisted tries to have C extensions that are optionally 
built, based on whether a compiler is available and what libraries are 
available.  This is another scenario that doesn't really work in the 
distutils' concept framework.



> > So, distutils' extensibility *could* be improved dramatically in the 2.x
> > line by adding setuptools-style plugin entry points.  However, adding
> > any *new* build functionality to the distutils is probably going to be
> > better off being based on something that offers an extensible
> > object-dependency layer, ala zc.buildout.
>
>I don't mind adding a time-stamp-or-content-based dependency
>infrastructure to the standard library, either, and it might be useful
>to reimplement the existing build commands on top of that
>(for example, it bothers me that the standard setup.py doesn't take
>  header file dependencies into account, for the modules of the standard
>  library).
>That's still no reason to drop distutils.

I'm not sure what you mean.  Do you mean that there should be a set of 
modules *called* distutils?  I agree with that, and that it should offer 
something closely resembling the existing API, so that a large subset of 
existing packages could continue to work.

But do you also mean that those "distutils" modules must be created only by 
modifying the existing ones?  In that case I disagree.

I personally don't believe that the distutils code can be rehabilitated 
without clean-rooming it for testability.  The lifecycle for effective 
testing against the *enormous* variations in how it is used, means that to 
make any substantial change you must rule out backward compatibility except 
for a strictly defined feature set.  And if you do that, there is nothing 
stopping you from simply implementing a testable and less-fragile 
architecture from the beginning, and then putting a limited 
backward-compatibility layer on top of that.  The result would be far less 
fragile than building new floors on a shaky foundation, as setuptools has 
tried to do.


> > (Which reminds me.  Does anybody have any idea why the distutils compile
> > modules in build/?  There seems to be no point to this, since any
> > installation of the modules is going to need to recompile them.  Even
> > bdist operations end up recompiling them in a different location!)
>
>It was added to build_py.py with this checkin:
>
>r17820 | gward | 2000-10-02 04:19:04 +0200 (Mon, 02 Oct 2000) | 8 lines
>
>Added the ability to do byte-compilation at build time, currently off
>by default (since compiling at install time works just fine).  Details:
>   - added 'compile' and 'optimize' options
>   - added 'byte_compile()' method
>   - changed 'get_outputs()' so it includes bytecode files
>A lot of the code added is very similar to code in install_lib.py;
>would be nice to factor it out further.
>
>Notice that it is meant to be off by default, and, indeed, for me,
>it is off. Not sure why you are getting .pyc files in your
>build directory.

That's not what I said.  I asked why it even has this capability at 
all.  The checkin comment seems to agree that it is useless, "since 
compiling at install time works just fine".  :)


From greg.ewing at canterbury.ac.nz  Mon Oct 16 03:37:05 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 16 Oct 2006 14:37:05 +1300
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45321E83.2030401@v.loewis.de>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de>
Message-ID: <4532E241.8070904@canterbury.ac.nz>

Martin v. L?wis wrote:

> It's only a terminology issue, but I think most people would assume
> a different definition for "extensible". I'd call a system extensible
> if it supports being extended, period.

What matters is how easily and in what ways it can be
extended. My experiences with attempting to extend
distutils or modify its behaviour have not been happy.

1. Pyrex: The distutils extension for compiling Pyrex
modules (which was contributed, not written by me)
seems to work by abusing an existing mechanism for
dealing with swig files. That it was necessary to do
this rather than just adding something new is not a
good sign.

Extension of distutils seems to revolve around the
idea of adding new commands. But I don't want to add
a command, I want to add a new kind of *object* (a
.pyx file) and specify how to transform it into
another kind of object (a .c file) which distutils
already knows how to deal with.

2. Warnings: Distutils insists on compiling C code
with a ridiculously high warning level. For code
generated by Pyrex this is counterproductive, so
I tried to figure out how I could get it to invoke
the C compiler with a different set of options. I
got hopelessly lost trying to follow the code and
gave up.

3. Frameworks: On MacOSX it's often necessary to
link to something that's installed as a framework
instead of a library. Putting -F options in the
extra_link_args doesn't always work because
sometimes they need to be at the beginning. I
haven't found a way to make that happen.

A well-designed extension mechanism would provide
fairly obvious ways of going about all these things.

 > If certain functionality is
> missing in distutils (e.g. means for more transparent extension), then
> distutils should be enhanced to provide this functionality.

Unless the basic structure is so far from what's
needed that it can't be reasonably fixed.

--
Greg

From greg.ewing at canterbury.ac.nz  Mon Oct 16 03:37:30 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 16 Oct 2006 14:37:30 +1300
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
Message-ID: <4532E25A.2090607@canterbury.ac.nz>

Phillip J. Eby wrote:
> Meanwhile, the higher-level layer is based on the idea of dependency 
> between sequentially invoked commands, rather than dependency between 
> things being built.

This is what bothers me most about the design of distutils,
I think. A setup.py file should be structured like a
Makefile, and processed in a Make-like way.

Martin v. L?wis wrote:
 > Couldn't there be a subcommand of "build" that
 > creates the documentation? Of course, that command would have to do
 > the time-stamp checking,

Which it *shouldn't* have to do. Timestamp checking
is fundamental to any kind of build process. It
should be built into the very foundations of the tool,
not left for each extension to re-invent separately.

--
Greg

From greg.ewing at canterbury.ac.nz  Mon Oct 16 02:12:10 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 16 Oct 2006 13:12:10 +1300
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <2DE9DE61-6D77-4D0D-B623-B513FFF63E96@mac.com>
References: <452F26A3.7060506@acm.org> <2ab101c6ee90$83796470$9a4c2a97@bagio>
	<52208.62.39.9.251.1160728013.squirrel@webmail.nerim.net>
	<2f7601c6eeac$d85fcb20$9a4c2a97@bagio> <452FC988.5000608@acm.org>
	<bbaeab100610131604k2d948b9bw7c5efbe4150c1e87@mail.gmail.com>
	<45304A7E.2050209@acm.org> <4530551E.3000400@solarsail.hcs.harvard.edu>
	<45316BE1.8060209@canterbury.ac.nz>
	<2DE9DE61-6D77-4D0D-B623-B513FFF63E96@mac.com>
Message-ID: <4532CE5A.6020902@canterbury.ac.nz>

Ronald Oussoren wrote:

> Even then easy_install support won't be univeral. I still run into  
> packages that don't use distutils, mostly stuff that is or was part  of 
> a larger C code base.

However, at least there would be One Obvious Way for
those that do want to use a standard build system.

Something that might help is if there were a way
for distutils to use something external, such as a
Makefile-based build process, as a component of the
overall build process. Currently distutils likes
to be in charge of everthing, which can be
inconvenient.

How well would setuptools or zc.build handle this
sort of thing?

--
Greg

From martin at v.loewis.de  Mon Oct 16 06:46:20 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Oct 2006 06:46:20 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4532E241.8070904@canterbury.ac.nz>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de> <4532E241.8070904@canterbury.ac.nz>
Message-ID: <45330E9C.9030300@v.loewis.de>

Greg Ewing schrieb:
> 1. Pyrex: The distutils extension for compiling Pyrex
> modules (which was contributed, not written by me)
> seems to work by abusing an existing mechanism for
> dealing with swig files. That it was necessary to do
> this rather than just adding something new is not a
> good sign.

One would have to study it in more detail. What
is the processing need of a Pyrex file? I would
expect there is some activity on building, and
some activity on installing (perhaps not so,
as it may install as an extension).

> Extension of distutils seems to revolve around the
> idea of adding new commands. But I don't want to add
> a command, I want to add a new kind of *object* (a
> .pyx file) and specify how to transform it into
> another kind of object (a .c file) which distutils
> already knows how to deal with.

Right, and the appropriate mechanism here is to
create a sub-command of build: you are trying to
explain how to *build* the .c file.

> 2. Warnings: Distutils insists on compiling C code
> with a ridiculously high warning level.

Well, it really uses the settings that were determined
when Python itself was compiled. It doesn't know at
all what a compiler warning is and how to turn it on
or off.

> For code
> generated by Pyrex this is counterproductive, so
> I tried to figure out how I could get it to invoke
> the C compiler with a different set of options. I
> got hopelessly lost trying to follow the code and
> gave up.

Right; changing that isn't easy (you could change
the installed Makefile, and have the change in
effect for all C extensions). However, I would argue
that it is really bad that Pyrex generates code
that compiles with warnings. It shouldn't do that.

> 3. Frameworks: On MacOSX it's often necessary to
> link to something that's installed as a framework
> instead of a library. Putting -F options in the
> extra_link_args doesn't always work because
> sometimes they need to be at the beginning. I
> haven't found a way to make that happen.

Can't comment here, since I never had this problem myself.

> A well-designed extension mechanism would provide
> fairly obvious ways of going about all these things.

Well, can you design a mechanism that would allow to
solve your problem with frameworks? How would it
look like? How would it operate on systems other than
the Mac?

> Unless the basic structure is so far from what's
> needed that it can't be reasonably fixed.

See, and I believe this isn't the case for distutils.

Regards,
Martin

From martin at v.loewis.de  Mon Oct 16 06:48:42 2006
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 16 Oct 2006 06:48:42 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4532E25A.2090607@canterbury.ac.nz>
References: <5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
	<4532E25A.2090607@canterbury.ac.nz>
Message-ID: <45330F2A.6010202@v.loewis.de>

Greg Ewing schrieb:
> Which it *shouldn't* have to do. Timestamp checking
> is fundamental to any kind of build process. It
> should be built into the very foundations of the tool,
> not left for each extension to re-invent separately.

And indeed, distutils already has support for that:
distutils.dep_util offers a couple of routines that
do time-stamp checking. You don't have to reinvent,
but you do have to specify what should happen when
some file is newer than some other file. distutils
cannot know this.

Regards,
Martin

From ronaldoussoren at mac.com  Mon Oct 16 08:04:25 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Mon, 16 Oct 2006 08:04:25 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45330F2A.6010202@v.loewis.de>
References: <5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
	<4532E25A.2090607@canterbury.ac.nz> <45330F2A.6010202@v.loewis.de>
Message-ID: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>


On Oct 16, 2006, at 6:48 AM, Martin v. L?wis wrote:

> Greg Ewing schrieb:
>> Which it *shouldn't* have to do. Timestamp checking
>> is fundamental to any kind of build process. It
>> should be built into the very foundations of the tool,
>> not left for each extension to re-invent separately.
>
> And indeed, distutils already has support for that:
> distutils.dep_util offers a couple of routines that
> do time-stamp checking.

This points to at least part of the problem: there is very little  
useful documentation on how to extend, or even use, distutils.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061016/e82b4037/attachment.bin 

From aahz at pythoncraft.com  Mon Oct 16 16:11:43 2006
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 16 Oct 2006 07:11:43 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45330E9C.9030300@v.loewis.de>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de> <4532E241.8070904@canterbury.ac.nz>
	<45330E9C.9030300@v.loewis.de>
Message-ID: <20061016141143.GB14145@panix.com>

On Mon, Oct 16, 2006, "Martin v. L?wis" wrote:
> Greg Ewing schrieb:
>> 
>> Unless the basic structure is so far from what's needed that it can't
>> be reasonably fixed.
>
> See, and I believe this isn't the case for distutils.

Can you suggest a good mechanism for resolving the deadlock?  We have the
primary maintainer (and some other users) claiming that distutils is a
mess that needs rewriting.  You think it doesn't, but based on other
comments you've made, I believe you don't have available time.  I haven't
seen anyone else agreeing with you.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"If you don't know what your program is supposed to do, you'd better not
start writing it."  --Dijkstra

From martin at v.loewis.de  Mon Oct 16 18:21:57 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Oct 2006 18:21:57 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
References: <5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
	<4532E25A.2090607@canterbury.ac.nz> <45330F2A.6010202@v.loewis.de>
	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
Message-ID: <4533B1A5.5070203@v.loewis.de>

Ronald Oussoren schrieb:
>> And indeed, distutils already has support for that:
>> distutils.dep_util offers a couple of routines that
>> do time-stamp checking.
> 
> This points to at least part of the problem: there is very little useful
> documentation on how to extend, or even use, distutils.

Thats certainly the case, and indeed a problem (I readily
admit there are many problems with distutils). These problems
can only be fixed through people contributing documentation,
though. This *specific* problem certainly cannot be resolved
by replacing the package with something else, unless that
something else has already excellent documentation to start
with (which it likely doesn't).

I believe that the whole issue of packaging and deployment
is indeed one of the biggest challenges in IT engineering,
so it is no surprise that the technology attempting to deal
with it is complex, incomplete, and non-intuitive. And that
statement is IMO not only true about distutils, but also
about its direct competitors, and about the more-remote
relatives (such as dpkg, rpm, msi, OSX packages, ...)

Regards,
Martin

From pje at telecommunity.com  Mon Oct 16 18:53:46 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 16 Oct 2006 12:53:46 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4533B1A5.5070203@v.loewis.de>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
	<4532E25A.2090607@canterbury.ac.nz> <45330F2A.6010202@v.loewis.de>
	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
Message-ID: <5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>

At 06:21 PM 10/16/2006 +0200, Martin v. L?wis wrote:
>Ronald Oussoren schrieb:
> >> And indeed, distutils already has support for that:
> >> distutils.dep_util offers a couple of routines that
> >> do time-stamp checking.
> >
> > This points to at least part of the problem: there is very little useful
> > documentation on how to extend, or even use, distutils.
>
>Thats certainly the case, and indeed a problem (I readily
>admit there are many problems with distutils). These problems
>can only be fixed through people contributing documentation,
>though. This *specific* problem certainly cannot be resolved
>by replacing the package with something else, unless that
>something else has already excellent documentation to start
>with (which it likely doesn't).

One nice thing about zc.buildout in this respect is that its tutorial is 
part of its doctest suite.  The more-complete dependency framework offered 
by zc.buildout also highlights another aspect of the problem: saying that 
you can create a useful build system from distutils.dep_util is like saying 
you can create one using the standard library.  There's only about 40 lines 
of actual code in that module, and it's hardly rocket science.  Meanwhile, 
most of the routines have some flaws for being used as anything resembling 
a "make" facility; for example, newer() and newer_pairwise() raise 
DistutilsFileError if a source file is missing!  This is a minor example of 
what I mean about distutils often encoding high-level policy assumptions 
into low-level routines.  Its "util" modules aren't really suitable for 
reuse by anything but the distutils, and you can't change them without 
breaking the distutils.

('newer_group()' is better for reuse than its neighbors, of course, but if 
one were writing new code, there would still be little gained by borrowing 
this brief routine, assuming that you even knew it existed, given the 
undocumented nature of the module.)


>I believe that the whole issue of packaging and deployment
>is indeed one of the biggest challenges in IT engineering,
>so it is no surprise that the technology attempting to deal
>with it is complex, incomplete, and non-intuitive. And that
>statement is IMO not only true about distutils, but also
>about its direct competitors,

What competitors?  Do you mean Ruby gems, CPAN, etc.?  I would say that 
those competitors actually show that we could do much better than we 
are.  I think that Perl may actually have us beat in terms of having "one 
way to do it" with respect to package layout, for example.  (Ah, the irony, 
given the Perl TMTOWTDI motto, and our own "There should be one and 
preferably only one obvious way to do it.")


From krstic at solarsail.hcs.harvard.edu  Mon Oct 16 19:02:50 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Mon, 16 Oct 2006 13:02:50 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>	<4532E25A.2090607@canterbury.ac.nz>
	<45330F2A.6010202@v.loewis.de>	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
Message-ID: <4533BB3A.1040709@solarsail.hcs.harvard.edu>

Phillip J. Eby wrote:
> I think that Perl may actually have us beat in terms of having "one 
> way to do it" with respect to package layout, for example. 

I don't think there's much of a 'may' about it. I've often wondered why
non-Perl languages haven't been able to -- culturally -- create their
own version of CPAN, and failing that, why they haven't copied CPAN
outright.

The Perl world has pretty much standardized on the "if it's out there,
it's on CPAN" model, and it's one of the language's greatest assets.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From pje at telecommunity.com  Mon Oct 16 19:23:08 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 16 Oct 2006 13:23:08 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4533BB3A.1040709@solarsail.hcs.harvard.edu>
References: <5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
	<4532E25A.2090607@canterbury.ac.nz> <45330F2A.6010202@v.loewis.de>
	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061016132018.026df408@sparrow.telecommunity.com>

At 01:02 PM 10/16/2006 -0400, Ivan Krsti?? wrote:
>Phillip J. Eby wrote:
> > I think that Perl may actually have us beat in terms of having "one
> > way to do it" with respect to package layout, for example.
>
>I don't think there's much of a 'may' about it. I've often wondered why
>non-Perl languages haven't been able to -- culturally -- create their
>own version of CPAN, and failing that, why they haven't copied CPAN
>outright.
>
>The Perl world has pretty much standardized on the "if it's out there,
>it's on CPAN" model, and it's one of the language's greatest assets.

I was actually referring to the standardization of the *layout* of the 
contents of a package, and standardization on the interface for performing 
build, install, and test steps.

But yes, CPAN is more a cultural phenomenon than a technical one, as its 
creators have previously pointed out.


From talin at acm.org  Mon Oct 16 19:27:23 2006
From: talin at acm.org (Talin)
Date: Mon, 16 Oct 2006 10:27:23 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <20061016141143.GB14145@panix.com>
References: <45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45321E83.2030401@v.loewis.de>
	<4532E241.8070904@canterbury.ac.nz>	<45330E9C.9030300@v.loewis.de>
	<20061016141143.GB14145@panix.com>
Message-ID: <4533C0FB.7070808@acm.org>

Aahz wrote:
> On Mon, Oct 16, 2006, "Martin v. L?wis" wrote:
>> Greg Ewing schrieb:
>>> Unless the basic structure is so far from what's needed that it can't
>>> be reasonably fixed.
>> See, and I believe this isn't the case for distutils.
> 
> Can you suggest a good mechanism for resolving the deadlock?  We have the
> primary maintainer (and some other users) claiming that distutils is a
> mess that needs rewriting.  You think it doesn't, but based on other
> comments you've made, I believe you don't have available time.  I haven't
> seen anyone else agreeing with you.

One thing that I think would be helpful is if there were a requirements 
doc for distutils. Any attempt to refactor distutils is going to fail 
IMHO, unless the person doing the refactoring knows exactly what it 
needs to accomplish, and as we all know, its not always possible to 
infer from an existing system (even a documented one) what its 
*supposed* to do, as opposed to what it actually does now. I know that 
in my case, I have a hard enough time even understanding the existing 
distutils docs - if I were to foolishly attempt to create a replacement, 
what would happen is that I would end up solving my specific problems 
and no one else's.

I would suggest perhaps a wiki page or discussion devoted to collecting 
requirements and use cases for a new distutils. This should incorporate 
not only the existing distutils requirements, but should also 
incorporate lessons from the creation of setuptools, and an awareness of 
what's going on in the refactoring of the import machinery.

One thing to discuss would be how to handle backwards compatibility; I 
see three alternatives: (1) Make the APIs of the new system match the 
old one, (2) Make a "compatibility module" that plugs in to the new 
system, allowing development of a new API while maintaining the existing 
one, and (3) leave the existing distutils system in place, but 
deprecated in favor of the new one.

If the new system is enough better than the previous one, I don't see 
that there will be a huge problem getting people to migrate over, as 
long as the new system supports older versions of Python as well as 
Py3k. Given that many distutils scripts are generated from boilerplate 
or templates, automatic conversion may also be possible.

Once the requirements are in place and there is rough consensus on what 
the tool actually needs to do, we can start generating PEPs that attempt 
to meet those requirements.

-- Talin

From martin at v.loewis.de  Mon Oct 16 19:48:43 2006
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 16 Oct 2006 19:48:43 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4533BB3A.1040709@solarsail.hcs.harvard.edu>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>	<4532E25A.2090607@canterbury.ac.nz>
	<45330F2A.6010202@v.loewis.de>	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
	<4533BB3A.1040709@solarsail.hcs.harvard.edu>
Message-ID: <4533C5FB.6090708@v.loewis.de>

Ivan Krsti? schrieb:
> I don't think there's much of a 'may' about it. I've often wondered why
> non-Perl languages haven't been able to -- culturally -- create their
> own version of CPAN, and failing that, why they haven't copied CPAN
> outright.
> 
> The Perl world has pretty much standardized on the "if it's out there,
> it's on CPAN" model, and it's one of the language's greatest assets.

I hope Python won't copy CPAN literally. Every time I used it, it was
a big pain, and I happily stopped using it when everything I needed
came as a Debian package.

The problem is the dependencies: Literally *everytime* I invoked CPAN
(the CPAN shell) it told me it had to update sometime I already had,
in many cases, it had to update the CPAN software itself (I typically
used it on Solaris every 6 months for a period of about four years).
In two cases, it insisted that it had to install a new version
of Perl *itself*, because some intermediary package depended on a
more recent Perl version. It would download perl, run the test suite,
and, after several hours, determine that one of the tests that was
totally irrelevant to me failed, and therefore, it couldn't do what
I wanted to start with. I learned to hate CPAN.

Regards,
Martin

From krstic at solarsail.hcs.harvard.edu  Mon Oct 16 19:56:11 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Mon, 16 Oct 2006 13:56:11 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <5.1.1.6.0.20061016132018.026df408@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
	<4532E25A.2090607@canterbury.ac.nz> <45330F2A.6010202@v.loewis.de>
	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
	<5.1.1.6.0.20061016132018.026df408@sparrow.telecommunity.com>
Message-ID: <4533C7BB.2030805@solarsail.hcs.harvard.edu>

Phillip J. Eby wrote:
> I was actually referring to the standardization of the *layout* of the
> contents of a package, and standardization on the interface for
> performing build, install, and test steps.

Right, sorry, I'm running in awake-for-40-hours mode so I didn't
explicitly connect my thoughts. CPAN's cultural success is predicated on
a generally standardized build/install/test system and package layout.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From krstic at solarsail.hcs.harvard.edu  Mon Oct 16 20:11:52 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Mon, 16 Oct 2006 14:11:52 -0400
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4533C5FB.6090708@v.loewis.de>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>	<4532E25A.2090607@canterbury.ac.nz>
	<45330F2A.6010202@v.loewis.de>	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
	<4533BB3A.1040709@solarsail.hcs.harvard.edu>
	<4533C5FB.6090708@v.loewis.de>
Message-ID: <4533CB68.5040706@solarsail.hcs.harvard.edu>

Martin v. L?wis wrote:
> I hope Python won't copy CPAN literally. Every time I used it, it was
> a big pain, and I happily stopped using it when everything I needed
> came as a Debian package.

Note my emphasis on `culturally` in the previous mail. CPAN's largest
value isn't as a package management system, but as an actual canonical
repository of useful Perl code, large and small.

So my three propositions here are:

1) Having a large, canonical repository of useful Python code that's
much more encompassing than Cheeseshop would be a good thing for Python.

2) It appears the way to create and maintain such a repository is to
ingrain it into the language culture, whereby it becomes unusual that
useful redistributable code /isn't/ in the repository.

3) To address 1) and 2), we need a canonical package layout and
build/install/test cycle that are easy for developers to work with,
well-documented, and for which there exist better and less arcane tools
than what we have now.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From bob at redivi.com  Mon Oct 16 20:35:24 2006
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 16 Oct 2006 11:35:24 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4533CB68.5040706@solarsail.hcs.harvard.edu>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
	<4532E25A.2090607@canterbury.ac.nz> <45330F2A.6010202@v.loewis.de>
	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
	<4533BB3A.1040709@solarsail.hcs.harvard.edu>
	<4533C5FB.6090708@v.loewis.de>
	<4533CB68.5040706@solarsail.hcs.harvard.edu>
Message-ID: <6a36e7290610161135h228afc20je71d1893233cbd1d@mail.gmail.com>

On 10/16/06, Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> wrote:
> Martin v. L?wis wrote:
> > I hope Python won't copy CPAN literally. Every time I used it, it was
> > a big pain, and I happily stopped using it when everything I needed
> > came as a Debian package.
>
> Note my emphasis on `culturally` in the previous mail. CPAN's largest
> value isn't as a package management system, but as an actual canonical
> repository of useful Perl code, large and small.
>
> So my three propositions here are:
>
> 1) Having a large, canonical repository of useful Python code that's
> much more encompassing than Cheeseshop would be a good thing for Python.
>
> 2) It appears the way to create and maintain such a repository is to
> ingrain it into the language culture, whereby it becomes unusual that
> useful redistributable code /isn't/ in the repository.
>
> 3) To address 1) and 2), we need a canonical package layout and
> build/install/test cycle that are easy for developers to work with,
> well-documented, and for which there exist better and less arcane tools
> than what we have now.

This is basically solved at a technical level with setuptools as-is.
setuptools encourages a canonical package layout via its
find_packages() function and allows you to point it at a test suite
with the test_suite kwarg to setup(). It saves you the trouble of
writing a MANIFEST.in because it sucks in metadata from SVN or CVS.

Registering and uploading to CheeseShop takes mere seconds, especially
if you have an alias that does "register bdist_egg sdist upload -s" or
whatever.

What seems to be necessary is hand-holding documentation that states
what the encouraged practices are and walks through the release cycle
of a typical project (using the develop command, uninstalling an egg,
doing a release on cheese shop, etc.).

It may also be a good idea to encourage use of a "quickstart" tool
which creates a skeleton setup.py and an appropriate directory
structure for a new project. This already exists too (the create
command of the "paster" script that comes with Paste Script). Hell,
paster will even do the "svn mkdir" for you to create the
trunk/tags/branches...

-bob

From qrczak at knm.org.pl  Mon Oct 16 21:03:31 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Mon, 16 Oct 2006 21:03:31 +0200
Subject: [Python-3000] Pre-PEP: Easy Text File Decoding
References: <1cb725390609092029v6dae753ajf8bf85e4974bb4d0@mail.gmail.com>
	<87d5a366xd.fsf@qrnik.zagroda>
	<1cb725390609102142m3a1dd33ha8400ec8e2005ba@mail.gmail.com>
	<8764fu3cxy.fsf@qrnik.zagroda> <45311859.6080102@v.loewis.de>
	<87odsed9ct.fsf@qrnik.zagroda> <4531324E.4040206@v.loewis.de>
	<87zmbyejs6.fsf@qrnik.zagroda> <4531D495.5030909@v.loewis.de>
Message-ID: <87u0242id8.fsf@qrnik.zagroda>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Marcin 'Qrczak' Kowalczyk schrieb:
>> It is true that it can change the interpretation of file contents.
>> This is unavoidable. Unless someone uses unpaired surrogates for this
>> purpose (or code points above U+10FFFF) - I've seen such proposals,
>> but IMHO they are abusing rules too far.
>
> It's not exactly unavoidable: any escaping mechanism can support the
> full range of valid input. In your escaping mechanism, you could
> duplicate 0 bytes on decoding, and write a null byte if you have two
> subsequent NUL characters on encoding.

This is exactly what I am doing. The encoding is able to decode
arbitrary byte sequences, including '\0' bytes, and encodes them back
losslessly.

The point is that it differs from true UTF-8 for strings which contain
'\0' or U+0000. It's unavoidable that it differs from UTF-8 for some
strings, unless code points not encodable in UTF-8 are used.

It doesn't differ from true UTF-8 when there is no '\0' or U+0000.
The fact that it doesn't differ from UTF-8 for some strings means that
for such strings it fires only when UTF-8 decoder would have reported
an error, i.e. that it only changes the behavior of code which would
fail otherwise, that it doesn't break what would work in UTF-8.

My encoder is injective: it accepts U+0000 prefixes only in sequences
which would have been invalid UTF-8.

I agree that it's not suitable for showing the filename for a user.

> I still think that PUA characters would be a better use

What if the filename contains the correct UTF-8 encoding of such PUA
character?

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From martin at v.loewis.de  Mon Oct 16 21:43:36 2006
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 16 Oct 2006 21:43:36 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4533CB68.5040706@solarsail.hcs.harvard.edu>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>	<4532E25A.2090607@canterbury.ac.nz>
	<45330F2A.6010202@v.loewis.de>	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
	<4533BB3A.1040709@solarsail.hcs.harvard.edu>
	<4533C5FB.6090708@v.loewis.de>
	<4533CB68.5040706@solarsail.hcs.harvard.edu>
Message-ID: <4533E0E8.6000700@v.loewis.de>

Ivan Krsti? schrieb:
> 1) Having a large, canonical repository of useful Python code that's
> much more encompassing than Cheeseshop would be a good thing for Python.

It's probably due to deficiencies in my mastering of the English
language: What do you mean by "encompassing" here?

If you are requesting that all Python software in the world should
be listed in a canonical repository, I say that the Cheeseshop *is*
that canonical repository. If some package isn't listed, it's the
"fault" of the author of the package and its users, not the fault
of the Cheeseshop. I can't see how a *different* repository could
have any chance of becoming canonical.

> 2) It appears the way to create and maintain such a repository is to
> ingrain it into the language culture, whereby it becomes unusual that
> useful redistributable code /isn't/ in the repository.

Well, the Cheeseshop is indeed ingrained into the language. That
it becomes unusual that software isn't listed there is merely a
matter of time.

> 3) To address 1) and 2), we need a canonical package layout and
> build/install/test cycle that are easy for developers to work with,
> well-documented, and for which there exist better and less arcane tools
> than what we have now.

The only realistic chance for that to happen is to make the distutils
less arcane and better.

Regards,
Martin


From jcarlson at uci.edu  Mon Oct 16 22:10:29 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 16 Oct 2006 13:10:29 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4533E0E8.6000700@v.loewis.de>
References: <4533CB68.5040706@solarsail.hcs.harvard.edu>
	<4533E0E8.6000700@v.loewis.de>
Message-ID: <20061016130019.0A33.JCARLSON@uci.edu>


"Martin v. L?wis" <martin at v.loewis.de> wrote:
> The only realistic chance for that to happen is to make the distutils
> less arcane and better.

I've been staying out of the discussion, but I'll toss in a hearty +1 in
support of making distutils "less arcane and better".

In my own experience, I have found that with the packages I usually end
up having to ship (source files with related data), sdist is the only
distutils command that gets it right.  Any of the bdist commands bork
the Windows (.msi) Python installation (data files go into site-packages,
source goes into python binary path), and I've been too afraid to try
any of the linux bdist options (would require a tedious backup and
restore of a VM).

Because of the frustration that is distutils, I've considered dropping
distutils and switching to make + Makefile for all but my py2exe needs,
and even not supplying setup.py with any distribution.

 - Josiah


From talin at acm.org  Mon Oct 16 21:59:22 2006
From: talin at acm.org (Talin)
Date: Mon, 16 Oct 2006 12:59:22 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4533E0E8.6000700@v.loewis.de>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>	<4532E25A.2090607@canterbury.ac.nz>	<45330F2A.6010202@v.loewis.de>	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>	<4533BB3A.1040709@solarsail.hcs.harvard.edu>	<4533C5FB.6090708@v.loewis.de>	<4533CB68.5040706@solarsail.hcs.harvard.edu>
	<4533E0E8.6000700@v.loewis.de>
Message-ID: <4533E49A.9030800@acm.org>

Martin v. L?wis wrote:
> Ivan Krsti? schrieb:
>> 1) Having a large, canonical repository of useful Python code that's
>> much more encompassing than Cheeseshop would be a good thing for Python.
> 
> It's probably due to deficiencies in my mastering of the English
> language: What do you mean by "encompassing" here?
> 
> If you are requesting that all Python software in the world should
> be listed in a canonical repository, I say that the Cheeseshop *is*
> that canonical repository. If some package isn't listed, it's the
> "fault" of the author of the package and its users, not the fault
> of the Cheeseshop. I can't see how a *different* repository could
> have any chance of becoming canonical.

Agreed - I don't have any problems with Cheeseshop as it exists today.

-- Talin


From martin at v.loewis.de  Mon Oct 16 22:42:02 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Oct 2006 22:42:02 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <20061016130019.0A33.JCARLSON@uci.edu>
References: <4533CB68.5040706@solarsail.hcs.harvard.edu>
	<4533E0E8.6000700@v.loewis.de>
	<20061016130019.0A33.JCARLSON@uci.edu>
Message-ID: <4533EE9A.7010105@v.loewis.de>

Josiah Carlson schrieb:
> In my own experience, I have found that with the packages I usually end
> up having to ship (source files with related data), sdist is the only
> distutils command that gets it right.  Any of the bdist commands bork
> the Windows (.msi) Python installation (data files go into site-packages,
> source goes into python binary path), and I've been too afraid to try
> any of the linux bdist options (would require a tedious backup and
> restore of a VM).

So did you make bug reports about these issues (in the abstract, it's
hard to see what is happening, and why it is: what precisely is
"source", and what is the "python binary path" that is going into?;
the bug report should provide these details).

Without useful bug reports, there is no chance of improvements.
With useful bug reports, there is unfortunately still little chance
of improvements, since distutils is unmaintained. However, these
issues seem to be easily fixable (if they aren't a pilot errors).

Regards,
Martin

From python at zesty.ca  Tue Oct 17 02:57:23 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Mon, 16 Oct 2006 19:57:23 -0500 (CDT)
Subject: [Python-3000] Cheeseshop
In-Reply-To: <4533E49A.9030800@acm.org>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
	<4532E25A.2090607@canterbury.ac.nz> <45330F2A.6010202@v.loewis.de>
	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
	<4533BB3A.1040709@solarsail.hcs.harvard.edu>
	<4533C5FB.6090708@v.loewis.de>
	<4533CB68.5040706@solarsail.hcs.harvard.edu>
	<4533E0E8.6000700@v.loewis.de> <4533E49A.9030800@acm.org>
Message-ID: <Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>

On Mon, 16 Oct 2006, Talin wrote:
> Agreed - I don't have any problems with Cheeseshop as it exists today.

I think a good search feature is essential -- a big code repository
is only as useful to the degree that its contents are accessible.

I don't want to hurt anyone's feelings, but i think the site could
really use improved search and navigation.  It used to have a search
form that was (in my opinion) broken enough to be misleading, and now
the search form appears to be gone.  The Google-based form feels like
a stopgap measure rather than a proper search.

Right now, the site doesn't provide an obvious, direct way to get
to a package *even if you know the exact name of the package*.

By the way, is it called Cheeseshop, PyPI, or "Package Index", and
is the URL http://python.org/pypi, http://cheeseshop.python.org/,
or http://cheeseshop.python.org/pypi?  Could we have OOWTDI?


-- ?!ng

From bob at redivi.com  Tue Oct 17 03:06:23 2006
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 16 Oct 2006 18:06:23 -0700
Subject: [Python-3000] Cheeseshop
In-Reply-To: <Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45330F2A.6010202@v.loewis.de>
	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
	<4533BB3A.1040709@solarsail.hcs.harvard.edu>
	<4533C5FB.6090708@v.loewis.de>
	<4533CB68.5040706@solarsail.hcs.harvard.edu>
	<4533E0E8.6000700@v.loewis.de> <4533E49A.9030800@acm.org>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
Message-ID: <6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>

On 10/16/06, Ka-Ping Yee <python at zesty.ca> wrote:
> On Mon, 16 Oct 2006, Talin wrote:
> > Agreed - I don't have any problems with Cheeseshop as it exists today.
>
> I think a good search feature is essential -- a big code repository
> is only as useful to the degree that its contents are accessible.
>
> I don't want to hurt anyone's feelings, but i think the site could
> really use improved search and navigation.  It used to have a search
> form that was (in my opinion) broken enough to be misleading, and now
> the search form appears to be gone.  The Google-based form feels like
> a stopgap measure rather than a proper search.

It should be easy enough to fix search these days, given that SQLite
comes with a full text search engine now.

http://www.sqlite.org/cvstrac/wiki?p=FtsOne

> Right now, the site doesn't provide an obvious, direct way to get
> to a package *even if you know the exact name of the package*.

URL hacking ;) Unfortunately you also need to know the exact case of
the package name, which can often be confusing.. e.g. SQLAlchemy or
sqlalchemy? We should enforce uniqueness on the lowercase package
name, and allow that everywhere (but preserve case for display).

> By the way, is it called Cheeseshop, PyPI, or "Package Index", and
> is the URL http://python.org/pypi, http://cheeseshop.python.org/,
> or http://cheeseshop.python.org/pypi?  Could we have OOWTDI?

According to distutils (and setuptools) it's still:

DEFAULT_REPOSITORY = 'http://www.python.org/pypi'

However I think that many (most?) links on the web have started to
migrate to http://cheeseshop.python.org/pypi

-bob

From fdrake at acm.org  Tue Oct 17 03:28:36 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon, 16 Oct 2006 21:28:36 -0400
Subject: [Python-3000] Cheeseshop
In-Reply-To: <6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>
Message-ID: <200610162128.37557.fdrake@acm.org>

On Monday 16 October 2006 21:06, Bob Ippolito wrote:
 > However I think that many (most?) links on the web have started to
 > migrate to http://cheeseshop.python.org/pypi

Which is really, really, REALLY sad.  The "Cheese Shop" name may reflect the 
proper cultural bias toward Monty Python, but it's a sure way to alienate 
people by presenting the hard work of the community under a completely silly 
name that communicates nothing about what it's about.

In case it's not obvious, I *HATE* the "Cheese Shop" name.  Grr.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From talin at acm.org  Tue Oct 17 04:20:28 2006
From: talin at acm.org (Talin)
Date: Mon, 16 Oct 2006 19:20:28 -0700
Subject: [Python-3000] Cheeseshop
In-Reply-To: <Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>
	<4532E25A.2090607@canterbury.ac.nz> <45330F2A.6010202@v.loewis.de>
	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>
	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>
	<4533BB3A.1040709@solarsail.hcs.harvard.edu>
	<4533C5FB.6090708@v.loewis.de>
	<4533CB68.5040706@solarsail.hcs.harvard.edu>
	<4533E0E8.6000700@v.loewis.de> <4533E49A.9030800@acm.org>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
Message-ID: <45343DEC.7030604@acm.org>

Ka-Ping Yee wrote:
> On Mon, 16 Oct 2006, Talin wrote:
>> Agreed - I don't have any problems with Cheeseshop as it exists today.
> 
> I think a good search feature is essential -- a big code repository
> is only as useful to the degree that its contents are accessible.
> 
> I don't want to hurt anyone's feelings, but i think the site could
> really use improved search and navigation.  It used to have a search
> form that was (in my opinion) broken enough to be misleading, and now
> the search form appears to be gone.  The Google-based form feels like
> a stopgap measure rather than a proper search.
> 
> Right now, the site doesn't provide an obvious, direct way to get
> to a package *even if you know the exact name of the package*.

I see. Maybe I have no problem because I never actually use the *web* 
interface to Cheeseshop. I just say "easy_install mercurial" or 
whatever, and everything just works - unless of course, the package you 
are trying to install is wxwidgets or psycopg2 or openldap, or any 
others of a bunch I could name, in which case the download works fine by 
the installation just fails. :(

- Talin

From greg.ewing at canterbury.ac.nz  Tue Oct 17 07:32:48 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 17 Oct 2006 18:32:48 +1300
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45330E9C.9030300@v.loewis.de>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de> <4532E241.8070904@canterbury.ac.nz>
	<45330E9C.9030300@v.loewis.de>
Message-ID: <45346B00.1000506@canterbury.ac.nz>

Martin v. L?wis wrote:
> One would have to study it in more detail. What
> is the processing need of a Pyrex file?

Pretty simple -- just pass the .pyx file through
the Pyrex compiler, then do whatever you would
normally do to the resulting .c file to turn it
into an extension module.

A refinement would be to do this only if the
.pyx or any .pyd it uses is newer than the last
generated .c file.

> I would
> expect there is some activity on building, and
> some activity on installing

Nope, it installs just like any other extension
once it's built.

> Right, and the appropriate mechanism here is to
> create a sub-command of build:

Not sure exactly what you mean by a "sub-command".
If you mean a subclass of the class that implements
the "build" command, that's far too heavyweight.
I want to deal just with the .pyx->.c transformation,
and not have to worry that I might mess up something
else that the build class does.

Also, what happens if I want to use my extended
build class together with someone else's extended
build class? Do I have to merge them using multiple
inheritance? Without knowing a *lot* about their
internals, I'd be very nervous doing that lest they
interact in unforeseen ways.

> Right; changing that isn't easy (you could change
> the installed Makefile, and have the change in
> effect for all C extensions).

That's actually what I ended up doing in my own
Python installation. But it doesn't help anyone
else.

> However, I would argue
> that it is really bad that Pyrex generates code
> that compiles with warnings. It shouldn't do that.

It *doesn't* generate any warnings using the
normal, default C compiler settings -- only the
extremely paranoid ones that the Python installation
defaults to. It would take what I regard as an
unreasonable amount of work on the part of Pyrex to
change that.

But that's a different rant. My point is that
I failed to find a workaround, even by hacking.
"Use the source, Luke" doesn't work if the source
is so convoluted that ordinary mortals can't
understand it.

> Well, can you design a mechanism that would allow to
> solve your problem with frameworks?

If I were dealing with a Makefile, and there
weren't a macro to override that would have the
desired effect, I could replace the whole rule
for linking C files with one of my own that
did whatever I wanted.

So if I were designing an architecture for this,
I would base it on the way Make goes about things.
That's something people are already familiar with,
so they would have some idea how to approach
extending and modifying it.

Note that, again, the point isn't really about
-F options, but that there are too few hooks
exposed for modifying things. Maybe those hooks
exist somewhere, but it seems you have to dig
very deep and understand a lot more than you
should have to in order to find and use them.

 > How would it operate on systems other than
 > the Mac?

It wouldn't have to, because I'm only talking
about tweaking someone's setup.py file so that
it works on my system.

A long-term solution to the framework problem
would be to build knowledge of frameworks into
the standard rules for compiling things on a
Mac. If the rules were isolated from each other
in a more Make-like way, this would be easier
as well.

> See, and I believe this isn't the case for distutils.

Then we seem to have a difference of opinion on
what the basic architecture should be like. I
think it should be more like Make, and I don't
see a way of getting there from here incrementally.

--
Greg

From greg.ewing at canterbury.ac.nz  Tue Oct 17 07:48:31 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 17 Oct 2006 18:48:31 +1300
Subject: [Python-3000] Cheeseshop
In-Reply-To: <200610162128.37557.fdrake@acm.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>
	<200610162128.37557.fdrake@acm.org>
Message-ID: <45346EAF.1080901@canterbury.ac.nz>

Fred L. Drake, Jr. wrote:
> The "Cheese Shop" name may reflect the 
> proper cultural bias toward Monty Python, but it's a sure way to alienate 
> people by presenting the hard work of the community under a completely silly 
> name that communicates nothing about what it's about.

Well, I don't think it's any worse than CPAN, which
doesn't mean anything unless you already know what
it stands for. It could even be misleading (suggesting
it has something to do with C) or taken as derogatory
(since one of the meanings of "pan" is to cast
aspersions).

As to the alternatives proposed so far, PyPI is
too easily confused with PyPy, especially when
spoken, and "Package Index" doesn't say it has
anything to do with Python. Besides, they're
boring.

Anyone have any other ideas? I've been trying to
make something out of SPAM -- Superlative Python
Archive of Modules?

--
Greg


From talin at acm.org  Tue Oct 17 08:12:44 2006
From: talin at acm.org (Talin)
Date: Mon, 16 Oct 2006 23:12:44 -0700
Subject: [Python-3000] Cheeseshop
In-Reply-To: <45346EAF.1080901@canterbury.ac.nz>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>	<200610162128.37557.fdrake@acm.org>
	<45346EAF.1080901@canterbury.ac.nz>
Message-ID: <4534745C.3020304@acm.org>

Greg Ewing wrote:
> Anyone have any other ideas? I've been trying to
> make something out of SPAM -- Superlative Python
> Archive of Modules?

Good luck getting that one through people's email filters :)

Cheeseshop may be overly cute, however it has at least this mnemonic 
benefit is that it suggests a "shop", in other words a place where you 
can browse through various products and select which ones you want. And 
its just silly enough that people won't think that its a real shop where 
you have to pay money for things.

At some point we're going to run out of all the really well-known Monty 
Python sketches, at which point people will either come to their senses, 
or we are going to see abominations like open source projects named 
"dead bishop" or something.

My own preference for naming is more along the lines of finding 
*exactly* the right word, based on the observation that there are a lot 
of programmers out there who are sloppy with their use of language, and 
will often use a word that "sort of" means what they are trying to say. 
This ties in with my interest in Thesaurii and other word tools.

-- Talin

From greg.ewing at canterbury.ac.nz  Tue Oct 17 08:28:41 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 17 Oct 2006 19:28:41 +1300
Subject: [Python-3000] Cheeseshop
In-Reply-To: <4534745C.3020304@acm.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>
	<200610162128.37557.fdrake@acm.org> <45346EAF.1080901@canterbury.ac.nz>
	<4534745C.3020304@acm.org>
Message-ID: <45347819.5000002@canterbury.ac.nz>

Talin wrote:
> Greg Ewing wrote:
> 
>>I've been trying to
>>make something out of SPAM -- Superlative Python
>>Archive of Modules?
> 
> Good luck getting that one through people's email filters :)

Well, I figure that real spam isn't going to announce
itself as such, so we should be all right. Besides,
I've never had trouble with messages containing
code snippets using the name 'spam'.

> Cheeseshop may be overly cute, however it has at least this mnemonic 
> benefit is that it suggests a "shop",

I actually quite like Cheese Shop too, and I'd be
perfectly happy for it to remain.

> This ties in with my interest in Thesaurii and other word tools.

That would just be "thesauri", wouldn't it? (The
singular is "thesaurus", not "thesaurius".)

--
Greg

From fdrake at acm.org  Tue Oct 17 08:39:08 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 17 Oct 2006 02:39:08 -0400
Subject: [Python-3000] Cheeseshop
In-Reply-To: <45346EAF.1080901@canterbury.ac.nz>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<200610162128.37557.fdrake@acm.org>
	<45346EAF.1080901@canterbury.ac.nz>
Message-ID: <200610170239.08591.fdrake@acm.org>

On Tuesday 17 October 2006 01:48, Greg Ewing wrote:
 > Well, I don't think it's any worse than CPAN, which
 > doesn't mean anything unless you already know what

The fact that it doesn't mean anything could just as easily be a point in 
favor; it isn't anything else, either.

 > As to the alternatives proposed so far, PyPI is
 > too easily confused with PyPy, especially when
 > spoken, and "Package Index" doesn't say it has
 > anything to do with Python. Besides, they're
 > boring.

Perhaps my recall of history is lacking, but I'd say PyPy is too easily 
confused with PyPI; wasn't PyPI around first, at least as a moniker?  I may 
be biased, since PyPI is my preferred name for it by far (the "Python Package 
Index" says exactly what it is, and "PyPI" isn't a real stretch as a short 
form).


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From fredrik at pythonware.com  Tue Oct 17 08:44:18 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 17 Oct 2006 08:44:18 +0200
Subject: [Python-3000] Cheeseshop
In-Reply-To: <200610170239.08591.fdrake@acm.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<200610162128.37557.fdrake@acm.org>	<45346EAF.1080901@canterbury.ac.nz>
	<200610170239.08591.fdrake@acm.org>
Message-ID: <eh1u44$p1v$2@sea.gmane.org>

Fred L. Drake, Jr. wrote:

 > (the "Python Package Index" says exactly what it is

fwiw, "package index" is the name used on the python.org home page.

</F>


From greg.ewing at canterbury.ac.nz  Tue Oct 17 09:41:56 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 17 Oct 2006 20:41:56 +1300
Subject: [Python-3000] Cheeseshop
In-Reply-To: <200610170239.08591.fdrake@acm.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<200610162128.37557.fdrake@acm.org> <45346EAF.1080901@canterbury.ac.nz>
	<200610170239.08591.fdrake@acm.org>
Message-ID: <45348944.90903@canterbury.ac.nz>

Fred L. Drake, Jr. wrote:

> Perhaps my recall of history is lacking, but I'd say PyPy is too easily 
> confused with PyPI; wasn't PyPI around first, at least as a moniker?

That may be so. But I like PyPy as a name for what it
is better than PyPI, so if one of them had to go, I
wouldn't want to lose PyPy.

How do you pronounce PyPI, btw? Is it "pie-pie" or
"pie-pee-eye"? (And don't tell me it's actually
pronounced "pippy" -- acronyms with non-obvious
pronunciations are a minor peeve of mine. People
are going to pronounce it the way they think it
looks, however much you try to educate them
otherwise, so it would be better to spell it the
way you want it pronounced in the first place.)

Another thing is that PyPI, and any other name
starting with Py, looks like the name of a Python
package. I thought it *was* some kind of package
the first few times I saw the acronym used.

And why is it PyPI and not PPI? It seems kind
of arbitrary.

Somehow, Cheese Shop works for me, on a variety
of levels. Not only does it suggest a place to go
to get things, it suggests an *interesting* place,
where I might find a variety of tasty morsels on
display. It invites me to go and browse. And if
I didn't know what it was, I'd be intrigued enough
to go there and find out.

Also, in this acronym-laden age, it has the advantage
of *not* being an acronym! There are far too many
TLAs and ETLAs being bandied around these days. We
need to rediscover the sublime art of using actual
words to talk about things.

HTH. YMMV. HAND.
YT, GCE

From solipsis at pitrou.net  Tue Oct 17 10:16:11 2006
From: solipsis at pitrou.net (Antoine)
Date: Tue, 17 Oct 2006 10:16:11 +0200 (CEST)
Subject: [Python-3000] Cheeseshop
In-Reply-To: <200610162128.37557.fdrake@acm.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>
	<200610162128.37557.fdrake@acm.org>
Message-ID: <46546.62.39.9.251.1161072971.squirrel@webmail.nerim.net>


> Which is really, really, REALLY sad.  The "Cheese Shop" name may reflect
> the
> proper cultural bias toward Monty Python, but it's a sure way to alienate
> people by presenting the hard work of the community under a completely
> silly
> name that communicates nothing about what it's about.

Isn't "Red Hat" silly ? How about "Power Point" ?
There are lots of silly names out there, it doesn't stop people from using
those products.

Cheeseshop is fine IMHO, and easier to spell correctly than PYpi (or is it
PiPy ? pyPY ?? whatever).



From ncoghlan at gmail.com  Tue Oct 17 11:00:55 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Oct 2006 19:00:55 +1000
Subject: [Python-3000] Cheeseshop
In-Reply-To: <45347819.5000002@canterbury.ac.nz>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>	<200610162128.37557.fdrake@acm.org>
	<45346EAF.1080901@canterbury.ac.nz>	<4534745C.3020304@acm.org>
	<45347819.5000002@canterbury.ac.nz>
Message-ID: <45349BC7.20606@gmail.com>

Greg Ewing wrote:
> Talin wrote:
>> Cheeseshop may be overly cute, however it has at least this mnemonic 
>> benefit is that it suggests a "shop",
> 
> I actually quite like Cheese Shop too, and I'd be
> perfectly happy for it to remain.

I became reconciled to the name when I realized it could be taken as saying 
that it's somewhere you won't find any cheese, but will find plenty of Python 
code :)

I must say I was surprised people found potential confusion between PyPy and 
PyPI, though. I'd always pronounced the latter as Py-Pea-Eye (and it was a 
tool for finding useful Python packages, so the other meaning of PI fit, too)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From p.f.moore at gmail.com  Tue Oct 17 11:59:39 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 17 Oct 2006 10:59:39 +0100
Subject: [Python-3000] Cheeseshop
In-Reply-To: <45348944.90903@canterbury.ac.nz>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<200610162128.37557.fdrake@acm.org>
	<45346EAF.1080901@canterbury.ac.nz>
	<200610170239.08591.fdrake@acm.org> <45348944.90903@canterbury.ac.nz>
Message-ID: <79990c6b0610170259s41fc292r2d1e8760f7d829d@mail.gmail.com>

On 10/17/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> How do you pronounce PyPI, btw? Is it "pie-pie" or
> "pie-pee-eye"? (And don't tell me it's actually
> pronounced "pippy" -- acronyms with non-obvious
> pronunciations are a minor peeve of mine. People
> are going to pronounce it the way they think it
> looks, however much you try to educate them
> otherwise, so it would be better to spell it the
> way you want it pronounced in the first place.)

Pie-pee. And PyPy is pie-pie. So to me, the two never really clashed at all.

Personally, I find "cheeseshop" a bit too cute - if I was talking to
someone about Python, I'd hesitate before mentioning the name. Given
the choice, I'd go back to PyPI, but either way, there should only be
one name, used everywhere.

On the "one place for everything" front, we're getting there - it's
far easier these days to put new packages onto the cheeseshop[1] with
the new distutils upload stuff. But for the "major" packages (things
like pywin32, PIL, numpy, twisted, pygame...) I wouldn't automatically
go to the cheeseshop for them. Once the canonical download location
for the big stuff is the cheeseshop, that's when the balance will have
tipped (IMHO). I don't know how likely that is to happen in practice,
though...

Paul.

[1] In the course of this message, I've also discovered another reason
I don't like "cheeseshop" - it's a pain to type... :-)

From barry at python.org  Tue Oct 17 13:50:05 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Oct 2006 07:50:05 -0400
Subject: [Python-3000] Cheeseshop
In-Reply-To: <200610162128.37557.fdrake@acm.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>
	<200610162128.37557.fdrake@acm.org>
Message-ID: <A99B9283-D9B6-47B2-9223-241776708457@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Oct 16, 2006, at 9:28 PM, Fred L. Drake, Jr. wrote:

> Which is really, really, REALLY sad.  The "Cheese Shop" name may  
> reflect the
> proper cultural bias toward Monty Python, but it's a sure way to  
> alienate
> people by presenting the hard work of the community under a  
> completely silly
> name that communicates nothing about what it's about.

I now invoke Warsaw's Fifth Law (a.k.a. "the Pink Floyd rule"): All  
names are stupid until you become rich and famous with it.

Cheese Shop seems fine to me (and not just 'cause I came up with  
it).  You /need/ a little humor and levity in a software project or  
it just gets boring.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRTTDc3EjvBPtnXfVAQJojAP+M6HOXbZWMMXqPkVlsSSoBSFrs31L23xv
y+QDnmjL31IHh/8aDjKoec7XpuDcHwUaY1obJ7p1UThyj8swcDdog0dBMXiVFXkf
E9ef3Dgh0Rl63OyHg+pHmOuieBpBOJ2xxP039OcqbnW0G3zSHLE3wHxoL6wL44VT
SsyWVSrx8hE=
=PJoZ
-----END PGP SIGNATURE-----

From fredrik at pythonware.com  Tue Oct 17 13:57:41 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 17 Oct 2006 13:57:41 +0200
Subject: [Python-3000] Cheeseshop
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com><Pine.LNX.4.58.0610161949120.3499@server1.LFW.org><6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com><200610162128.37557.fdrake@acm.org>
	<A99B9283-D9B6-47B2-9223-241776708457@python.org>
Message-ID: <eh2gfl$1vm$1@sea.gmane.org>

Barry Warsaw wrote:

> Cheese Shop seems fine to me (and not just 'cause I came up with
> it).  You /need/ a little humor and levity in a software project or
> it just gets boring.

using a "silly name" for the project, software, and even for the URL is one
thing.  using it to describe an important part of the infrastructure is another
thing.

</F> 




From python3000 at discworld.dyndns.org  Tue Oct 17 16:37:06 2006
From: python3000 at discworld.dyndns.org (Charles Cazabon)
Date: Tue, 17 Oct 2006 08:37:06 -0600
Subject: [Python-3000] Cheeseshop
In-Reply-To: <eh2gfl$1vm$1@sea.gmane.org>
References: <A99B9283-D9B6-47B2-9223-241776708457@python.org>
	<eh2gfl$1vm$1@sea.gmane.org>
Message-ID: <20061017143706.GA31727@discworld.dyndns.org>

Fredrik Lundh <fredrik at pythonware.com> wrote:
> Barry Warsaw wrote:
> 
> > Cheese Shop seems fine to me (and not just 'cause I came up with it).  You
> > /need/ a little humor and levity in a software project or it just gets
> > boring.
> 
> using a "silly name" for the project, software, and even for the URL is one
> thing.  using it to describe an important part of the infrastructure is
> another thing.

So calling it 'the "Cheese Shop" central Python module/package repository and
index' would be fine then?

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                       <python3000 at discworld.dyndns.org>
-----------------------------------------------------------------------

From martin at v.loewis.de  Tue Oct 17 19:35:31 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 17 Oct 2006 19:35:31 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45346B00.1000506@canterbury.ac.nz>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de> <4532E241.8070904@canterbury.ac.nz>
	<45330E9C.9030300@v.loewis.de> <45346B00.1000506@canterbury.ac.nz>
Message-ID: <45351463.8010804@v.loewis.de>

Greg Ewing schrieb:
> Not sure exactly what you mean by a "sub-command".
> If you mean a subclass of the class that implements
> the "build" command, that's far too heavyweight.

distutils has a special concept called a "subcommand"
(apparently, people creating such packages have an
intuition for creating hard-to-understand terms for
concepts).

Executing a command (such as build) split into
multiple sub-commands (build_py, build_ext, ...).
Invoking the build command will invoke all its
subcommands. For pyrex, creating a build_pyx command
might be appropriate.

> I want to deal just with the .pyx->.c transformation,
> and not have to worry that I might mess up something
> else that the build class does.

And you don't need to. The build command itself does nearly
nothing; it's all in the sub-commands.

You still have to override the build class, to rewrite
the get_sub_commands methods. Actually, just
providing a sub_commands attribute would be enough:

class BuildWithPyx(build):
  sub_commands = [('build_pyx', 'compile Pyrex files')
                 ] + build.sub_commands

> Also, what happens if I want to use my extended
> build class together with someone else's extended
> build class? Do I have to merge them using multiple
> inheritance?

If they also created a new sub-command for build,
you would have to make your get_sub_command implementation
return them both.

> Without knowing a *lot* about their
> internals, I'd be very nervous doing that lest they
> interact in unforeseen ways.

The sub-command acts on its own, and you don't interfere
with it. The order of sub-commands may matter: you likely
would run the build_pyx sub-command before build_ext,
e.g. by adding it at the beginning

> It *doesn't* generate any warnings using the
> normal, default C compiler settings -- only the
> extremely paranoid ones that the Python installation
> defaults to.

It uses -Wall -Wstrict-prototypes. That's *not*
"extremely paranoid". It is standard in any reasonable
C project.

> If I were dealing with a Makefile, and there
> weren't a macro to override that would have the
> desired effect, I could replace the whole rule
> for linking C files with one of my own that
> did whatever I wanted.

So extra_link_args for the Extension objects didn't
work for you?

> Then we seem to have a difference of opinion on
> what the basic architecture should be like. I
> think it should be more like Make, and I don't
> see a way of getting there from here incrementally.

Right. It's currently not declarative; it's primarily
imperative (at least wrt. how the build steps work,
not wrt. what is being built).

It would be possible to integrate declarative
features for the build steps also (like what you
wanted for the link step); to do that, some
dependency machinery (like SCons or zc.something)
could be integrated as the back-end for the
build steps.

Regards,
Martin

From mbk.lists at gmail.com  Tue Oct 17 19:36:40 2006
From: mbk.lists at gmail.com (Mike Krell)
Date: Tue, 17 Oct 2006 10:36:40 -0700
Subject: [Python-3000] Cheeseshop
In-Reply-To: <4534745C.3020304@acm.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>
	<200610162128.37557.fdrake@acm.org>
	<45346EAF.1080901@canterbury.ac.nz> <4534745C.3020304@acm.org>
Message-ID: <da7032ce0610171036w11fa84cbs693d565a63656735@mail.gmail.com>

> At some point we're going to run out of all the really well-known Monty
> Python sketches, at which point people will either come to their senses,
> or we are going to see abominations like open source projects named
> "dead bishop" or something.

Cf. "crunchy frog", a really cool application concept hobbled by a
so-nonsensical-as-to-not-even-be-cute name that can only serve as a
barrier to newbies.

Yes, I have a sense of humor, and yes, I enjoy Monty Python, but come on....

"Cheeseshop" is at least vaguely suggestive of its purpose.

IMHO, of course.  No offense intended.

   Mike

From fdrake at acm.org  Tue Oct 17 19:54:59 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 17 Oct 2006 13:54:59 -0400
Subject: [Python-3000] Cheeseshop
In-Reply-To: <45349BC7.20606@gmail.com>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45347819.5000002@canterbury.ac.nz> <45349BC7.20606@gmail.com>
Message-ID: <200610171354.59960.fdrake@acm.org>

On Tuesday 17 October 2006 05:00, Nick Coghlan wrote:
 > I must say I was surprised people found potential confusion between PyPy
 > and PyPI, though. I'd always pronounced the latter as Py-Pea-Eye (and it
 > was a tool for finding useful Python packages, so the other meaning of PI
 > fit, too)

I seem to recall Richard Jones saying he pronounces it "pippy", which I never 
could understand based on the spelling.

So, the biggest problem "PyPI" is that no one can agree on how to pronounce 
it.  (I pronounce it "Python Package Index".)


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From barry at python.org  Tue Oct 17 20:02:12 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Oct 2006 14:02:12 -0400
Subject: [Python-3000] Cheeseshop
In-Reply-To: <200610171354.59960.fdrake@acm.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45347819.5000002@canterbury.ac.nz> <45349BC7.20606@gmail.com>
	<200610171354.59960.fdrake@acm.org>
Message-ID: <550D5794-187F-4500-BFE2-2EF64D0D7BEE@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Oct 17, 2006, at 1:54 PM, Fred L. Drake, Jr. wrote:

> I seem to recall Richard Jones saying he pronounces it "pippy",  
> which I never
> could understand based on the spelling.
>
> So, the biggest problem "PyPI" is that no one can agree on how to  
> pronounce
> it.  (I pronounce it "Python Package Index".)

No no no, it's pronounced "Throatwobbler Mangrove".

speaking-pythonical-ly y'rs,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRTUapHEjvBPtnXfVAQKMcwQAjbMSJBBhx40OEq2L5t1S87XG9/GBBH7p
zpnCr8gAN3eSmatiS4cSqUp4a2qUdyDEE19imk/bH8eJAi/IE++vvyAfucDZDybB
Brhk66p6ToS3zaWV+0n+Bq4nUUFVSDBZpmhiTJ8lFQe3KzITqMSd4mQ/VJE2FPcz
dNBqmw30jwY=
=lIAP
-----END PGP SIGNATURE-----

From paul at prescod.net  Tue Oct 17 20:59:14 2006
From: paul at prescod.net (Paul Prescod)
Date: Tue, 17 Oct 2006 11:59:14 -0700
Subject: [Python-3000] Cheeseshop
In-Reply-To: <550D5794-187F-4500-BFE2-2EF64D0D7BEE@python.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45347819.5000002@canterbury.ac.nz> <45349BC7.20606@gmail.com>
	<200610171354.59960.fdrake@acm.org>
	<550D5794-187F-4500-BFE2-2EF64D0D7BEE@python.org>
Message-ID: <1cb725390610171159q3a2f9104hc7981b0041921e7a@mail.gmail.com>

Perhaps it should be Python Package Repository (PYPR).

Or even the Py'd Pypr.

Like other package repositories it would sing you a nice song at first
and then when the versioning trouble hits you'll feel like you're
drowning.

 Paul Prescod

On 10/17/06, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Oct 17, 2006, at 1:54 PM, Fred L. Drake, Jr. wrote:
>
> > I seem to recall Richard Jones saying he pronounces it "pippy",
> > which I never
> > could understand based on the spelling.
> >
> > So, the biggest problem "PyPI" is that no one can agree on how to
> > pronounce
> > it.  (I pronounce it "Python Package Index".)
>
> No no no, it's pronounced "Throatwobbler Mangrove".
>
> speaking-pythonical-ly y'rs,
> - -Barry
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (Darwin)
>
> iQCVAwUBRTUapHEjvBPtnXfVAQKMcwQAjbMSJBBhx40OEq2L5t1S87XG9/GBBH7p
> zpnCr8gAN3eSmatiS4cSqUp4a2qUdyDEE19imk/bH8eJAi/IE++vvyAfucDZDybB
> Brhk66p6ToS3zaWV+0n+Bq4nUUFVSDBZpmhiTJ8lFQe3KzITqMSd4mQ/VJE2FPcz
> dNBqmw30jwY=
> =lIAP
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/paul%40prescod.net
>

From tjreedy at udel.edu  Tue Oct 17 22:28:50 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 17 Oct 2006 16:28:50 -0400
Subject: [Python-3000] Cheeseshop
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com><Pine.LNX.4.58.0610161949120.3499@server1.LFW.org><6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com><200610162128.37557.fdrake@acm.org>
	<45346EAF.1080901@canterbury.ac.nz>
Message-ID: <eh3ee0$a5j$1@sea.gmane.org>


"Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote in message 
news:45346EAF.1080901 at canterbury.ac.nz...
> Fred L. Drake, Jr. wrote:
>> The "Cheese Shop" name may reflect the
>> proper cultural bias toward Monty Python, but it's a sure way to 
>> alienate
>> people by presenting the hard work of the community under a completely 
>> silly
>> name that communicates nothing about what it's about.

As someone who did *not* get the M.P. connection, I found it pretty nutty 
and confusing.  I aggrreee with Fred.

> Anyone have any other ideas?

Pie/Pi/Py/Pyeshop -- pun intended

Package index (pi) shop
Python extension/expansion (pye) shop
Python index of extensions (pie) shop

Take your pick of acronyms, but pie is yummy in about any flavor.

Terry Jan Reedy




From phd at phd.pp.ru  Tue Oct 17 22:36:05 2006
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Wed, 18 Oct 2006 00:36:05 +0400
Subject: [Python-3000] Cheeseshop
In-Reply-To: <eh3ee0$a5j$1@sea.gmane.org>
References: <45346EAF.1080901@canterbury.ac.nz> <eh3ee0$a5j$1@sea.gmane.org>
Message-ID: <20061017203605.GD9534@phd.pp.ru>

On Tue, Oct 17, 2006 at 04:28:50PM -0400, Terry Reedy wrote:
> Pie/Pi/Py/Pyeshop -- pun intended
> 
> Package index (pi) shop
> Python extension/expansion (pye) shop
> Python index of extensions (pie) shop

   Peggy - Python eggs repositorY ;) Or just PER.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From barry at python.org  Tue Oct 17 22:41:15 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Oct 2006 16:41:15 -0400
Subject: [Python-3000] Cheeseshop
In-Reply-To: <eh3ee0$a5j$1@sea.gmane.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com><Pine.LNX.4.58.0610161949120.3499@server1.LFW.org><6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com><200610162128.37557.fdrake@acm.org>
	<45346EAF.1080901@canterbury.ac.nz> <eh3ee0$a5j$1@sea.gmane.org>
Message-ID: <53457157-5081-459E-9B80-35FAF6430443@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Oct 17, 2006, at 4:28 PM, Terry Reedy wrote:

> Pie/Pi/Py/Pyeshop -- pun intended
>
> Package index (pi) shop
> Python extension/expansion (pye) shop
> Python index of extensions (pie) shop
>
> Take your pick of acronyms, but pie is yummy in about any flavor.

Maybe we should call it the PyWhole as in, it makes your Python whole.

- -B

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRTU/8HEjvBPtnXfVAQINgwP/fpw2MGmS+/RQicO/Yw03lPotnxY1OcWo
NpLirVObK1bm1OevTIhwVKmIQbe7yoyxzHrsU/m53F4MFJ5Vx3YPDiqPArWMk4z3
VfNRzAmT9lxd1XvqXNh4sm66upjDSc4qZuEMcKsBaoLVcUcQN5fjb0KdWSig5tTs
0N9FAeEcnpM=
=TPFT
-----END PGP SIGNATURE-----

From janssen at parc.com  Wed Oct 18 00:29:55 2006
From: janssen at parc.com (Bill Janssen)
Date: Tue, 17 Oct 2006 15:29:55 PDT
Subject: [Python-3000] Cheeseshop
In-Reply-To: Your message of "Tue, 17 Oct 2006 13:41:15 PDT."
	<53457157-5081-459E-9B80-35FAF6430443@python.org> 
Message-ID: <06Oct17.152956pdt."58648"@synergy1.parc.xerox.com>

I like the PyPR -- "piper" or "pie-per" -- suggestion.  Good sound,
reasonable acronym, fairly straightforward pronunciation.

Bill

From greg.ewing at canterbury.ac.nz  Wed Oct 18 02:34:22 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 18 Oct 2006 13:34:22 +1300
Subject: [Python-3000] Cheeseshop
In-Reply-To: <eh2gfl$1vm$1@sea.gmane.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>
	<200610162128.37557.fdrake@acm.org>
	<A99B9283-D9B6-47B2-9223-241776708457@python.org>
	<eh2gfl$1vm$1@sea.gmane.org>
Message-ID: <4535768E.9090400@canterbury.ac.nz>

Fredrik Lundh wrote:
> using a "silly name" for the project, software, and even for the URL is one
> thing.  using it to describe an important part of the infrastructure is another
> thing.

If it's okay to use a silly name for the whole
project, why not for part of its infrastructure?

--
Greg

From greg.ewing at canterbury.ac.nz  Wed Oct 18 02:34:35 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 18 Oct 2006 13:34:35 +1300
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <45351463.8010804@v.loewis.de>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de> <4532E241.8070904@canterbury.ac.nz>
	<45330E9C.9030300@v.loewis.de> <45346B00.1000506@canterbury.ac.nz>
	<45351463.8010804@v.loewis.de>
Message-ID: <4535769B.4000703@canterbury.ac.nz>

Martin v. L?wis wrote:

> If they also created a new sub-command for build,
> you would have to make your get_sub_command implementation
> return them both.

If these sub-commands really are independent,
why can't I just add one to a list somewhere?
Then it would be no problem at all for different
extensions to coexist.

> The sub-command acts on its own, and you don't interfere
> with it. The order of sub-commands may matter: you likely
> would run the build_pyx sub-command before build_ext,
> e.g. by adding it at the beginning

This is another problem. The order in which things
are done, and whether things are done at all,
should be driven by the dependency graph. You
shouldn't have to program it explicitly.

> It uses -Wall -Wstrict-prototypes. That's *not*
> "extremely paranoid". It is standard in any reasonable
> C project.

It might be considered reasonable for human-written
code, but it's not reasonable to expect machine
generated code to pass such a strict test. It serves
no useful purpose to complain about things like
unused variables and labels when the C compiler
is quite capable of optimising them away, thus
saving the Pyrex compiler from having to do that
itself.

> So extra_link_args for the Extension objects didn't
> work for you?

No. Like I said, there are circumstances in which
the -F options need to go at the beginning of the
link command, but the extra_link_args get put at
the end.

The point about this is that no matter how many
options you provide to a command, there will always
be circumstances in which they're not sufficient.
So there needs to be a clear and understandable
way to override anything at any stage in the
process.

> It would be possible to integrate declarative
> features for the build steps also (like what you
> wanted for the link step); to do that, some
> dependency machinery (like SCons or zc.something)
> could be integrated as the back-end for the
> build steps.

And if you're doing that, why bother with the
build steps layer at all? Why not just let the
user specify the dependency graph directly and
let it rip?

--
Greg

From martin at v.loewis.de  Wed Oct 18 07:15:25 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 18 Oct 2006 07:15:25 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4535769B.4000703@canterbury.ac.nz>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de> <4532E241.8070904@canterbury.ac.nz>
	<45330E9C.9030300@v.loewis.de> <45346B00.1000506@canterbury.ac.nz>
	<45351463.8010804@v.loewis.de> <4535769B.4000703@canterbury.ac.nz>
Message-ID: <4535B86D.6080307@v.loewis.de>

Greg Ewing schrieb:
>> If they also created a new sub-command for build,
>> you would have to make your get_sub_command implementation
>> return them both.
> 
> If these sub-commands really are independent,
> why can't I just add one to a list somewhere?
> Then it would be no problem at all for different
> extensions to coexist.

You can, sure. distutils.commands.build.sub_commands.
To do so, you either have to edit the source code of
build.py, or you have to arrange that the addition
is done programmatically somehow when setup.py runs.

> It might be considered reasonable for human-written
> code, but it's not reasonable to expect machine
> generated code to pass such a strict test.

Given that it is *generated* code, I find that very
reasonable. Just fix the generator once, and the
warnings will go away. For hand-written code, any
new piece of code might cause a new warning. For
generated code, because of the systematic way, the
warnings can be removed in the generator, not in the
input (the pyx file). I've been writing generators
that produce C++ for years, and never found it a
problem make it write warning-less, legible code.

> The point about this is that no matter how many
> options you provide to a command, there will always
> be circumstances in which they're not sufficient.
> So there needs to be a clear and understandable
> way to override anything at any stage in the
> process.

Ah, ok. So you'd want a "build command override" for an
extension object. Such a feature could be added to
distutils.

> And if you're doing that, why bother with the
> build steps layer at all? Why not just let the
> user specify the dependency graph directly and
> let it rip?

build_py, for example, *only* copies the source
files into the build tree (with time stamp checking).
The user shouldn't have to specify this, because
a) it is tedious and error-prone to do so for each
   source file, and
b) you can't know in a static way what the target
   directory is, because the platform name goes
   into the target name.

Regards,
Martin

From fredrik at pythonware.com  Wed Oct 18 08:29:19 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 18 Oct 2006 08:29:19 +0200
Subject: [Python-3000] Cheeseshop
In-Reply-To: <4535768E.9090400@canterbury.ac.nz>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>	<200610162128.37557.fdrake@acm.org>	<A99B9283-D9B6-47B2-9223-241776708457@python.org>	<eh2gfl$1vm$1@sea.gmane.org>
	<4535768E.9090400@canterbury.ac.nz>
Message-ID: <eh4hk0$t66$1@sea.gmane.org>

Greg Ewing wrote:

> If it's okay to use a silly name for the whole
> project, why not for part of its infrastructure?

because it's difficult enough to do a Python elevator pitch as it is ?

(why is a package index so different from things like "documentation" 
and "source" and "installer" and "bug tracker" and "conference", by the 
way?  I'm sure we could get tons of cute name proposals if we asked the 
community...)

</F>


From talin at acm.org  Wed Oct 18 09:23:53 2006
From: talin at acm.org (Talin)
Date: Wed, 18 Oct 2006 00:23:53 -0700
Subject: [Python-3000] Cheeseshop
In-Reply-To: <eh4hk0$t66$1@sea.gmane.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>	<200610162128.37557.fdrake@acm.org>	<A99B9283-D9B6-47B2-9223-241776708457@python.org>	<eh2gfl$1vm$1@sea.gmane.org>	<4535768E.9090400@canterbury.ac.nz>
	<eh4hk0$t66$1@sea.gmane.org>
Message-ID: <4535D689.2040202@acm.org>

Fredrik Lundh wrote:
> Greg Ewing wrote:
> 
>> If it's okay to use a silly name for the whole
>> project, why not for part of its infrastructure?
> 
> because it's difficult enough to do a Python elevator pitch as it is ?
> 
> (why is a package index so different from things like "documentation" 
> and "source" and "installer" and "bug tracker" and "conference", by the 
> way?  I'm sure we could get tons of cute name proposals if we asked the 
> community...)

OK I'll bite:

documentation ==> "News for Parrots"
source ==> "Confuse-a-Cat"
installer ==> "Society for Putting Things on top of Other Things"
bug tracker ==> "Spanish Inquisition"
conference ==> "Argument Clinic"

Seriously, though - this is actually my way of saying that this thread 
has gone too long...what are we hoping to accomplish here? Are you 
really going to re-name PyPI or cheeseshop or whatever you want to call 
it, or are folks just trying to score points? Even if its not the ideal 
name, is the cost of the disruption really worth it at this point?

To be honest, because I always use easy_install to install packages, I 
never actually *see* the name of the package index.

You could call it 82.94.237.219 for all I care.

-- Talin


From behnel_ml at gkec.informatik.tu-darmstadt.de  Wed Oct 18 09:02:37 2006
From: behnel_ml at gkec.informatik.tu-darmstadt.de (Stefan Behnel)
Date: Wed, 18 Oct 2006 09:02:37 +0200
Subject: [Python-3000] Cheeseshop
In-Reply-To: <200610171354.59960.fdrake@acm.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45347819.5000002@canterbury.ac.nz>
	<45349BC7.20606@gmail.com> <200610171354.59960.fdrake@acm.org>
Message-ID: <4535D18D.9080302@gkec.informatik.tu-darmstadt.de>

Hi,

Fred L. Drake, Jr. wrote:
> On Tuesday 17 October 2006 05:00, Nick Coghlan wrote:
>  > I must say I was surprised people found potential confusion between PyPy
>  > and PyPI, though. I'd always pronounced the latter as Py-Pea-Eye (and it
>  > was a tool for finding useful Python packages, so the other meaning of PI
>  > fit, too)
> 
> I seem to recall Richard Jones saying he pronounces it "pippy", which I never 
> could understand based on the spelling.

Time to learn the rules of English pronunciation, then. This may help:

http://mipmip.org/tidbits/pronunciation.shtml

Stefan

From talin at acm.org  Wed Oct 18 09:49:44 2006
From: talin at acm.org (Talin)
Date: Wed, 18 Oct 2006 00:49:44 -0700
Subject: [Python-3000] Cheeseshop
In-Reply-To: <eh3ee0$a5j$1@sea.gmane.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com><Pine.LNX.4.58.0610161949120.3499@server1.LFW.org><6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com><200610162128.37557.fdrake@acm.org>	<45346EAF.1080901@canterbury.ac.nz>
	<eh3ee0$a5j$1@sea.gmane.org>
Message-ID: <4535DC98.9030906@acm.org>

Terry Reedy wrote:
> Pie/Pi/Py/Pyeshop -- pun intended

'PyeShoppe' brings back fond Renaissance Faire memories :)

-- Talin

From python at zesty.ca  Thu Oct 19 03:22:04 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Wed, 18 Oct 2006 20:22:04 -0500 (CDT)
Subject: [Python-3000] Cheeseshop
In-Reply-To: <4535D689.2040202@acm.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>
	<200610162128.37557.fdrake@acm.org>
	<A99B9283-D9B6-47B2-9223-241776708457@python.org>
	<eh2gfl$1vm$1@sea.gmane.org> <4535768E.9090400@canterbury.ac.nz>
	<eh4hk0$t66$1@sea.gmane.org> <4535D689.2040202@acm.org>
Message-ID: <Pine.LNX.4.58.0610182015090.3499@server1.LFW.org>

On Wed, 18 Oct 2006, Talin wrote:
> Seriously, though - this is actually my way of saying that this thread
> has gone too long...what are we hoping to accomplish here? Are you
> really going to re-name PyPI or cheeseshop or whatever you want to call
> it, or are folks just trying to score points? Even if its not the ideal
> name, is the cost of the disruption really worth it at this point?

I feel there are two things that would be good to resolve here.

1.  We need to agree on a single standard name for the central
    repository of Python packages.  Right now there are 2.5
    (Cheeseshop, PyPI, Package Index).  The URL should reflect
    the name in an obvious way.

2.  We need to pick a name that is easy to spell and pronounce.
    ("PyPI", "pieshop", "pyshop", "pyeshop" all have the problem
    that one is unlikely to guess the correct spelling.)

But that's just about the name.  The original motivation for my
post about Cheeseshop, is that it would be nice if it were easier
to find packages in it.  The issue of search/navigation/access
seems to have been forgotten in this flurry about the name.


-- ?!ng

From greg.ewing at canterbury.ac.nz  Thu Oct 19 04:41:03 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 19 Oct 2006 15:41:03 +1300
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4535B86D.6080307@v.loewis.de>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de> <4532E241.8070904@canterbury.ac.nz>
	<45330E9C.9030300@v.loewis.de> <45346B00.1000506@canterbury.ac.nz>
	<45351463.8010804@v.loewis.de> <4535769B.4000703@canterbury.ac.nz>
	<4535B86D.6080307@v.loewis.de>
Message-ID: <4536E5BF.8050703@canterbury.ac.nz>

Martin v. L?wis wrote:

> Given that it is *generated* code, I find that very
> reasonable. Just fix the generator once, and the
> warnings will go away.

I don't accept that the code generator needs to be
"fixed", because I don't regard it as broken. The
code it generates is perfectly correct. Its only
crime is not conforming to some C compiler writer's
notion of what spotlessly tidy C code should look
like. That's completely irrelevant for Pyrex-
generated code, since humans aren't required to
ever look at it.

I'm using the C compiler as an assembler. I don't
expect my assembler to make moral judgements about
my code. I expect it to do what I tell it and not
talk back.

> build_py, for example, *only* copies the source
> files into the build tree (with time stamp checking).
> The user shouldn't have to specify this, because
> a) it is tedious and error-prone to do so for each
>    source file, and
> b) you can't know in a static way what the target
>    directory is, because the platform name goes
>    into the target name.

I'm not sure what your point is here. I don't see
why this can't be done under a Make-like system.
The user just needs a way to say "My package depends
on these .py files". The system can still be left to
figure out where they need to go and whether they need
updating.

If you're saying the user needs the ability to
explicitly invoke just that part of the process on
its own, that could be arranged, similarly to the
way you can tell Make to build a specific target.

I don't see why it's all that useful, though. If
the only thing that's changed is your .py files,
running the whole process will still only update
the .py files, due to the dependency tracking.
The only time you really need something like that
is when you *don't* have full dependency tracking.

--
Greg

From talin at acm.org  Thu Oct 19 04:43:14 2006
From: talin at acm.org (Talin)
Date: Wed, 18 Oct 2006 19:43:14 -0700
Subject: [Python-3000] Cheeseshop
In-Reply-To: <Pine.LNX.4.58.0610182015090.3499@server1.LFW.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>
	<200610162128.37557.fdrake@acm.org>
	<A99B9283-D9B6-47B2-9223-241776708457@python.org>
	<eh2gfl$1vm$1@sea.gmane.org> <4535768E.9090400@canterbury.ac.nz>
	<eh4hk0$t66$1@sea.gmane.org> <4535D689.2040202@acm.org>
	<Pine.LNX.4.58.0610182015090.3499@server1.LFW.org>
Message-ID: <4536E642.5040604@acm.org>

Ka-Ping Yee wrote:

> But that's just about the name.  The original motivation for my
> post about Cheeseshop, is that it would be nice if it were easier
> to find packages in it.  The issue of search/navigation/access
> seems to have been forgotten in this flurry about the name.

The package server doesn't seem to have any problem searching for names 
when they are typed from the shell command line, which is why it seems 
odd to me that you can't find things on the web-based interface. Two 
different search algorithms?

-- Talin


From fdrake at acm.org  Thu Oct 19 07:18:40 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu, 19 Oct 2006 01:18:40 -0400
Subject: [Python-3000] Cheeseshop
In-Reply-To: <4535D18D.9080302@gkec.informatik.tu-darmstadt.de>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<200610171354.59960.fdrake@acm.org>
	<4535D18D.9080302@gkec.informatik.tu-darmstadt.de>
Message-ID: <200610190118.40482.fdrake@acm.org>

On Wednesday 18 October 2006 03:02, Stefan Behnel wrote:
 > Time to learn the rules of English pronunciation, then. This may help:
 >
 > http://mipmip.org/tidbits/pronunciation.shtml

That's nice.  :)  I definately find myself predisposed to pronounce the "Py" 
as "pie", and I find that hard to get past.  Oh, well...


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From solipsis at pitrou.net  Thu Oct 19 10:04:50 2006
From: solipsis at pitrou.net (Antoine)
Date: Thu, 19 Oct 2006 10:04:50 +0200 (CEST)
Subject: [Python-3000] pyrex vs. gcc -Wall
In-Reply-To: <4536E5BF.8050703@canterbury.ac.nz>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de> <4532E241.8070904@canterbury.ac.nz>
	<45330E9C.9030300@v.loewis.de> <45346B00.1000506@canterbury.ac.nz>
	<45351463.8010804@v.loewis.de> <4535769B.4000703@canterbury.ac.nz>
	<4535B86D.6080307@v.loewis.de> <4536E5BF.8050703@canterbury.ac.nz>
Message-ID: <15305.62.39.9.251.1161245090.squirrel@webmail.nerim.net>


Hi,

> I don't accept that the code generator needs to be
> "fixed", because I don't regard it as broken. The
> code it generates is perfectly correct. Its only
> crime is not conforming to some C compiler writer's
> notion of what spotlessly tidy C code should look
> like. That's completely irrelevant for Pyrex-
> generated code, since humans aren't required to
> ever look at it.

In the rare cases where the C compiler gives an error on Pyrex-generated
code, I've found it desirable to read the C code to diagnose the problem.
It happened to me a few times when using function pointers and casts
(IIRC). Pyrex would generate C code but gcc would choke on it.

That's when you realize that C code generated by Pyrex is not very
readable ;-P
But Pyrex is a great tool anyway.

Regards

Antoine.



From martin at v.loewis.de  Sat Oct 21 17:33:45 2006
From: martin at v.loewis.de (=?ISO-8859-2?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Oct 2006 17:33:45 +0200
Subject: [Python-3000] Pre-PEP: Easy Text File Decoding
In-Reply-To: <87u0242id8.fsf@qrnik.zagroda>
References: <1cb725390609092029v6dae753ajf8bf85e4974bb4d0@mail.gmail.com>	<87d5a366xd.fsf@qrnik.zagroda>	<1cb725390609102142m3a1dd33ha8400ec8e2005ba@mail.gmail.com>	<8764fu3cxy.fsf@qrnik.zagroda>
	<45311859.6080102@v.loewis.de>	<87odsed9ct.fsf@qrnik.zagroda>
	<4531324E.4040206@v.loewis.de>	<87zmbyejs6.fsf@qrnik.zagroda>
	<4531D495.5030909@v.loewis.de> <87u0242id8.fsf@qrnik.zagroda>
Message-ID: <453A3DD9.2060308@v.loewis.de>

Marcin 'Qrczak' Kowalczyk schrieb:
>> I still think that PUA characters would be a better use
> 
> What if the filename contains the correct UTF-8 encoding of such PUA
> character?

Escaping would need to be used, e.g. by picking some character as the
escape character.

Regards,
Martin

From martin at v.loewis.de  Sat Oct 21 17:41:36 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Oct 2006 17:41:36 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <20061016141143.GB14145@panix.com>
References: <45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45321E83.2030401@v.loewis.de>
	<4532E241.8070904@canterbury.ac.nz>	<45330E9C.9030300@v.loewis.de>
	<20061016141143.GB14145@panix.com>
Message-ID: <453A3FB0.7040804@v.loewis.de>

Aahz schrieb:
>> See, and I believe this isn't the case for distutils.
> 
> Can you suggest a good mechanism for resolving the deadlock?  We have the
> primary maintainer (and some other users) claiming that distutils is a
> mess that needs rewriting.  You think it doesn't, but based on other
> comments you've made, I believe you don't have available time.  I haven't
> seen anyone else agreeing with you.

There isn't really a deadlock until some action gets blocked. Discussion
can't really deadlock, only livelock (i.e. go in circles). I believe
Philipp Eby (assuming he is the one you mentioned as the "primary
maintainer" - distutils in itself has no maintainer) understands my
concerns, and if he eventually proposes an action, he'll consider them.
So far, he hasn't proposed any action that got blocked (atleast not for
Py3k), so there wasn't any deadlock (yet).

However, I'm seriously concerned about libraries with only a single
maintainer. I believe this is what caused the problems with distutils
(the single maintainer ran away), and it can potentially cause problems
with any replacement of it, if that replacement has only a single
maintainer, as well.

Regards,
Martin

From martin at v.loewis.de  Sat Oct 21 17:47:01 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Oct 2006 17:47:01 +0200
Subject: [Python-3000] Cheeseshop
In-Reply-To: <Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>	<4532E25A.2090607@canterbury.ac.nz>
	<45330F2A.6010202@v.loewis.de>	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>	<4533BB3A.1040709@solarsail.hcs.harvard.edu>	<4533C5FB.6090708@v.loewis.de>	<4533CB68.5040706@solarsail.hcs.harvard.edu>	<4533E0E8.6000700@v.loewis.de>
	<4533E49A.9030800@acm.org>
	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
Message-ID: <453A40F5.3080300@v.loewis.de>

Ka-Ping Yee schrieb:
> I don't want to hurt anyone's feelings, but i think the site could
> really use improved search and navigation.  It used to have a search
> form that was (in my opinion) broken enough to be misleading, and now
> the search form appears to be gone.  The Google-based form feels like
> a stopgap measure rather than a proper search.

Unfortunately, it appears that people disagree about the value of that
specific search form: some say that bringing it back as-is would be
an improvement over the status quo, you seem to be saying that it
would make things worse.

In any case, that's "just" code. Contributions are welcome. I hope
nobody seriously considers writing a package registry/repository
from scratch just because the current one doesn't have a good
search feature. Instead, people willing to contribute should just
do so.

> Right now, the site doesn't provide an obvious, direct way to get
> to a package *even if you know the exact name of the package*.

As people pointed out on the SIG list: sure it does. If you
know the package name is "ctypes", you enter into your browser

http://cheeseshop.python.org/pypi/ctypes

(I know this works because I implemented it)

Regards,
Martin

From martin at v.loewis.de  Sat Oct 21 17:52:39 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Oct 2006 17:52:39 +0200
Subject: [Python-3000] Cheeseshop
In-Reply-To: <eh4hk0$t66$1@sea.gmane.org>
References: <5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>	<6a36e7290610161806m7ae11481l5ba50f1d9975b948@mail.gmail.com>	<200610162128.37557.fdrake@acm.org>	<A99B9283-D9B6-47B2-9223-241776708457@python.org>	<eh2gfl$1vm$1@sea.gmane.org>	<4535768E.9090400@canterbury.ac.nz>
	<eh4hk0$t66$1@sea.gmane.org>
Message-ID: <453A4247.1050601@v.loewis.de>

Fredrik Lundh schrieb:
> (why is a package index so different from things like "documentation" 
> and "source" and "installer" and "bug tracker" and "conference", by the 
> way?  I'm sure we could get tons of cute name proposals if we asked the 
> community...)

And, indeed, the Python conference was called "SPAM" for a while.

Regards,
Martin

From fredrik at pythonware.com  Sat Oct 21 18:36:46 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat, 21 Oct 2006 18:36:46 +0200
Subject: [Python-3000] Cheeseshop
In-Reply-To: <453A40F5.3080300@v.loewis.de>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>	<4532E25A.2090607@canterbury.ac.nz>	<45330F2A.6010202@v.loewis.de>	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>	<4533BB3A.1040709@solarsail.hcs.harvard.edu>	<4533C5FB.6090708@v.loewis.de>	<4533CB68.5040706@solarsail.hcs.harvard.edu>	<4533E0E8.6000700@v.loewis.de>	<4533E49A.9030800@acm.org>	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>
	<453A40F5.3080300@v.loewis.de>
Message-ID: <ehdiaj$hsp$1@sea.gmane.org>

Martin v. L?wis wrote:

>> Right now, the site doesn't provide an obvious, direct way to get
>> to a package *even if you know the exact name of the package*.
> 
> As people pointed out on the SIG list: sure it does. If you
> know the package name is "ctypes", you enter into your browser
> 
> http://cheeseshop.python.org/pypi/ctypes

I don't thing Ping had URL hacking in mind when he wrote "obvious, 
direct way".  even a heavily restian site like del.icio.us has an URL 
manipulation interface available *inside* the browser window.  maybe
we should steal their approach?

</F>


From martin at v.loewis.de  Sun Oct 22 06:57:10 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 22 Oct 2006 06:57:10 +0200
Subject: [Python-3000] Cheeseshop
In-Reply-To: <ehdiaj$hsp$1@sea.gmane.org>
References: <B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<5.1.1.6.0.20061015112240.026e5850@sparrow.telecommunity.com>	<4532E25A.2090607@canterbury.ac.nz>	<45330F2A.6010202@v.loewis.de>	<B6C933FC-BBAC-48D1-9689-0E4DA7F6B447@mac.com>	<5.1.1.6.0.20061016123402.03cf9638@sparrow.telecommunity.com>	<4533BB3A.1040709@solarsail.hcs.harvard.edu>	<4533C5FB.6090708@v.loewis.de>	<4533CB68.5040706@solarsail.hcs.harvard.edu>	<4533E0E8.6000700@v.loewis.de>	<4533E49A.9030800@acm.org>	<Pine.LNX.4.58.0610161949120.3499@server1.LFW.org>	<453A40F5.3080300@v.loewis.de>
	<ehdiaj$hsp$1@sea.gmane.org>
Message-ID: <453AFA26.7020700@v.loewis.de>

Fredrik Lundh schrieb:
> I don't thing Ping had URL hacking in mind when he wrote "obvious, 
> direct way".  even a heavily restian site like del.icio.us has an URL 
> manipulation interface available *inside* the browser window.  maybe
> we should steal their approach?

This brings up the question again who this "we" is people keep
mentioning. If I meet them, I mention your idea to them, perhaps
we will indeed steal their approach when we learn that we could.

Regards,
Martin

From ndunn at ndunn.com  Sun Oct 22 13:15:47 2006
From: ndunn at ndunn.com (Neil Dunn)
Date: Sun, 22 Oct 2006 12:15:47 +0100
Subject: [Python-3000] Optional type checking for Python
In-Reply-To: <f56dda5c0610220350q6a48684dw2826f89cb8552923@mail.gmail.com>
References: <f56dda5c0610210708p3508ac4axca28de31e9b4f07d@mail.gmail.com>
	<f56dda5c0610220350q6a48684dw2826f89cb8552923@mail.gmail.com>
Message-ID: <f56dda5c0610220415s7ea5407ap5243826da12b0dd7@mail.gmail.com>

Dear All,

(I'm forwarding this from the main pydev mailing list)

I'm a Master's student at Imperial College London currently selecting
a Master's thesis subject. I am exploring the possibility of "optional
typing" and "pluggable type systems" (Bracha) for Python. Reading
around I see that the closest we ever got was PEP 246 (object
adaption) which has been dropped in favour of generic functions.

Guido informs me optional static typing probably won't appear in
Python 3 so I'm safe exploring the ideas for my thesis. I'm planning
on making the project quite research based and explore implementations
in CPython or Jython.

Does anyone have any thoughts about the project? Useful background
reading, ideas, flames etc?

Thanks,
Neil Dunn

From martin at v.loewis.de  Sun Oct 22 16:29:09 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 22 Oct 2006 16:29:09 +0200
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <4536E5BF.8050703@canterbury.ac.nz>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de> <4532E241.8070904@canterbury.ac.nz>
	<45330E9C.9030300@v.loewis.de> <45346B00.1000506@canterbury.ac.nz>
	<45351463.8010804@v.loewis.de> <4535769B.4000703@canterbury.ac.nz>
	<4535B86D.6080307@v.loewis.de> <4536E5BF.8050703@canterbury.ac.nz>
Message-ID: <453B8035.3000907@v.loewis.de>

Greg Ewing schrieb:
>> Given that it is *generated* code, I find that very
>> reasonable. Just fix the generator once, and the
>> warnings will go away.
> 
> I don't accept that the code generator needs to be
> "fixed", because I don't regard it as broken. The
> code it generates is perfectly correct.

If that is so, just ignore the warnings. If the fact
that warnings are issued constitutes a problem for you,
one way to solve the problem is to make the warnings
go away.

>> The user shouldn't have to specify this, because
>> a) it is tedious and error-prone to do so for each
>>    source file, and
>> b) you can't know in a static way what the target
>>    directory is, because the platform name goes
>>    into the target name.
> 
> I'm not sure what your point is here. I don't see
> why this can't be done under a Make-like system.
> The user just needs a way to say "My package depends
> on these .py files". The system can still be left to
> figure out where they need to go and whether they need
> updating.

I'm wondering what a make-like system would look
in this case. Would I write

$(TARGETDIR)/foo/bar.py: foo/bar.py
    cp foo/bar.py $(TARGETDIR)/foo/bar.py

once for each source file? That would be tedious.
What would I write instead, and how could I then
override the actual command that is being run
(cp(1) in this case).

If you look at Python's "make install" rule, you
might agree that "install" is not something that
plain old make is very good at.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Mon Oct 23 02:25:33 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 23 Oct 2006 13:25:33 +1300
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <453B8035.3000907@v.loewis.de>
References: <45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>
	<45316DFC.8020006@canterbury.ac.nz>
	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>
	<45321E83.2030401@v.loewis.de> <4532E241.8070904@canterbury.ac.nz>
	<45330E9C.9030300@v.loewis.de> <45346B00.1000506@canterbury.ac.nz>
	<45351463.8010804@v.loewis.de> <4535769B.4000703@canterbury.ac.nz>
	<4535B86D.6080307@v.loewis.de> <4536E5BF.8050703@canterbury.ac.nz>
	<453B8035.3000907@v.loewis.de>
Message-ID: <453C0BFD.5090309@canterbury.ac.nz>

Martin v. L?wis wrote:

> If that is so, just ignore the warnings.

Not acceptable, because it means I have to examine
all the warnings every time I compile to check that
another one hasn't appeared that represents a real
problem.

> one way to solve the problem is to make the warnings
> go away.

The most straightforward way to solve the problem
is not to ask for these warnings in the first place.
Or at least it would be if I had a decent amount of
control over what warnings get asked for.

> I'm wondering what a make-like system would look
> in this case. Would I write
> 
> $(TARGETDIR)/foo/bar.py: foo/bar.py
>     cp foo/bar.py $(TARGETDIR)/foo/bar.py

Don't take "make-like" too literally. All I mean
is something organised around a dependency graph
rather than a list of things to do one after the
other.

The nodes in the graph needn't be individual files.
The whole package-to-be-build would be a node,
which would depend on the nodes representing the
.py files.

The specification would probably look fairly
similar in some ways to what you currently pass
to the setup function, i.e. a list of .py files that
go into the package. But the internal structure
would be built out of dependency graph nodes
rather than commands and sub-commands.

> If you look at Python's "make install" rule, you
> might agree that "install" is not something that
> plain old make is very good at.

I agree that Make itself doesn't handle install-type
operations very well. I think that's partly because
it's too rigidly fixated on a target being a single
file.

--
Greg

From talin at acm.org  Mon Oct 23 02:50:47 2006
From: talin at acm.org (Talin)
Date: Sun, 22 Oct 2006 17:50:47 -0700
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <453C0BFD.5090309@canterbury.ac.nz>
References: <45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45321E83.2030401@v.loewis.de>
	<4532E241.8070904@canterbury.ac.nz>	<45330E9C.9030300@v.loewis.de>
	<45346B00.1000506@canterbury.ac.nz>	<45351463.8010804@v.loewis.de>
	<4535769B.4000703@canterbury.ac.nz>	<4535B86D.6080307@v.loewis.de>
	<4536E5BF.8050703@canterbury.ac.nz>	<453B8035.3000907@v.loewis.de>
	<453C0BFD.5090309@canterbury.ac.nz>
Message-ID: <453C11E7.4000406@acm.org>

Greg Ewing wrote:
> Martin v. L?wis wrote:
> 
> I agree that Make itself doesn't handle install-type
> operations very well. I think that's partly because
> it's too rigidly fixated on a target being a single
> file.

This isn't related to Py3K, except that it would solve a lot of problems 
in Python if it were true: When is the world going to invent a decent 
make replacement?

I bring this up because I recently (like - today) spent about 8 hours 
trying to debug a Python installation problem - and realizing that much 
of the reason that installation and deployment infrastructure of Python 
programs is so ****ing complex is because of the poverty of tools like Make.

Right now, the top competitors for Make are (as far as I know):

   -- pljam: Nice system, but effectively unmaintained
   -- SCons: again, nice system, but doesn't scale, performance is
             unacceptable for large projects.
   -- ant / nant: Heavy requirement on Java / .Net

There are others out there, nmake, image, etc. None of them have 
succeeding in supplanting 'make' in any serious way.

Most of the potential make alternatives have a fatal flaw in that they 
are not bootstrappable - they require a large body of software to be 
installed before they can even run (like Python, Java, .Net. etc.) Any 
new make-replacement would have to deal with the fact that 'make' is 
already pre-installed on everything except Windows, whereas this system 
would need a way to bootstrap itself onto systems where it was not 
already installed (perhaps using a single C source file as the initial 
bootstrap starting point.) Otherwise, no one will seriously consider 
using it as a basis for deployment of products.

Another fatal flaw of (some) of these systems is that the build script 
files are designed to be 'interpreted' as opposed to 'inspected' - in 
other words, they are useless as metadata. A modern system would need to 
do both - to not only be executable, but to be manipulable/transformable 
in a larger context.

I've often thought about taking a few months and building something that 
would solve all of these problems, but then I stop and I realize that 
I've only personally encountered a small fraction of the problems that 
need to be solved - so all I would accomplish is solving all of my own 
problems, and not everyone else's.

(Sorry for going off-topic, but its a hot issue for me.)

-- Talin

From jan.grant at bristol.ac.uk  Mon Oct 23 10:45:25 2006
From: jan.grant at bristol.ac.uk (Jan Grant)
Date: Mon, 23 Oct 2006 09:45:25 +0100 (BST)
Subject: [Python-3000] Optional type checking for Python
In-Reply-To: <f56dda5c0610220415s7ea5407ap5243826da12b0dd7@mail.gmail.com>
References: <f56dda5c0610210708p3508ac4axca28de31e9b4f07d@mail.gmail.com>
	<f56dda5c0610220350q6a48684dw2826f89cb8552923@mail.gmail.com>
	<f56dda5c0610220415s7ea5407ap5243826da12b0dd7@mail.gmail.com>
Message-ID: <20061023093930.N52313@tribble.ilrt.bris.ac.uk>

On Sun, 22 Oct 2006, Neil Dunn wrote:

> Dear All,
> 
> (I'm forwarding this from the main pydev mailing list)
> 
> I'm a Master's student at Imperial College London currently selecting
> a Master's thesis subject. I am exploring the possibility of "optional
> typing" and "pluggable type systems" (Bracha) for Python. Reading
> around I see that the closest we ever got was PEP 246 (object
> adaption) which has been dropped in favour of generic functions.
> 
> Guido informs me optional static typing probably won't appear in
> Python 3 so I'm safe exploring the ideas for my thesis. I'm planning
> on making the project quite research based and explore implementations
> in CPython or Jython.
> 
> Does anyone have any thoughts about the project? Useful background
> reading, ideas, flames etc?

Only in general support of strong typing with type inferencing :-)

If you've not seen it, "boo" (boo.codehaus.org) is a static typed 
language that riffs on python - although it definitely _isn't_ python.

-- 
jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/
Tel +44 (0)117 3317661   http://ioctl.org/jan/
They modified their trousers secretly.

From solipsis at pitrou.net  Mon Oct 23 11:25:14 2006
From: solipsis at pitrou.net (Antoine)
Date: Mon, 23 Oct 2006 11:25:14 +0200 (CEST)
Subject: [Python-3000] Proposal: No more standard library additions
In-Reply-To: <453C11E7.4000406@acm.org>
References: <45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061013232011.03bef830@sparrow.telecommunity.com>	<45316DFC.8020006@canterbury.ac.nz>	<5.1.1.6.0.20061015022726.026e5368@sparrow.telecommunity.com>	<45321E83.2030401@v.loewis.de>
	<4532E241.8070904@canterbury.ac.nz>	<45330E9C.9030300@v.loewis.de>
	<45346B00.1000506@canterbury.ac.nz>	<45351463.8010804@v.loewis.de>
	<4535769B.4000703@canterbury.ac.nz>	<4535B86D.6080307@v.loewis.de>
	<4536E5BF.8050703@canterbury.ac.nz>	<453B8035.3000907@v.loewis.de>
	<453C0BFD.5090309@canterbury.ac.nz> <453C11E7.4000406@acm.org>
Message-ID: <47942.62.39.9.251.1161595514.squirrel@webmail.nerim.net>

> I bring this up because I recently (like - today) spent about 8 hours
> trying to debug a Python installation problem - and realizing that much
> of the reason that installation and deployment infrastructure of Python
> programs is so ****ing complex is because of the poverty of tools like
> Make.
>
> Right now, the top competitors for Make are (as far as I know):
>
>    -- pljam: Nice system, but effectively unmaintained
>    -- SCons: again, nice system, but doesn't scale, performance is
>              unacceptable for large projects.
>    -- ant / nant: Heavy requirement on Java / .Net

I've heard CMake is quite nice. It has been chosen by KDE as the default
build tool. It generates Makefiles on Unix and MSVC project files under
Windows, which makes it an integrated replacement rather than a complete
ad-hoc solution like ant & friends.

I don't know if it's practical for non C/C++ projects though.

http://www.cmake.org/HTML/Index.html

Oh and I've just found out this one, which promises to be superior to
anything else (of course ;-)):
http://freehackers.org/~tnagy/bksys.html



From cfbolz at gmx.de  Mon Oct 23 14:21:36 2006
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 23 Oct 2006 14:21:36 +0200
Subject: [Python-3000] Practical uses of stackless technology
Message-ID: <ehic4h$nk1$1@sea.gmane.org>

Hi stackless-dev!

As some of you might know, the PyPy project is aiming at including many
of stackless' features into its Python interpreter. Most of them are now
implemented and we're now working on the report we are writing for the
EU about this part of our work.

Being technical hacker types, we can cope with describing the ins and
outs of how our code works, but are less sure on the motivations for
stackless-style technologies as used in real-world applications :)
(especially since this part of PyPy is not used for any applications
yet).

So this is where we come to you, the existing stackless community: why
are you interested in stackless? What benefits does it bring to you and
your business? (the EU always likes to hear that businesses are using
things :)

As well as the more general stackless ideas (tasklets, channels, etc) we
are interested in hearing about any use people are making of the tasklet
pickling features.

It would be wonderful if we could get some nice quotable, attributable
snippets, but general ideas would also be very welcome.

Cheers and thanks,

mwh & Carl Friedrich Bolz


From cfbolz at gmx.de  Mon Oct 23 14:25:14 2006
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 23 Oct 2006 14:25:14 +0200
Subject: [Python-3000] Practical uses of stackless technology
In-Reply-To: <ehic4h$nk1$1@sea.gmane.org>
References: <ehic4h$nk1$1@sea.gmane.org>
Message-ID: <453CB4AA.4030409@gmx.de>

Carl Friedrich Bolz wrote:
> Hi stackless-dev!
> 
> As some of you might know, the PyPy project is aiming at including many
> of stackless' features into its Python interpreter. Most of them are now
> implemented and we're now working on the report we are writing for the
> EU about this part of our work.
> 
> Being technical hacker types, we can cope with describing the ins and
> outs of how our code works, but are less sure on the motivations for
> stackless-style technologies as used in real-world applications :)
> (especially since this part of PyPy is not used for any applications
> yet).
> 
> So this is where we come to you, the existing stackless community: why
> are you interested in stackless? What benefits does it bring to you and
> your business? (the EU always likes to hear that businesses are using
> things :)
> 
> As well as the more general stackless ideas (tasklets, channels, etc) we
> are interested in hearing about any use people are making of the tasklet
> pickling features.
> 
> It would be wonderful if we could get some nice quotable, attributable
> snippets, but general ideas would also be very welcome.
> 
> Cheers and thanks,
> 
> mwh & Carl Friedrich Bolz
> 

argh! completely wrong list of course. Very sorry for the noise.


From talin at acm.org  Mon Oct 23 18:56:20 2006
From: talin at acm.org (Talin)
Date: Mon, 23 Oct 2006 09:56:20 -0700
Subject: [Python-3000] Practical uses of stackless technology
In-Reply-To: <453CB4AA.4030409@gmx.de>
References: <ehic4h$nk1$1@sea.gmane.org> <453CB4AA.4030409@gmx.de>
Message-ID: <453CF434.3050906@acm.org>

Carl Friedrich Bolz wrote:
> Carl Friedrich Bolz wrote:
>> Hi stackless-dev!
>>
>> As some of you might know, the PyPy project is aiming at including many
>> of stackless' features into its Python interpreter. Most of them are now
>> implemented and we're now working on the report we are writing for the
>> EU about this part of our work.
>>
>> Being technical hacker types, we can cope with describing the ins and
>> outs of how our code works, but are less sure on the motivations for
>> stackless-style technologies as used in real-world applications :)
>> (especially since this part of PyPy is not used for any applications
>> yet).
>>
>> So this is where we come to you, the existing stackless community: why
>> are you interested in stackless? What benefits does it bring to you and
>> your business? (the EU always likes to hear that businesses are using
>> things :)
>>
>> As well as the more general stackless ideas (tasklets, channels, etc) we
>> are interested in hearing about any use people are making of the tasklet
>> pickling features.
>>
>> It would be wonderful if we could get some nice quotable, attributable
>> snippets, but general ideas would also be very welcome.
>>
>> Cheers and thanks,
>>
>> mwh & Carl Friedrich Bolz
>>
> 
> argh! completely wrong list of course. Very sorry for the noise.

Maybe the wrong list - but maybe the right discussion. Specifically, how 
  much of the stackless technology should be adopted in Python3K?

I haven't used stackless myself, but I *have* written code that pushes 
generators to their limits and beyond, and wanted something very much 
*like* stackless's capabilities to be integrated into the mainstream.

My particular application is writing backtracking search engines, 
Prolog-style unification, and so on. This requires writing generators 
that communicate across many recursive stack levels.

One of these days I want to re-write my algebraic solver using a 
stackless-type interpreter. Unfortunately I already have 3 versions of 
Python installed on this machine (2.3 cause Mac OS X requires it, 2.4.4 
because my primary apps haven't been ported to 2.5 yet, and 2.5 for 
writing new code) and I really don't want to install yet another.

I understand, from reading the docs on stackless, that it jumps through 
some fairly significant hoops to try and keep the Python C API intact. 
Given the "cruft-removal" nature of Py3K, what could be done to minimize 
the amount of hoop-jumping that stackless has to do?

-- Talin

From rasky at develer.com  Mon Oct 23 19:51:41 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Mon, 23 Oct 2006 19:51:41 +0200
Subject: [Python-3000] Practical uses of stackless technology
References: <ehic4h$nk1$1@sea.gmane.org> <453CB4AA.4030409@gmx.de>
	<453CF434.3050906@acm.org>
Message-ID: <05a201c6f6cb$e58217a0$e303030a@trilan>

Talin wrote:

> Maybe the wrong list - but maybe the right discussion. Specifically,
>   how much of the stackless technology should be adopted in Python3K?

Nowadays, I have moved to greenlet. It is a bare-bone implementation of
continuations, without channels, schedulers, ecc. It is fine for basic
continuation usages, and it is just a standard Python extension. If it were
standardized more, I'm sure it would grow a Python higher-level library
implementing missing constructs on it.
-- 
Giovanni Bajo


From richard.m.tew at gmail.com  Mon Oct 23 20:28:26 2006
From: richard.m.tew at gmail.com (Richard Tew)
Date: Mon, 23 Oct 2006 18:28:26 +0000
Subject: [Python-3000] Practical uses of stackless technology
In-Reply-To: <453CF434.3050906@acm.org>
References: <ehic4h$nk1$1@sea.gmane.org> <453CB4AA.4030409@gmx.de>
	<453CF434.3050906@acm.org>
Message-ID: <952d92df0610231128p7317d9dfs92aa9be7b8a460ea@mail.gmail.com>

On 10/23/06, Talin <talin at acm.org> wrote:
> Carl Friedrich Bolz wrote:
> >> Being technical hacker types, we can cope with describing the ins and
> >> outs of how our code works, but are less sure on the motivations for
> >> stackless-style technologies as used in real-world applications :)
> >> (especially since this part of PyPy is not used for any applications
> >> yet).
>
> Maybe the wrong list - but maybe the right discussion. Specifically, how
>   much of the stackless technology should be adopted in Python3K?

I currently maintain Stackless for what its worth and I also use it in
my day to day work at CCP.

The primary benefits of Stackless IMO are gained through the use of
the minimal but solid set of features which it provides, implemented on
top of its switching.

I am refering to the scheduler, the tasklets (aka microthreads) and
channels which it provides.  These are all tools which I consider a
natural enhancement to Python because of the way they facilitate
writing simple and readable code.

What I am saying is that the biggest gain is to adopt it in its
entirely.

> I understand, from reading the docs on stackless, that it jumps through
> some fairly significant hoops to try and keep the Python C API intact.
> Given the "cruft-removal" nature of Py3K, what could be done to minimize
> the amount of hoop-jumping that stackless has to do?

I think that any docs you may find on the web should be taken
with a grain of salt.  Pretty much all of them are out of date.  And
while there might be some aspects that are still relevant, I know
they only served to confuse me when I was initially trying to
understand Stackless so that I could port it to later releases of
mainline Python.  Stackless has changes a lot since it had
continuations at its core (nowadays the complication they
gave is nowhere to be seen and it has had several rewrites
since then IIRC).

As I understand it, Stackless does not do its hoop-jumping
to keep the C API intact.  Rather the smaller part of its
patches which could be qualified as hoop jumping serve
two main purposes.

 a) Unwinding and reducing how much of the stack is used.
 b) Switching to another tasklet without doing the "hard"
     switching. (otherwise known as "soft" switching).

"Hard" switching is where the stack contents are saved, and
the hand-written assembly code is invoked to change the
stack pointer and save any registers.  Reducing the how much
stack is used also of course reduces how much the "hard"
switching has to save.

The bulk of the Stackless code is in the Stackless
directory, and implements hard switching, the
scheduler, tasklets and channels.  And pickling of
running code of course.

The smaller part of it is the changes to mainline code.

Hope this helps,
Richard.

From ark-mlist at att.net  Tue Oct 24 06:41:48 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Tue, 24 Oct 2006 00:41:48 -0400
Subject: [Python-3000] Practical uses of stackless technology
In-Reply-To: <ehic4h$nk1$1@sea.gmane.org>
Message-ID: <007301c6f726$bdf202d0$6402a8c0@arkdesktop>

> Being technical hacker types, we can cope with describing the ins and
> outs of how our code works, but are less sure on the motivations for
> stackless-style technologies as used in real-world applications :)

Continuations.  Which in turn are useful in multiuser interactive
applications, among other places.



From garys at ihug.com.au  Sun Oct 29 07:30:26 2006
From: garys at ihug.com.au (Gary Stephenson)
Date: Sun, 29 Oct 2006 17:30:26 +1100
Subject: [Python-3000] Alternatives to 'outer'
Message-ID: <00b101c6fb23$ba0bf210$0301010a@VIMES>


I would like to suggest the spelling "free", as I believe the term "free 
variable" has been widely used to describe the concept (albeit
usually in reference to undeclared variables.)

> What this discussion needs now is not further keyword suggestions, but
> a PEP and an implementation.

Is anybody working on such a PEP?

gary

http://www.stephensong.com.au