From tim.peters at gmail.com  Sun Aug  1 01:32:04 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Aug  1 01:32:13 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407281903.i6SJ3ea06175@guido.python.org>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
	<1f7befae04072719401df511c3@mail.gmail.com> 
	<200407281841.i6SIfo205648@guido.python.org>
	<200407281903.i6SJ3ea06175@guido.python.org>
Message-ID: <1f7befae0407311632442e488b@mail.gmail.com>

[Guido, patches PyImport_ExecCodeModuleEx to remove a non-pre-existing
 module from sys.modules if executing the module code raises an exception]

It fixes at least one spurious -r test failure on Windows.  Before:

$ regrtest.py -uall test_sundry test___all__
test_sundry
test___all__
test test___all__ failed -- Traceback (most recent call last):
  File "C:\Code\python\lib\test\test___all__.py", line 150, in test_all
    self.check_all("rlcompleter")
  File "C:\Code\python\lib\test\test___all__.py", line 40, in check_all
    "%s has no __all__ attribute" % modname)
  File "C:\Code\python\lib\test\test_support.py", line 208, in verify
    raise TestFailed(reason)
TestFailed: rlcompleter has no __all__ attribute

After:

$ regrtest.py -uall test_sundry test___all__
test_sundry
test___all__
All 2 tests OK.

It does not fix at least one other one, because a second attempt to
import curses still "works" on Windows:

>>> import curses
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Code\python\lib\curses\__init__.py", line 15, in ?
    from _curses import *
ImportError: No module named _curses
>>> import curses
>>> curses
<module 'curses' from 'C:\Code\python\lib\curses\__init__.pyc'>
>>>

So whatever this patch does <wink>, it breaks down somehow for
non-importable packages.  I guess that any path thru import.c calling
PyImport_AddModule() before calling PyImport_ExecCodeModuleEx() will
suffer similarly (because the former convinces the latter then that
the module "is pre-existing", even if it really wasn't, and the latter
won't remove it from sys.modules then despite that running its code
fails).

It breaks this Zope3 doctest twice, for an obvious reason:

def test_bad_import():
    """

    >>> c = config.ConfigurationContext()

    >>> c.resolve('zope.configuration.tests.victim.x')
    Traceback (most recent call last):
    ...
    ConfigurationError: Couldn't import zope.configuration.tests.victim,""" \
                                       """ No module named bad_to_the_bone

    Cleanup:

    >>> del sys.modules['zope.configuration.tests.victim']
    >>> del sys.modules['zope.configuration.tests.bad']

    """

That is, this test's cleanup currently requires that insane modules
get left behind.

It also breaks this Zope3 test, which uses a custom import hook:

ERROR: test_reporting_exception_during_import
(zope.importtool.tests.test_hook.HookTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\python\lib\unittest.py", line 260, in run
    testMethod()
  File "C:\Code\ZopeX3\src\zope\importtool\tests\test_hook.py", line
198, in test_reporting_exception_during_import
    import error
  File "C:\Code\ZopeX3\src\zope\importtool\hook.py", line 72, in importhook
    v = self.previous(name, globals, locals, fromlist)
KeyError: 'zope.importtool.tests.error'

self.previous there appears to be bound to __builtin__.__import__.   I
don't understand this code, but expect it's shallow to someone who
does.

Overall, I'm +1, but think it needs more work to plug the "curses on
Windows" kind of hole.
From tismer at stackless.com  Sun Aug  1 02:18:48 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sun Aug  1 02:16:25 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200407291820.i6TIKRi10773@guido.python.org>
References: <200407290353.i6T3rtL8019361@cosc353.cosc.canterbury.ac.nz>
	<200407291820.i6TIKRi10773@guido.python.org>
Message-ID: <410C36E8.7060102@stackless.com>

Guido van Rossum wrote:

>>It's going to be difficult to ensure that nothing ever gets a
>>reference to a broken module, because of circular imports.  Suppose A
>>imports B, which imports A. If A's initialisation subsequently fails,
>>then even if A is removed from sys.modules, B still contains a
>>reference to the broken A.

Well, circular imports first appeared to me like the reverse
problem of circular references. The latter was only solvable
by a garbage collector. Well, it's simpler after all.

Circular imports, to be made consistent, would need
something like a final "commit" after all imports succeeded.
I think of a simple, generator-like "undoable assignment"
to sys.modules, like ICON's "<-" operator. When a top-level
import starts, the module gets into sys.modules, but this
is also recorded as not finished. Further imports can happen,
until this is all done. Then, the pending undos are cancelled,
this is the "commit".
If any uncaught exception occours, the latent undo is executed
for all imports started in this context, and we end up with
an exception and (hopefully) almost no side effects.
Maybe this could be user-extensible, as on-failure-undo actions.

> Yeah, but I don't mind; my patch is still a big improvement in other
> cases and doesn't really make things any worse in the above case (B
> has a reference to a broken A, just like without my patch).

It is definately an improvement, much more at what people
would expect, anyway.

> Did anybody look at my patch?  Shall I check it in?

Yes I did.

+1

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tim.peters at gmail.com  Sun Aug  1 07:50:57 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Aug  1 07:51:01 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae0407311632442e488b@mail.gmail.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
	<1f7befae04072719401df511c3@mail.gmail.com> 
	<200407281841.i6SIfo205648@guido.python.org>
	<200407281903.i6SJ3ea06175@guido.python.org>
	<1f7befae0407311632442e488b@mail.gmail.com>
Message-ID: <1f7befae04073122501f7e185f@mail.gmail.com>

Attached patch builds on Guido's:

- refactored
- load_package() now also removes stuff from sys.modules if an error
  occurs and the stuff wasn't in sys.modules to begin with
- patches test_pkgimport so it passes again (it was relying on
  junk remaining in sys.modules in one spot)
- regrtest -uall passes on Windows with this patch (247 tests OK,
  37 tests skipped).
- oops -- fixes obsolute stuff in a docstring too (the LaTeX docs had
  already been corrected)
- does not remove insane workarounds in .py modules (contortions trying
  to worm around that non-working modules were left behind by failing
  imports)

"curses on Windows" no longer screws up.  Before:

$ regrtest.py -uall test_sundry test___all__ test_curses
test_sundry
test___all__
test test___all__ failed -- Traceback (most recent call last):
  File "C:\Code\python\lib\test\test___all__.py", line 150, in test_all
    self.check_all("rlcompleter")
  File "C:\Code\python\lib\test\test___all__.py", line 40, in check_all
    "%s has no __all__ attribute" % modname)
  File "C:\Code\python\lib\test\test_support.py", line 208, in verify
    raise TestFailed(reason)
TestFailed: rlcompleter has no __all__ attribute

test_curses
test test_curses crashed -- exceptions.AttributeError: 'module' object
has no attribute 'endwin'
1 test OK.
2 tests failed:
    test___all__ test_curses

After:

$ regrtest.py -uall test_sundry test___all__ test_curses
test_sundry
test___all__
test_curses
test_curses skipped -- No module named _curses
2 tests OK.
1 test skipped:
    test_curses
Those skips are all expected on win32.
-------------- next part --------------
Index: Lib/test/test_pkgimport.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_pkgimport.py,v
retrieving revision 1.7
diff -u -r1.7 test_pkgimport.py
--- Lib/test/test_pkgimport.py	9 Aug 2002 16:37:36 -0000	1.7
+++ Lib/test/test_pkgimport.py	1 Aug 2004 05:30:10 -0000
@@ -66,12 +66,11 @@
         try: __import__(self.module_name)
         except NameError: pass
         else: raise RuntimeError, 'Failed to induce NameError.'
-        module = __import__(self.module_name).foo
 
         # ...now  change  the module  so  that  the NameError  doesn't
         # happen
         self.rewrite_file('%s = 1' % var)
-        reload(module)
+        module = __import__(self.module_name).foo
         self.assertEqual(getattr(module, var), 1)
 
 
Index: Python/import.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/import.c,v
retrieving revision 2.233
diff -u -r2.233 import.c
--- Python/import.c	27 Jun 2004 16:51:46 -0000	2.233
+++ Python/import.c	1 Aug 2004 05:30:11 -0000
@@ -557,6 +557,33 @@
 	return m;
 }
 
+/* The same as PyImport_AddModule, except also sets created to 1 if a
+ * new module object is created, or to 0 if an existing module object
+ * is found.
+ */
+static PyObject *
+_AddModuleEx(const char *name, int *created)
+{
+	PyObject *modules = PyImport_GetModuleDict();
+	PyObject *m;
+
+	*created = 0;
+	if ((m = PyDict_GetItemString(modules, name)) != NULL &&
+	    PyModule_Check(m))
+		return m;
+	m = PyImport_AddModule((char *)name);
+	if (m != NULL)
+		*created = 1;
+	return m;
+}
+
+/* Remove name from sys.modules.  It must already be in sys.modules. */
+static void
+_RemoveModule(const char *name)
+{
+	PyObject *modules = PyImport_GetModuleDict();
+	PyDict_DelItemString(modules, name); /* if it fails, too bad */
+}
 
 /* Execute a code object in a module and return the module object
    WITH INCREMENTED REFERENCE COUNT */
@@ -572,8 +599,9 @@
 {
 	PyObject *modules = PyImport_GetModuleDict();
 	PyObject *m, *d, *v;
+	int created;
 
-	m = PyImport_AddModule(name);
+	m = _AddModuleEx(name, &created);
 	if (m == NULL)
 		return NULL;
 	/* If the module is being reloaded, we get the old module back
@@ -582,7 +610,7 @@
 	if (PyDict_GetItemString(d, "__builtins__") == NULL) {
 		if (PyDict_SetItemString(d, "__builtins__",
 					 PyEval_GetBuiltins()) != 0)
-			return NULL;
+			goto error;
 	}
 	/* Remember the filename as the __file__ attribute */
 	v = NULL;
@@ -601,7 +629,7 @@
 
 	v = PyEval_EvalCode((PyCodeObject *)co, d, d);
 	if (v == NULL)
-		return NULL;
+		goto error;
 	Py_DECREF(v);
 
 	if ((m = PyDict_GetItemString(modules, name)) == NULL) {
@@ -614,6 +642,11 @@
 	Py_INCREF(m);
 
 	return m;
+
+  error:
+	if (created)
+		_RemoveModule(name);
+	return NULL;
 }
 
 
@@ -888,13 +921,16 @@
 static PyObject *
 load_package(char *name, char *pathname)
 {
-	PyObject *m, *d, *file, *path;
+	PyObject *m, *d;
+	PyObject *file = NULL;
+	PyObject *path = NULL;
 	int err;
 	char buf[MAXPATHLEN+1];
 	FILE *fp = NULL;
 	struct filedescr *fdp;
+	int created;
 
-	m = PyImport_AddModule(name);
+	m = _AddModuleEx(name, &created);
 	if (m == NULL)
 		return NULL;
 	if (Py_VerboseFlag)
@@ -903,19 +939,15 @@
 	d = PyModule_GetDict(m);
 	file = PyString_FromString(pathname);
 	if (file == NULL)
-		return NULL;
+		goto error;
 	path = Py_BuildValue("[O]", file);
-	if (path == NULL) {
-		Py_DECREF(file);
-		return NULL;
-	}
+	if (path == NULL)
+		goto error;
 	err = PyDict_SetItemString(d, "__file__", file);
 	if (err == 0)
 		err = PyDict_SetItemString(d, "__path__", path);
-	if (err != 0) {
-		m = NULL;
-		goto cleanup;
-	}
+	if (err != 0)
+		goto error;
 	buf[0] = '\0';
 	fdp = find_module(name, "__init__", path, buf, sizeof(buf), &fp, NULL);
 	if (fdp == NULL) {
@@ -930,9 +962,15 @@
 	m = load_module(name, fp, buf, fdp->type, NULL);
 	if (fp != NULL)
 		fclose(fp);
+	goto cleanup;
+
+  error:
+  	m = NULL;
   cleanup:
 	Py_XDECREF(path);
 	Py_XDECREF(file);
+	if (created && m == NULL)
+		_RemoveModule(name);
 	return m;
 }
 
@@ -1272,7 +1310,7 @@
 			if (Py_VerboseFlag > 1)
 				PySys_WriteStderr("# trying %s\n", buf);
 			filemode = fdp->mode;
-			if (filemode[0] == 'U') 
+			if (filemode[0] == 'U')
 				filemode = "r" PY_STDIOTEXTMODE;
 			fp = fopen(buf, filemode);
 			if (fp != NULL) {
@@ -2520,7 +2558,7 @@
 {
 	FILE *fp;
 	if (fob == NULL) {
-		if (mode[0] == 'U') 
+		if (mode[0] == 'U')
 			mode = "r" PY_STDIOTEXTMODE;
 		fp = fopen(pathname, mode);
 		if (fp == NULL)
@@ -2692,9 +2730,9 @@
 The module name must include the full package name, if any.");
 
 PyDoc_STRVAR(doc_lock_held,
-"lock_held() -> 0 or 1\n\
-Return 1 if the import lock is currently held.\n\
-On platforms without threads, return 0.");
+"lock_held() -> boolean\n\
+Return True if the import lock is currently held, else False.\n\
+On platforms without threads, return False.");
 
 PyDoc_STRVAR(doc_acquire_lock,
 "acquire_lock() -> None\n\
From martin at v.loewis.de  Sun Aug  1 12:47:59 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug  1 12:47:59 2004
Subject: [Python-Dev] Changes to the installer builder
Message-ID: <410CCA5F.3070900@v.loewis.de>

I have added some documentation to msi.py, please let me know if
something is still missing.

I have also rearranged the versioning. msi.py now supports two
"product lines": snapshots, and releases. Snapshots can be generated
by anybody, whereas releases are only published by python.org.
Each line has their own ProductVersion scheme: snapshots use
major.minor.currentday; official releases use the version from the
Python DLL. The relevant information is now extracted from
patchlevel.h.

Accordingly, there is no need to specify current_version anymore
in config.py, and the "alpha" setting has been renamed to "snapshot".

Regards,
Martin
From tim.peters at gmail.com  Sun Aug  1 18:41:32 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Aug  1 18:41:46 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae0407311632442e488b@mail.gmail.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
	<1f7befae04072719401df511c3@mail.gmail.com> 
	<200407281841.i6SIfo205648@guido.python.org>
	<200407281903.i6SJ3ea06175@guido.python.org>
	<1f7befae0407311632442e488b@mail.gmail.com>
Message-ID: <1f7befae04080109413fe3a967@mail.gmail.com>

[Tim]
> ...
> I guess that any path thru import.c calling PyImport_AddModule() before
> calling PyImport_ExecCodeModuleEx() will suffer similarly (because the
> former convinces the latter then that the module "is pre-existing", even if it
> really wasn't, and the latter won't remove it from sys.modules then despite
> that running its code fails).

And I posted a patch later that "fixed this" in one case involving
packages.  I don't know much about import details, but I believe there
are other cases -- just search for PyImport_AddModule.  Some look
harmless, but some don't:

- PyImport_ImportFrozenModule.  In case of a package, creates a module
  object first to set its __path__, before running the code.

- zipimporter_load_module (in zipimport.c).  Similarly, but also sets
  __loader__ before running the code, package or not.

- Py_InitModule4 (in modsupport.c) leaves a damaged module behind if
  initialization fails.

"A problem" is that PyImport_ExecCodeModuleEx seems to be the only
real choke point, and is making a distinction between "old" and "new"
modules that doesn't always match its callers' realities.  This
distinction came from here:

[Guido]
> What should it do if the module already existed (e.g. when used by reload())?
> Strawman answer: leave it there -- the reload() semantics and common use
> cases are best served by that.

I understand the point there for reload(), but don't know of other use
cases (let alone common ones <wink>) for wanting to leave a module in
sys.modules when its code fails.  What are they?  Maybe it would be
better for PyImport_ExecCodeModuleEx to purge a broken module
unconditionally, and change PyImport_ReloadModule to put it back?
From guido at python.org  Sun Aug  1 22:13:18 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Aug  1 22:13:26 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Sun, 01 Aug 2004 02:18:48 +0200."
	<410C36E8.7060102@stackless.com> 
References: <200407290353.i6T3rtL8019361@cosc353.cosc.canterbury.ac.nz>
	<200407291820.i6TIKRi10773@guido.python.org> 
	<410C36E8.7060102@stackless.com> 
Message-ID: <200408012013.i71KDI726275@guido.python.org>

> Circular imports, to be made consistent, would need
> something like a final "commit" after all imports succeeded.

Silly idea.  Typically, circular imports aren't planned, they just
happen, and (usually) just happen to work.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tismer at stackless.com  Sun Aug  1 23:03:32 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sun Aug  1 23:01:06 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200408012013.i71KDI726275@guido.python.org>
References: <200407290353.i6T3rtL8019361@cosc353.cosc.canterbury.ac.nz>
	<200407291820.i6TIKRi10773@guido.python.org>
	<410C36E8.7060102@stackless.com>
	<200408012013.i71KDI726275@guido.python.org>
Message-ID: <410D5AA4.2060901@stackless.com>

Guido van Rossum wrote:

>>Circular imports, to be made consistent, would need
>>something like a final "commit" after all imports succeeded.
> 
> 
> Silly idea.  Typically, circular imports aren't planned, they just
> happen, and (usually) just happen to work.

Yes, you name it, if you are talking about allowing circular
imports without precautions and not expecting problems.

Like circular references which aren't planned, they just happen,
but they truely work fine after GC was introduced.
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      want to see a silly guy's work?   http://www.stackless.com/

From tim.peters at gmail.com  Mon Aug  2 01:11:55 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  2 01:11:58 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae04080109413fe3a967@mail.gmail.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
	<1f7befae04072719401df511c3@mail.gmail.com> 
	<200407281841.i6SIfo205648@guido.python.org>
	<200407281903.i6SJ3ea06175@guido.python.org>
	<1f7befae0407311632442e488b@mail.gmail.com>
	<1f7befae04080109413fe3a967@mail.gmail.com>
Message-ID: <1f7befae04080116116175c3a1@mail.gmail.com>

[Tim]
> ...
> I understand the point there for reload(), but don't know of other use
> cases (let alone common ones <wink>) for wanting to leave a module in
> sys.modules when its code fails.  What are they?  Maybe it would be
> better for PyImport_ExecCodeModuleEx to purge a broken module
> unconditionally, and change PyImport_ReloadModule to put it back?

Attached is a patch that takes that approach.  Code feels
simpler/cleaner, since it makes a special case out of reload() instead
of a special case out of everything except reload().  Still haven't
thought of a case other than reload() that wants special treatment. 
The whole test suite still passes; it still fixes all the spurious
test failures I know about on Windows when running the suite with -r.

The internal _RemoveModule() is incidentally more careful in this
version of the patch.  The last patch's new _AddModuleEx() no longer
exists.
-------------- next part --------------
Index: fsrefs.py
===================================================================
RCS file: /cvs-repository/ZODB3/Tools/Attic/fsrefs.py,v
retrieving revision 1.7.4.3
diff -c -u -r1.7.4.3 fsrefs.py
--- fsrefs.py	10 Jul 2004 02:42:11 -0000	1.7.4.3
+++ fsrefs.py	11 Jul 2004 01:45:23 -0000
@@ -20,12 +20,13 @@
 
 fsrefs.py checks object sanity by trying to load the current revision of
 every object O in the database, and also verifies that every object
-directly reachable from each such O exists in the database.  Note that
-application code implementing objects (or at least defining their classes)
-must be available, an d on PYTHONPATH, for fsrefs to work, because objects
-are actually loaded. On a ZEO server, all the application code typically
-isn't present, so it may be difficult to run fsrefs usefully on a ZEO
-server.
+directly reachable from each such O exists in the database.
+
+It's hard to explain exactly what it does because it relies on undocumented
+features in Python's cPickle module:  many of the crucial steps of loading
+an object are taken, but application objects aren't actually created.  This
+saves a lot of time, and allows fsrefs to be run even if the code
+implementing the object classes isn't available.
 
 A read-only connection to the specified FileStorage is made, but it is not
 recommended to run fsrefs against a live FileStorage.  Because a live
@@ -36,10 +37,7 @@
 fsrefs doesn't normally produce any output.  If an object fails to load, the
 oid of the object is given in a message saying so, and if -v was specified
 then the traceback corresponding to the load failure is also displayed
-(this is the only effect of the -v flag, and is usually very helpful; -v is
-recommended for normal use).  Note that, as mentioned above, a common
-problem is to get a "failed to load" message simply because the module
-containing the class of the object isn't on PYTHONPATH.
+(this is the only effect of the -v flag).
 
 Two other kinds of errors are also detected, one strongly related to
 "failed to load", when an object O loads OK, and directly refers to a
@@ -56,6 +54,10 @@
    seen by fsrefs before P, an "O refers to the unloadable P" message will
    not be produced; a message saying that P can't be loaded will be
    produced when fsrefs later tries to load P, though.
+
+fsrefs also (indirectly) checks that the .index file is sane, because
+fsrefs uses the index to get its idea of what constitutes "all the objects
+in the database".
 
 Note these limitations:  because fsrefs only looks at the current revision
 of objects, it does not attempt to load objects in versions, or non-current
From tim.peters at gmail.com  Mon Aug  2 01:14:39 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  2 01:14:42 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae04080116116175c3a1@mail.gmail.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
	<1f7befae04072719401df511c3@mail.gmail.com> 
	<200407281841.i6SIfo205648@guido.python.org>
	<200407281903.i6SJ3ea06175@guido.python.org>
	<1f7befae0407311632442e488b@mail.gmail.com>
	<1f7befae04080109413fe3a967@mail.gmail.com>
	<1f7befae04080116116175c3a1@mail.gmail.com>
Message-ID: <1f7befae0408011614190382dc@mail.gmail.com>

Oops, attached a wrong patch.  Correct patch attached.
-------------- next part --------------
Index: Lib/test/test_pkgimport.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_pkgimport.py,v
retrieving revision 1.7
diff -u -r1.7 test_pkgimport.py
--- Lib/test/test_pkgimport.py	9 Aug 2002 16:37:36 -0000	1.7
+++ Lib/test/test_pkgimport.py	1 Aug 2004 23:01:43 -0000
@@ -66,12 +66,11 @@
         try: __import__(self.module_name)
         except NameError: pass
         else: raise RuntimeError, 'Failed to induce NameError.'
-        module = __import__(self.module_name).foo
 
         # ...now  change  the module  so  that  the NameError  doesn't
         # happen
         self.rewrite_file('%s = 1' % var)
-        reload(module)
+        module = __import__(self.module_name).foo
         self.assertEqual(getattr(module, var), 1)
 
 
Index: Python/import.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/import.c,v
retrieving revision 2.233
diff -u -r2.233 import.c
--- Python/import.c	27 Jun 2004 16:51:46 -0000	2.233
+++ Python/import.c	1 Aug 2004 23:01:44 -0000
@@ -557,10 +557,25 @@
 	return m;
 }
 
+/* Remove name from sys.modules, if it's there. */
+static void
+_RemoveModule(const char *name)
+{
+	PyObject *modules = PyImport_GetModuleDict();
+	if (PyDict_GetItemString(modules, name) == NULL)
+		return;
+	if (PyDict_DelItemString(modules, name) < 0)
+		Py_FatalError("import:  deleting existing key in"
+			      "sys.modules failed");
+}
 
 /* Execute a code object in a module and return the module object
-   WITH INCREMENTED REFERENCE COUNT */
-
+ * WITH INCREMENTED REFERENCE COUNT.  If an error occurs, name is
+ * removed from sys.modules, to avoid leaving damaged module objects
+ * in sys.modules.  The caller may wish to restore the original
+ * module object (if any) in this case; PyImport_ReloadModule is an
+ * example.
+ */
 PyObject *
 PyImport_ExecCodeModule(char *name, PyObject *co)
 {
@@ -582,7 +597,7 @@
 	if (PyDict_GetItemString(d, "__builtins__") == NULL) {
 		if (PyDict_SetItemString(d, "__builtins__",
 					 PyEval_GetBuiltins()) != 0)
-			return NULL;
+			goto error;
 	}
 	/* Remember the filename as the __file__ attribute */
 	v = NULL;
@@ -601,7 +616,7 @@
 
 	v = PyEval_EvalCode((PyCodeObject *)co, d, d);
 	if (v == NULL)
-		return NULL;
+		goto error;
 	Py_DECREF(v);
 
 	if ((m = PyDict_GetItemString(modules, name)) == NULL) {
@@ -614,6 +629,10 @@
 	Py_INCREF(m);
 
 	return m;
+
+  error:
+	_RemoveModule(name);
+	return NULL;
 }
 
 
@@ -888,7 +907,9 @@
 static PyObject *
 load_package(char *name, char *pathname)
 {
-	PyObject *m, *d, *file, *path;
+	PyObject *m, *d;
+	PyObject *file = NULL;
+	PyObject *path = NULL;
 	int err;
 	char buf[MAXPATHLEN+1];
 	FILE *fp = NULL;
@@ -903,19 +924,15 @@
 	d = PyModule_GetDict(m);
 	file = PyString_FromString(pathname);
 	if (file == NULL)
-		return NULL;
+		goto error;
 	path = Py_BuildValue("[O]", file);
-	if (path == NULL) {
-		Py_DECREF(file);
-		return NULL;
-	}
+	if (path == NULL)
+		goto error;
 	err = PyDict_SetItemString(d, "__file__", file);
 	if (err == 0)
 		err = PyDict_SetItemString(d, "__path__", path);
-	if (err != 0) {
-		m = NULL;
-		goto cleanup;
-	}
+	if (err != 0)
+		goto error;
 	buf[0] = '\0';
 	fdp = find_module(name, "__init__", path, buf, sizeof(buf), &fp, NULL);
 	if (fdp == NULL) {
@@ -930,6 +947,10 @@
 	m = load_module(name, fp, buf, fdp->type, NULL);
 	if (fp != NULL)
 		fclose(fp);
+	goto cleanup;
+
+  error:
+  	m = NULL;
   cleanup:
 	Py_XDECREF(path);
 	Py_XDECREF(file);
@@ -1272,7 +1293,7 @@
 			if (Py_VerboseFlag > 1)
 				PySys_WriteStderr("# trying %s\n", buf);
 			filemode = fdp->mode;
-			if (filemode[0] == 'U') 
+			if (filemode[0] == 'U')
 				filemode = "r" PY_STDIOTEXTMODE;
 			fp = fopen(buf, filemode);
 			if (fp != NULL) {
@@ -2234,6 +2255,7 @@
 	char buf[MAXPATHLEN+1];
 	struct filedescr *fdp;
 	FILE *fp = NULL;
+	PyObject *newm;
 
 	if (m == NULL || !PyModule_Check(m)) {
 		PyErr_SetString(PyExc_TypeError,
@@ -2275,10 +2297,18 @@
 	Py_XDECREF(path);
 	if (fdp == NULL)
 		return NULL;
-	m = load_module(name, fp, buf, fdp->type, NULL);
+	newm = load_module(name, fp, buf, fdp->type, NULL);
 	if (fp)
 		fclose(fp);
-	return m;
+	if (newm == NULL) {
+		/* load_module probably removed name from modules because of
+		 * the error.  Put back the original module object.  We're
+		 * going to return NULL in this case regardless of whether
+		 * replacing name succeeds, so the return value is ignored.
+		 */
+		PyDict_SetItemString(modules, name, m);
+	}
+	return newm;
 }
 
 
@@ -2520,7 +2550,7 @@
 {
 	FILE *fp;
 	if (fob == NULL) {
-		if (mode[0] == 'U') 
+		if (mode[0] == 'U')
 			mode = "r" PY_STDIOTEXTMODE;
 		fp = fopen(pathname, mode);
 		if (fp == NULL)
@@ -2692,9 +2722,9 @@
 The module name must include the full package name, if any.");
 
 PyDoc_STRVAR(doc_lock_held,
-"lock_held() -> 0 or 1\n\
-Return 1 if the import lock is currently held.\n\
-On platforms without threads, return 0.");
+"lock_held() -> boolean\n\
+Return True if the import lock is currently held, else False.\n\
+On platforms without threads, return False.");
 
 PyDoc_STRVAR(doc_acquire_lock,
 "acquire_lock() -> None\n\
From tim.peters at gmail.com  Mon Aug  2 01:37:30 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  2 01:37:36 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae0408011614190382dc@mail.gmail.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
	<1f7befae04072719401df511c3@mail.gmail.com> 
	<200407281841.i6SIfo205648@guido.python.org>
	<200407281903.i6SJ3ea06175@guido.python.org>
	<1f7befae0407311632442e488b@mail.gmail.com>
	<1f7befae04080109413fe3a967@mail.gmail.com>
	<1f7befae04080116116175c3a1@mail.gmail.com>
	<1f7befae0408011614190382dc@mail.gmail.com>
Message-ID: <1f7befae0408011637458d824c@mail.gmail.com>

And one more time.

- regen'ed against current CVS import.c
- removes some now-needless workarounds in .py files
-------------- next part --------------
Index: Lib/pty.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/pty.py,v
retrieving revision 1.16
diff -u -r1.16 pty.py
--- Lib/pty.py	5 Jun 2004 16:27:16 -0000	1.16
+++ Lib/pty.py	1 Aug 2004 23:36:00 -0000
@@ -8,17 +8,6 @@
 
 from select import select
 import os
-
-# Absurd:  import termios and then delete it.  This is to force an attempt
-# to import pty to raise an ImportError on platforms that lack termios.
-# Without this explicit import of termios here, some other module may
-# import tty first, which in turn imports termios and dies with an
-# ImportError then.  But since tty *does* exist across platforms, that
-# leaves a damaged module object for tty in sys.modules, and the import
-# of tty here then appears to work despite that the tty imported is junk.
-import termios
-del termios
-
 import tty
 
 __all__ = ["openpty","fork","spawn"]
Index: Lib/test/test___all__.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test___all__.py,v
retrieving revision 1.39
diff -u -r1.39 test___all__.py
--- Lib/test/test___all__.py	1 Jul 2004 11:15:39 -0000	1.39
+++ Lib/test/test___all__.py	1 Aug 2004 23:36:00 -0000
@@ -21,20 +21,6 @@
         except ImportError:
             # Silent fail here seems the best route since some modules
             # may not be available in all environments.
-            # Since an ImportError may leave a partial module object in
-            # sys.modules, get rid of that first.  Here's what happens if
-            # you don't:  importing pty fails on Windows because pty tries to
-            # import FCNTL, which doesn't exist.  That raises an ImportError,
-            # caught here.  It also leaves a partial pty module in sys.modules.
-            # So when test_pty is called later, the import of pty succeeds,
-            # but shouldn't.  As a result, test_pty crashes with an
-            # AttributeError instead of an ImportError, and regrtest interprets
-            # the latter as a test failure (ImportError is treated as "test
-            # skipped" -- which is what test_pty should say on Windows).
-            try:
-                del sys.modules[modname]
-            except KeyError:
-                pass
             return
         verify(hasattr(sys.modules[modname], "__all__"),
                "%s has no __all__ attribute" % modname)
Index: Lib/test/test_pkgimport.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_pkgimport.py,v
retrieving revision 1.7
diff -u -r1.7 test_pkgimport.py
--- Lib/test/test_pkgimport.py	9 Aug 2002 16:37:36 -0000	1.7
+++ Lib/test/test_pkgimport.py	1 Aug 2004 23:36:00 -0000
@@ -66,12 +66,11 @@
         try: __import__(self.module_name)
         except NameError: pass
         else: raise RuntimeError, 'Failed to induce NameError.'
-        module = __import__(self.module_name).foo
 
         # ...now  change  the module  so  that  the NameError  doesn't
         # happen
         self.rewrite_file('%s = 1' % var)
-        reload(module)
+        module = __import__(self.module_name).foo
         self.assertEqual(getattr(module, var), 1)
 
 
Index: Python/import.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/import.c,v
retrieving revision 2.235
diff -u -r2.235 import.c
--- Python/import.c	1 Aug 2004 23:26:05 -0000	2.235
+++ Python/import.c	1 Aug 2004 23:36:01 -0000
@@ -557,10 +557,25 @@
 	return m;
 }
 
+/* Remove name from sys.modules, if it's there. */
+static void
+_RemoveModule(const char *name)
+{
+	PyObject *modules = PyImport_GetModuleDict();
+	if (PyDict_GetItemString(modules, name) == NULL)
+		return;
+	if (PyDict_DelItemString(modules, name) < 0)
+		Py_FatalError("import:  deleting existing key in"
+			      "sys.modules failed");
+}
 
 /* Execute a code object in a module and return the module object
-   WITH INCREMENTED REFERENCE COUNT */
-
+ * WITH INCREMENTED REFERENCE COUNT.  If an error occurs, name is
+ * removed from sys.modules, to avoid leaving damaged module objects
+ * in sys.modules.  The caller may wish to restore the original
+ * module object (if any) in this case; PyImport_ReloadModule is an
+ * example.
+ */
 PyObject *
 PyImport_ExecCodeModule(char *name, PyObject *co)
 {
@@ -582,7 +597,7 @@
 	if (PyDict_GetItemString(d, "__builtins__") == NULL) {
 		if (PyDict_SetItemString(d, "__builtins__",
 					 PyEval_GetBuiltins()) != 0)
-			return NULL;
+			goto error;
 	}
 	/* Remember the filename as the __file__ attribute */
 	v = NULL;
@@ -601,7 +616,7 @@
 
 	v = PyEval_EvalCode((PyCodeObject *)co, d, d);
 	if (v == NULL)
-		return NULL;
+		goto error;
 	Py_DECREF(v);
 
 	if ((m = PyDict_GetItemString(modules, name)) == NULL) {
@@ -614,6 +629,10 @@
 	Py_INCREF(m);
 
 	return m;
+
+  error:
+	_RemoveModule(name);
+	return NULL;
 }
 
 
@@ -888,7 +907,9 @@
 static PyObject *
 load_package(char *name, char *pathname)
 {
-	PyObject *m, *d, *file, *path;
+	PyObject *m, *d;
+	PyObject *file = NULL;
+	PyObject *path = NULL;
 	int err;
 	char buf[MAXPATHLEN+1];
 	FILE *fp = NULL;
@@ -903,19 +924,15 @@
 	d = PyModule_GetDict(m);
 	file = PyString_FromString(pathname);
 	if (file == NULL)
-		return NULL;
+		goto error;
 	path = Py_BuildValue("[O]", file);
-	if (path == NULL) {
-		Py_DECREF(file);
-		return NULL;
-	}
+	if (path == NULL)
+		goto error;
 	err = PyDict_SetItemString(d, "__file__", file);
 	if (err == 0)
 		err = PyDict_SetItemString(d, "__path__", path);
-	if (err != 0) {
-		m = NULL;
-		goto cleanup;
-	}
+	if (err != 0)
+		goto error;
 	buf[0] = '\0';
 	fdp = find_module(name, "__init__", path, buf, sizeof(buf), &fp, NULL);
 	if (fdp == NULL) {
@@ -930,6 +947,10 @@
 	m = load_module(name, fp, buf, fdp->type, NULL);
 	if (fp != NULL)
 		fclose(fp);
+	goto cleanup;
+
+  error:
+  	m = NULL;
   cleanup:
 	Py_XDECREF(path);
 	Py_XDECREF(file);
@@ -2234,6 +2255,7 @@
 	char buf[MAXPATHLEN+1];
 	struct filedescr *fdp;
 	FILE *fp = NULL;
+	PyObject *newm;
 
 	if (m == NULL || !PyModule_Check(m)) {
 		PyErr_SetString(PyExc_TypeError,
@@ -2275,10 +2297,18 @@
 	Py_XDECREF(path);
 	if (fdp == NULL)
 		return NULL;
-	m = load_module(name, fp, buf, fdp->type, NULL);
+	newm = load_module(name, fp, buf, fdp->type, NULL);
 	if (fp)
 		fclose(fp);
-	return m;
+	if (newm == NULL) {
+		/* load_module probably removed name from modules because of
+		 * the error.  Put back the original module object.  We're
+		 * going to return NULL in this case regardless of whether
+		 * replacing name succeeds, so the return value is ignored.
+		 */
+		PyDict_SetItemString(modules, name, m);
+	}
+	return newm;
 }
 
 
From guido at python.org  Mon Aug  2 01:52:18 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  2 01:52:24 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Sun, 01 Aug 2004 19:37:30 EDT."
	<1f7befae0408011637458d824c@mail.gmail.com> 
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
	<1f7befae04072719401df511c3@mail.gmail.com>
	<200407281841.i6SIfo205648@guido.python.org>
	<200407281903.i6SJ3ea06175@guido.python.org>
	<1f7befae0407311632442e488b@mail.gmail.com>
	<1f7befae04080109413fe3a967@mail.gmail.com>
	<1f7befae04080116116175c3a1@mail.gmail.com>
	<1f7befae0408011614190382dc@mail.gmail.com> 
	<1f7befae0408011637458d824c@mail.gmail.com> 
Message-ID: <200408012352.i71NqIm27124@guido.python.org>

> And one more time.
> 
> - regen'ed against current CVS import.c
> - removes some now-needless workarounds in .py files

Looks good.  I say, check it in.  Any thoughts on where this affects docs?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Mon Aug  2 03:17:49 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  2 03:17:53 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200408012352.i71NqIm27124@guido.python.org>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C2A268@au3010avexu1.global.avaya.com>
	<1f7befae04072719401df511c3@mail.gmail.com>
	<200407281841.i6SIfo205648@guido.python.org>
	<200407281903.i6SJ3ea06175@guido.python.org>
	<1f7befae0407311632442e488b@mail.gmail.com>
	<1f7befae04080109413fe3a967@mail.gmail.com>
	<1f7befae04080116116175c3a1@mail.gmail.com>
	<1f7befae0408011614190382dc@mail.gmail.com> 
	<1f7befae0408011637458d824c@mail.gmail.com>
	<200408012352.i71NqIm27124@guido.python.org>
Message-ID: <1f7befae04080118171be3f188@mail.gmail.com>

[Guido]
> Looks good.  I say, check it in.

I can't argue with that <wink>.

>  Any thoughts on where this affects docs?

The only clear places I find are NEWS and C API's
PyImport_ExecCodeModule docs.  The latter doesn't mention sys.modules
now, so I'll add stuff and note 2.4 changes.
From greg at cosc.canterbury.ac.nz  Mon Aug  2 03:37:42 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  2 03:37:49 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200408012013.i71KDI726275@guido.python.org>
Message-ID: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz>

> > Circular imports, to be made consistent, would need
> > something like a final "commit" after all imports succeeded.
> 
> Silly idea.  Typically, circular imports aren't planned, they just
> happen, and (usually) just happen to work.

I thought Christian meant this was something that would
be done automatically, not something the user would have
to do explicitly.

Maybe it could be as simple as saving a snapshot of
sys.modules whenever importing of a module is begun,
and if execution of its body doesn't complete, restoring
the snapshot?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From pje at telecommunity.com  Mon Aug  2 04:44:18 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 04:41:55 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz>
References: <200408012013.i71KDI726275@guido.python.org>
Message-ID: <5.1.1.6.0.20040801224216.02a65c40@mail.telecommunity.com>

At 01:37 PM 8/2/04 +1200, Greg Ewing wrote:
> > > Circular imports, to be made consistent, would need
> > > something like a final "commit" after all imports succeeded.
> >
> > Silly idea.  Typically, circular imports aren't planned, they just
> > happen, and (usually) just happen to work.
>
>I thought Christian meant this was something that would
>be done automatically, not something the user would have
>to do explicitly.
>
>Maybe it could be as simple as saving a snapshot of
>sys.modules whenever importing of a module is begun,
>and if execution of its body doesn't complete, restoring
>the snapshot?

That might be over-cautious.  It's probably safe to keep modules whose body 
code executed without error.

From pje at telecommunity.com  Mon Aug  2 05:52:52 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 05:50:59 2004
Subject: [Python-Dev] Safe to change a thread's interpreter?
Message-ID: <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com>

Recently I've been researching implementation strategies for adding Java 
classloader-like capabilities to Python.  I was pleasantly surprised to 
find out that CPython already supports multiple interpreters via the C API, 
where each "interpreter" includes fresh versions of 'sys', '__builtin__', etc.

The C API doc for PyInterpreter_New(), however, says:

"""It is possible to insert objects created in one sub-interpreter into a 
namespace of another sub-interpreter; this should be done with great care 
to avoid sharing user-defined functions, methods, instances or classes 
between sub-interpreters, since import operations executed by such objects 
may affect the wrong (sub-)interpreter's dictionary of loaded modules. (XXX 
This is a hard-to-fix bug that will be addressed in a future release.)"""

It seems to me that the bug described could be fixed (or at least worked 
around) by having __import__ temporarily change the 'interp' field of the 
current thread state to point to the interpreter that the __import__ 
function lives in.  Then, at the end of the __import__, reset the 'interp' 
field back to its original value.  (Of course, it would also have to fix up 
the linked lists of the interpreters' thread states during each swap, but 
that shouldn't be too difficult.)

My question is: does this make sense, or am I completely out in left field 
here?  The only thing I can think of that this would affect is the 
'threading' module, in that trying to get the current thread from there 
(during such an import) might see a foreign interpreter's thread as its 
own.  But, I'm hard-pressed to think of any damage that could possibly 
cause.  Indeed, it seems to me that Python itself doesn't really care how 
many interpreters or thread states there are running around, and that it 
only has the linked lists to support "advanced debuggers".

Even if it's undesirable to fix the problem this way in the Python core, 
would it be acceptable to do so in an extension module?

What I have in mind is to create an extension module that wraps 
Py_InterpreterState/Py_ThreadState objects up in a subclassable extension 
type, designed to ensure the integrity of Python as a whole, while still 
allowing various import-related methods to be overridden in order to 
implement Java-style classloader hierarchies.  So, you might do something like:

     from interpreter import Interpreter

     # Run 'somescript in its own interpreter.
     it = Interpreter()
     exit_code = it.run_main("somescript.py")

     # Release resources without waiting for GC
     it.close()

My thought here also is that performing operations such as running code in 
a given Interpreter would also operate by swapping the thread state's 
'interp' field.  Thus, exceptions in the child interpreter would be 
seamlessly carried through to the parent interpreter.

In order to implement the full Java classloader model, it would also be 
necessary to be able to force imports *not* to use the Interpreter that the 
code doing the import came from.  (i.e. the equivalent of using 
'java.lang.Thread.setContextClassLoader()').  This can also probably be 
implemented via a thread-local variable in the 'interpreter' module.

So...  must a thread state always reference the same interpreter 
object?  If not, then I think I see a way to safely implement access to 
multiple interpreters from within Python itself.

From martin at v.loewis.de  Mon Aug  2 06:19:57 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug  2 06:19:58 2004
Subject: [Python-Dev] Safe to change a thread's interpreter?
In-Reply-To: <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com>
References: <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com>
Message-ID: <410DC0ED.5070201@v.loewis.de>

Phillip J. Eby wrote:
> Recently I've been researching implementation strategies for adding Java 
> classloader-like capabilities to Python.  I was pleasantly surprised to 
> find out that CPython already supports multiple interpreters via the C 
> API, where each "interpreter" includes fresh versions of 'sys', 
> '__builtin__', etc.

You should be aware that many of us consider the feature of multiple
interpreters broken. For example, global variables in extension modules
are shared across interpreters, and there is nothing that can be done
about this, except for changing the entire C API.

Regards,
Martin
From martin at v.loewis.de  Mon Aug  2 06:41:44 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug  2 06:41:44 2004
Subject: [Python-Dev] New developer
Message-ID: <410DC608.2050403@v.loewis.de>

I just granted Sean Reifschneider CVS write access, so he can directly
edit the RPM spec files. Sean has been virtually the only author of that
file for a long time.

Regards,
Martin
From greg at cosc.canterbury.ac.nz  Mon Aug  2 06:49:38 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  2 06:49:44 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <5.1.1.6.0.20040801224216.02a65c40@mail.telecommunity.com>
Message-ID: <200408020449.i724nceH026667@cosc353.cosc.canterbury.ac.nz>

> That might be over-cautious.  It's probably safe to keep modules whose body 
> code executed without error.

No, it's not. If A and B import each other, and A fails after
importing B, then B ends up with a reference to a broken A
even though it executed without error.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From pje at telecommunity.com  Mon Aug  2 06:54:08 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 06:52:17 2004
Subject: [Python-Dev] Safe to change a thread's interpreter?
In-Reply-To: <410DC0ED.5070201@v.loewis.de>
References: <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com>
	<5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040802005132.02a64d00@mail.telecommunity.com>

At 06:19 AM 8/2/04 +0200, Martin v. L?wis wrote:
>Phillip J. Eby wrote:
>>Recently I've been researching implementation strategies for adding Java 
>>classloader-like capabilities to Python.  I was pleasantly surprised to 
>>find out that CPython already supports multiple interpreters via the C 
>>API, where each "interpreter" includes fresh versions of 'sys', 
>>'__builtin__', etc.
>
>You should be aware that many of us consider the feature of multiple
>interpreters broken. For example, global variables in extension modules
>are shared across interpreters, and there is nothing that can be done
>about this, except for changing the entire C API.

Yes, I saw that as a documented limitation.  Are there undocumented 
limitations as well?  Is the feature headed for deprecation?  I guess I'm 
not understanding your implication(s), if any.

From pje at telecommunity.com  Mon Aug  2 06:57:31 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 06:55:40 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200408020449.i724nceH026667@cosc353.cosc.canterbury.ac.nz>
References: <5.1.1.6.0.20040801224216.02a65c40@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040802005457.027ef150@mail.telecommunity.com>

At 04:49 PM 8/2/04 +1200, Greg Ewing wrote:
> > That might be over-cautious.  It's probably safe to keep modules whose 
> body
> > code executed without error.
>
>No, it's not. If A and B import each other, and A fails after
>importing B, then B ends up with a reference to a broken A
>even though it executed without error.

Argh.  I can't believe I missed that.  Now that you mention it, I think 
that point was already brought up just a few days ago here.  Maybe even by 
you.  :) Ah well, please pardon my sudden case of obliviousness.  :)

From anthony at interlink.com.au  Mon Aug  2 08:32:03 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Aug  2 08:32:16 2004
Subject: [Python-Dev] 2.4a2, and @decorators
Message-ID: <410DDFE3.7010006@interlink.com.au>

The @decorator patch has been landed on the trunk, after
getting the go-ahead from the BDFL. I'll update PEP-0318
with the final syntax in the next day or so.

We're aiming for 2.4a2 this Thursday - so can people please
hold off on checking into the trunk from Wednesday evening
onwards, please.

At the moment, I'm guessing we're going to have a 2.4a3 in
a couple of weeks.

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From anthony at interlink.com.au  Mon Aug  2 08:35:16 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Aug  2 08:36:13 2004
Subject: [Python-Dev] Optik 1.5 in Python 2.4
In-Reply-To: <20040730223936.GA26696@cthulhu.gerg.ca>
References: <20040730223936.GA26696@cthulhu.gerg.ca>
Message-ID: <410DE0A4.70408@interlink.com.au>

Greg Ward wrote:
> So: I'd like to checkin Optik 1.5a1 as optparse.py right now, and worry
> about the docs later.  Hopefully before Python 2.4a2 goes out the door,
> but no promises.  Objections?  If I don't hear any soon, I'll do the
> deed tomorrow morning.

+1

From martin at v.loewis.de  Mon Aug  2 09:34:20 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug  2 09:34:20 2004
Subject: [Python-Dev] Safe to change a thread's interpreter?
In-Reply-To: <5.1.1.6.0.20040802005132.02a64d00@mail.telecommunity.com>
References: <5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com>
	<5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com>
	<5.1.1.6.0.20040802005132.02a64d00@mail.telecommunity.com>
Message-ID: <410DEE7C.1060907@v.loewis.de>

Phillip J. Eby wrote:
> Yes, I saw that as a documented limitation.  Are there undocumented 
> limitations as well?  

Might be. However, there is one more documented the limitation: The PEP
311 extensions only work for a single interpreter state, as the PEP
explains.

> Is the feature headed for deprecation?  I guess 
> I'm not understanding your implication(s), if any.

That is not surprising, as the implication is not at all obvious :-)
Here it is: Because the feature is essentially ill-designed, I'm not
going to comment on the actual question (or on any other actual
question about that feature), except for pointing out that the feature
is fundamentally flawed. This is just my own policy, though.

Regards,
Martin
From tismer at stackless.com  Mon Aug  2 11:01:26 2004
From: tismer at stackless.com (Christian Tismer)
Date: Mon Aug  2 11:00:33 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz>
References: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz>
Message-ID: <410E02E6.9000205@stackless.com>

Greg Ewing wrote:
>>>Circular imports, to be made consistent, would need
>>>something like a final "commit" after all imports succeeded.
>>
>>Silly idea.  Typically, circular imports aren't planned, they just
>>happen, and (usually) just happen to work.
> 
> 
> I thought Christian meant this was something that would
> be done automatically, not something the user would have
> to do explicitly.
> 
> Maybe it could be as simple as saving a snapshot of
> sys.modules whenever importing of a module is begun,
> and if execution of its body doesn't complete, restoring
> the snapshot?

This is exactly what I thought of.
It seems to be correct, also with nested imports,
since this is a stack-like structure of undoable things.

Example

__main__:
     import A            #1
     #6

A:
     try:
         import B        #2
         #5
     except (ImportError, maybe others):
         # provide surrogate for B

     # other action that fails

B:
     import A            #3
     import C            #4
     # do some stuff that fails

#1: snapshot of sys.modules, A added   [__main__, A]
#2: snapshot of sys.modules, B added   [__main__, A, B]
#3: snapshot of sys.modules, A exists  [__main__, A, B]
#3: snapshot of sys.modules, C added   [__main__, A, B, C]

after failing imports:

#5: restore [__main__, A]
#5: restore [__main__]

So one side effect is that completely valid imports in the
context of a failing import are undone. I guess this is
intended.

A problem are side effects, of course.
It would be nice to have a cleanup construct for
completely or partially imported modules, to undo
any side effects.
How do we remove extension modules which have been loaded
as a side effect?

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From walter at livinglogic.de  Mon Aug  2 11:24:11 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon Aug  2 11:24:34 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410DDFE3.7010006@interlink.com.au>
References: <410DDFE3.7010006@interlink.com.au>
Message-ID: <410E083B.20109@livinglogic.de>

Anthony Baxter wrote:

> The @decorator patch has been landed on the trunk, after
> getting the go-ahead from the BDFL. I'll update PEP-0318
> with the final syntax in the next day or so.
> 
> We're aiming for 2.4a2 this Thursday - so can people please
> hold off on checking into the trunk from Wednesday evening
> onwards, please.
> 
> At the moment, I'm guessing we're going to have a 2.4a3 in
> a couple of weeks.

I'm getting the following assertion from a debug build:

Python 2.4a1+ (#1, Aug  2 2004, 11:21:35)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-113)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> @staticmethod
... def x():
...  pass
...
python: Python/ceval.c:868: PyEval_EvalFrame: Assertion `(stack_pointer 
- f->f_valuestack) <= f->f_stacksize' failed.
Aborted

Bye,
    Walter D?rwald

From anthony at interlink.com.au  Mon Aug  2 13:09:11 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Aug  2 13:09:36 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <1091442793.2017.10.camel@localhost>
References: <410DDFE3.7010006@interlink.com.au>	
	<410E083B.20109@livinglogic.de>
	<1091442793.2017.10.camel@localhost>
Message-ID: <410E20D7.2040900@interlink.com.au>

Mark Russell wrote:
> Sorry - my mistake for not running the test suite under the debug
> build.  I've attached a fix.  Anthony, can you check this in as well
> please?

I've added "run the test suite under a debug build" to my list of
build tasks for the release process thingy.

Patch applied.

Anthony
From jjl at pobox.com  Mon Aug  2 15:30:39 2004
From: jjl at pobox.com (John J Lee)
Date: Mon Aug  2 15:27:38 2004
Subject: httplib patch for 2.4 (was: Re: [Python-Dev] 2.4a2 ...)
In-Reply-To: <410DDFE3.7010006@interlink.com.au>
References: <410DDFE3.7010006@interlink.com.au>
Message-ID: <Pine.WNT.4.58.0408021425420.1720@vernon>

On Mon, 2 Aug 2004, Anthony Baxter wrote:
[...]
> We're aiming for 2.4a2 this Thursday - so can people please
> hold off on checking into the trunk from Wednesday evening
> onwards, please.
[...]

Would somebody mind checking this in:

http://www.python.org/sf/997626


It fixes a bug in httplib which badly breaks urllib2 now that urllib2 is
using httplib.HTTPConnection instead of httplib.HTTP (POSTs don't work).

It includes patch and a new test, and does not require doc updates, but
nobody has reviewed it yet.


John
From jim at zope.com  Mon Aug  2 16:01:28 2004
From: jim at zope.com (Jim Fulton)
Date: Mon Aug  2 16:05:02 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com>
References: <5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org> <Your message of "Wed,
	28 Jul 2004 13:40:03
	EDT."	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org>
	<5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>
	<5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com>
Message-ID: <410E4938.80408@zope.com>

Phillip J. Eby wrote:
> At 08:28 AM 7/30/04 -0400, Jim Fulton wrote:
> 
>> The problem with Python's current package system is that it's not 
>> possible,
>> in general, to write packages that can be moved around within the package
>> system, because relative imports aren't reobust or flexible enough.
>> PEP 328 would fix this. PEP 328 would allow creation of packages that
>> worked regardless of where there were places or that worked relative
>> to a containing package regardless of where that was placed. You could
>> then have different module spaces expressed by adding a new layer of 
>> top-level
>> modules.  You wouldn't need any new machinery beyond PEP 328.
> 
> 
> Hm.  The concept I'm proposing would allow code to be used in a module 
> space without needing to know about module spaces or be written to 
> co-operate in such a fashion.  If you can make everybody conform to some 
> sort of coding standard, you wouldn't even need PEP 328 to accomplish 
> your goal.  ;)

I don't think that's true.  If your package has sub-packages, there's
no way for a subackage to import from the containing package without
using an absolute import.  Similarly, a package currentlly can't import
from a sibling package without using an absolute import. With PEP 328,
relative imports from parent or sibling (or cousin ...) packages will
be possible.

> 
> As a counterexample, consider seven Zope products that depend on two 
> different versions of PIL.  Using PEP 328, you can't make this work 
> without either loading seven copies of PIL, or else requiring the 
> version number to be part of the package name. 

No, I don't think this is right.  PEP 328 should allow you to create
module spaces using container packages.  For your example, we create
two top-level packages, space1 and space 2.  You put version x of PIL
in space1.  That package's absolute name is space1.PIL. You put version y
of PIL in space2, creating space2.PIL.  Now you put the products that
depend on version x of PIL in space1. You put the products that depend
on version y of PIL on space 2.  The products mist use relative imports
to import from PIL:

   from ..PIL import somePILmodule

For this to work, PIL also has to use relative imports to import it's own
modules.


 > You actually have it
> somewhat better right now because you can at least fool the "this 
> package or absolute" mechanism into doing what you want. 

I assume you are fering to letting local modules hide global ones.
This only works if the modules are included directly in the package.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From goodger at python.org  Mon Aug  2 16:13:36 2004
From: goodger at python.org (David Goodger)
Date: Mon Aug  2 16:13:41 2004
Subject: [Python-Dev] ConfigParser behavior change
Message-ID: <410E4C10.3080009@python.org>

Could I get some feedback re <http://www.python.org/sf/997050>?
I'd like to resolve this before 2.4a2 if possible.

In summary,

RawConfigParser.set() doesn't allow non-string arguments for 'value'
any more.  This breaks Docutils code.  I specifically chose
RawConfigParser for Docutils internal settings support because it
*did* allow non-string values.

An earlier bug report (<http://www.python.org/sf/810843>, "Support for
non-string data in ConfigParser unclear/buggy") offered 4 options:

(1) Cast the option to a string explicitly instead of assuming it
     is a string
(2) Check that the option is a string
(3) State in the documentation that the set() method only
     takes string values
(4) State in the documentation that the raw parameter should be
     used when an option's value might not be of type string

Options 2 & 3 were implemented in the "bug fix" (just a type check).
I think that the behavior should not have been changed, rather that
the documentation needed updating (i.e., choose option 4 instead).  I
volunteer to undo the change and update the documentation ASAP.

The comment for bug 810843 says "This is documented behavior", but I
couldn't find any such documentation pre-dating this change.

Thanks!

-- 
David Goodger <http://python.net/~goodger>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/426c69c5/signature.pgp
From jim at zope.com  Mon Aug  2 16:30:57 2004
From: jim at zope.com (Jim Fulton)
Date: Mon Aug  2 16:31:00 2004
Subject: [Python-Dev] Re: Can we limit the effects of module execution to
 sys.modules? (was Fix import errors to have data)
In-Reply-To: <1f7befae04073021175bc48767@mail.gmail.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4107866C.7060609@zope.com>
	<1f7befae04073021175bc48767@mail.gmail.com>
Message-ID: <410E5021.2020903@zope.com>

Tim Peters wrote:
> [Tim, wants to keep insane modules out of sys.modules]
> 
> [Jim Fulton]
> 
...

>>Do you think it's practical to limit the effects of module import to
>>sys.modules, even by convention?
> 
> 
> I'm sure you didn't intend that to be *so* extreme -- like surely a
> module is allowed to initialize its own module-level variables.   If I
> read "effects" as "effects visible outside the module", then that's
> what you said <wink>.

Yup. :)


>>If it is possible to limit the effects of import (even by convention),
>>then I think it would be practical to roll-back changes to sys.modules.
>>If it's not practical to limit the effects of  module import then I think
>>the problem is effectively unsolveable, short of making Python transactional.
> 
> 
> There we don't agree -- I think it's already practical, based on that
> virtually no Python application *intends* to catch errors from imports
> other than ImportError, so that almost all "real bugs" in module
> initialization are intended to stop execution.  In turn, in the cases
> where ImportErrors are intentionally caught now, they generally occur
> in "import blocks" near the starts of all modules in the failing
> import chain, and so none of the modules involved have yet *done* any
> non-trivial initialization -- they're all still trying to import the
> stuff they need to *start* doing the meat of their initialization. 

Except in cases where imports are places later in a module to make
circular imports work.  It would be nice to disallow circular imports,
although I don't know if that's possible.  (Some time, when I have time,
I'd like to see if we can get rid of them in Zope 3.  I'd like to have a
tool to report circular imports, to keep them from creeping in, which
they do so easily.)

Having said that, you make a good point.  If all modules did their imports
before making any changes outside the modules, then it would be pretty safe
to clean up the imports.

> If
> some modules happen to import successfully along the way, fine, they
> should stay in sys.modules, and then importing them again later won't
> run their initialization code again. 

This only works if all of the modules do all of their imports
before doing other work.

If there are circular imports, you could have:

   A defines class C
   A imports B
   B imports C from A
   A imports D and gets an import error

If we clean up A and D, then B has a class from a module that doesn't exist.

Hm. Suppose we have:

   A imports B
   B imports A
   A imports D and gets an import error

We clean up A and D. What about the A that was imported into B?

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From fdrake at acm.org  Mon Aug  2 16:33:03 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon Aug  2 16:33:16 2004
Subject: [Python-Dev] ConfigParser behavior change
In-Reply-To: <410E4C10.3080009@python.org>
References: <410E4C10.3080009@python.org>
Message-ID: <200408021033.03830.fdrake@acm.org>

On Monday 02 August 2004 10:13 am, David Goodger wrote:
 > Could I get some feedback re <http://www.python.org/sf/997050>?
 > I'd like to resolve this before 2.4a2 if possible.

David has (properly) been asking me to look into this, and i've managed not to 
have enough time.  Sorry, David!

 > RawConfigParser.set() doesn't allow non-string arguments for 'value'
 > any more.  This breaks Docutils code.  I specifically chose
 > RawConfigParser for Docutils internal settings support because it
 > *did* allow non-string values.
 >
 > An earlier bug report (<http://www.python.org/sf/810843>, "Support for
 > non-string data in ConfigParser unclear/buggy") offered 4 options:

The ConfigParser documentation was certainly too vague, but the problem, as I 
see it, is that the module was never intended to store non-string values.  I 
think allowing them is just asking for still more trouble from that module 
down the road.  Sure, the module could be made happy by reverting the patch 
you cite, but happy-by-accident is a very fragile state to be in.

 > The comment for bug 810843 says "This is documented behavior", but I
 > couldn't find any such documentation pre-dating this change.

This may have been a bug in my memory.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From pje at telecommunity.com  Mon Aug  2 16:40:18 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 16:36:18 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410DDFE3.7010006@interlink.com.au>
Message-ID: <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>

At 04:32 PM 8/2/04 +1000, Anthony Baxter wrote:
>The @decorator patch has been landed on the trunk, after
>getting the go-ahead from the BDFL. I'll update PEP-0318
>with the final syntax in the next day or so.

@decorator won?  When did that happen?

From mwh at python.net  Mon Aug  2 16:39:36 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug  2 16:39:39 2004
Subject: [Python-Dev] refleak hunting season
Message-ID: <2mpt69bo47.fsf@starship.python.net>

Last time round I found a bunch of reference counting bugs in Python
just *after* a major release; I'm trying to do it all before 2.4.0
this time.

I've hacked regrtest again (patch attached).  Here's my notes file,
though as is the case with such things writing stuff down tends to
cause thoughts which cause experiments which cause the notes to be out
of date more or less immediately...

test_codeccallbacks leaked [2, 2, 2, 2] references
    registries

test_copy leaked [82, 82, 82, 82] references
    copy_reg.dispatch_table -- now taken care of in regrtest.py

test_descr leaked [2, 2, 2, 2] references
    resurrection, still

test_descrtut leaked [1, 1, 1, 1] references
    exec 'print x' in defaultdict()
    fixed, need to check in (bit subtle though)

test_distutils leaked [2, 2, 2, 2] references
    no idea

test_gc leaked [18, 18, 18, 18] references
    trashcan

test_hotshot leaked [111, 111, 111, 111] references
    there's been a patch on SF for this for about a year!

test_minidom leaked [1, 1, 1, 1] references
    testPickledDocument, don't know more

test_mutants leaked [74, -1123, 1092, -521] references
    randomized test

test_pkg leaked [3, 3, 3, 3] references
    something to do with execfile?  or tempfile.mkstemp?
    the leak is on the dummy key from dictobject.c!!
    very odd.  see bug #808596 for the explanation.

test_pkgimport leaked [6, 6, 6, 6] references
    no idea
    similar to the above?  (but certainly not the same)
    I think this might be caused by interning like the above.

test_sax leaked [1899, 1899, 1899, 1899] references
    no idea -- scary numbers, though!

test_threadedtempfile leaked [0, 1, -21, 9] references
    randomized test?

test_threading_local leaked [9, 0, 7, 0] references
    not sure what's going on here; not sure it's a leak, but it's
    definitely odd

test_unicode leaked [7, 7, 7, 7] references
    the codec registry

test_urllib2 leaked [120, -80, -70, 120] references
    CacheFTPHandler

test_xrange leaked [1, 1, 1, 1] references
    fixed in CVS

I'd really appreciate someone taking a look at test_sax.  The numbers
for test_gc and test_descr are artifacts and if someone sees how to
fix them, I'd be grateful.  test_threading_local behaves very
strangely.  test_distutils I haven't even begun to look at.

Generally, we're in pretty good shape!  (Or our tests really suck,
your call ;-).

Cheers,
mwh

-------------- next part --------------
Index: regrtest.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/regrtest.py,v
retrieving revision 1.156
diff -c -r1.156 regrtest.py
*** regrtest.py	26 Jul 2004 12:09:13 -0000	1.156
--- regrtest.py	2 Aug 2004 14:31:41 -0000
***************
*** 8,26 ****
  
  Command line options:
  
! -v: verbose   -- run tests in verbose mode with output to stdout
! -q: quiet     -- don't print anything except if a test fails
! -g: generate  -- write the output file for a test instead of comparing it
! -x: exclude   -- arguments are tests to *exclude*
! -s: single    -- run only a single test (see below)
! -r: random    -- randomize test execution order
! -f: fromfile  -- read names of tests to run from a file (see below)
! -l: findleaks -- if GC is available detect tests that leak memory
! -u: use       -- specify which special resource intensive tests to run
! -h: help      -- print this text and exit
! -t: threshold -- call gc.set_threshold(N)
! -T: coverage  -- turn on code coverage using the trace module
! -L: runleaks  -- run the leaks(1) command just before exit
  
  If non-option arguments are present, they are names for tests to run,
  unless -x is given, in which case they are names for tests not to run.
--- 8,27 ----
  
  Command line options:
  
! -v: verbose    -- run tests in verbose mode with output to stdout
! -q: quiet      -- don't print anything except if a test fails
! -g: generate   -- write the output file for a test instead of comparing it
! -x: exclude    -- arguments are tests to *exclude*
! -s: single     -- run only a single test (see below)
! -r: random     -- randomize test execution order
! -f: fromfile   -- read names of tests to run from a file (see below)
! -l: findleaks  -- if GC is available detect tests that leak memory
! -u: use        -- specify which special resource intensive tests to run
! -h: help       -- print this text and exit
! -t: threshold  -- call gc.set_threshold(N)
! -T: coverage   -- turn on code coverage using the trace module
! -L: runleaks   -- run the leaks(1) command just before exit
! -R: huntrleaks -- search for reference leaks (needs debug build, v. slow)
  
  If non-option arguments are present, they are names for tests to run,
  unless -x is given, in which case they are names for tests not to run.
***************
*** 84,89 ****
--- 85,91 ----
  import getopt
  import random
  import warnings
+ import sre
  import cStringIO
  import traceback
  
***************
*** 127,133 ****
  
  def main(tests=None, testdir=None, verbose=0, quiet=False, generate=False,
           exclude=False, single=False, randomize=False, fromfile=None,
!          findleaks=False, use_resources=None, trace=False, runleaks=False):
      """Execute a test suite.
  
      This also parses command-line options and modifies its behavior
--- 129,136 ----
  
  def main(tests=None, testdir=None, verbose=0, quiet=False, generate=False,
           exclude=False, single=False, randomize=False, fromfile=None,
!          findleaks=False, use_resources=None, trace=False, runleaks=False,
!          huntrleaks=False):
      """Execute a test suite.
  
      This also parses command-line options and modifies its behavior
***************
*** 152,162 ****
  
      test_support.record_original_stdout(sys.stdout)
      try:
!         opts, args = getopt.getopt(sys.argv[1:], 'hvgqxsrf:lu:t:TL',
                                     ['help', 'verbose', 'quiet', 'generate',
                                      'exclude', 'single', 'random', 'fromfile',
                                      'findleaks', 'use=', 'threshold=', 'trace',
!                                     'runleaks'
                                      ])
      except getopt.error, msg:
          usage(2, msg)
--- 155,165 ----
  
      test_support.record_original_stdout(sys.stdout)
      try:
!         opts, args = getopt.getopt(sys.argv[1:], 'hvgqxsrf:lu:t:TLR',
                                     ['help', 'verbose', 'quiet', 'generate',
                                      'exclude', 'single', 'random', 'fromfile',
                                      'findleaks', 'use=', 'threshold=', 'trace',
!                                     'runleaks', 'huntrleaks'
                                      ])
      except getopt.error, msg:
          usage(2, msg)
***************
*** 191,196 ****
--- 194,201 ----
              gc.set_threshold(int(a))
          elif o in ('-T', '--coverage'):
              trace = True
+         elif o in ('-R', '--huntrleaks'):
+             huntrleaks = True
          elif o in ('-u', '--use'):
              u = [x.lower() for x in a.split(',')]
              for r in u:
***************
*** 288,294 ****
              tracer.runctx('runtest(test, generate, verbose, quiet, testdir)',
                            globals=globals(), locals=vars())
          else:
!             ok = runtest(test, generate, verbose, quiet, testdir)
              if ok > 0:
                  good.append(test)
              elif ok == 0:
--- 293,299 ----
              tracer.runctx('runtest(test, generate, verbose, quiet, testdir)',
                            globals=globals(), locals=vars())
          else:
!             ok = runtest(test, generate, verbose, quiet, testdir, huntrleaks)
              if ok > 0:
                  good.append(test)
              elif ok == 0:
***************
*** 397,403 ****
      tests.sort()
      return stdtests + tests
  
! def runtest(test, generate, verbose, quiet, testdir=None):
      """Run a single test.
      test -- the name of the test
      generate -- if true, generate output, instead of running the test
--- 402,408 ----
      tests.sort()
      return stdtests + tests
  
! def runtest(test, generate, verbose, quiet, testdir=None, huntrleaks=False):
      """Run a single test.
      test -- the name of the test
      generate -- if true, generate output, instead of running the test
***************
*** 415,420 ****
--- 420,427 ----
          cfp = None
      else:
          cfp = cStringIO.StringIO()
+     if huntrleaks:
+         refrep = open("reflog.txt", "a")
      try:
          save_stdout = sys.stdout
          try:
***************
*** 435,440 ****
--- 442,478 ----
              indirect_test = getattr(the_module, "test_main", None)
              if indirect_test is not None:
                  indirect_test()
+             if huntrleaks:
+                 import copy_reg
+                 fs = warnings.filters[:]
+                 ps = copy_reg.dispatch_table.copy()
+                 import gc
+                 def cleanup():
+                     import _strptime, urlparse, warnings
+                     warnings.filters[:] = fs
+                     gc.collect()
+                     sre.purge()
+                     _strptime._regex_cache.clear()
+                     urlparse.clear_cache()
+                     copy_reg.dispatch_table.clear()
+                     copy_reg.dispatch_table.update(ps)
+                 deltas = []
+                 if indirect_test:
+                     for i in range(9):
+                         rc = sys.gettotalrefcount()
+                         indirect_test()
+                         cleanup()
+                         deltas.append(sys.gettotalrefcount() - rc - 2)
+                 else:
+                     for i in range(9):
+                         rc = sys.gettotalrefcount()
+                         reload(the_module)
+                         cleanup()
+                         deltas.append(sys.gettotalrefcount() - rc - 2)
+                 if max(map(abs, deltas[-4:])) > 0:
+                     print >>refrep, test, 'leaked', \
+                           deltas[-4:], 'references'
+                     #refrep.flush()
          finally:
              sys.stdout = save_stdout
      except test_support.ResourceDenied, msg:
***************
*** 486,492 ****
              fp.close()
          else:
              expected = test + "\n"
!         if output == expected:
              return 1
          print "test", test, "produced unexpected output:"
          sys.stdout.flush()
--- 524,530 ----
              fp.close()
          else:
              expected = test + "\n"
!         if output == expected or huntrleaks:
              return 1
          print "test", test, "produced unexpected output:"
          sys.stdout.flush()
-------------- next part --------------


-- 
  I never disputed the Perl hacking skill of the Slashdot creators. 
  My objections are to the editors' taste, the site's ugly visual 
  design, and the Slashdot community's raging stupidity.
     -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq
From pje at telecommunity.com  Mon Aug  2 16:59:19 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 16:55:18 2004
Subject: [Python-Dev] Safe to change a thread's interpreter?
In-Reply-To: <410DEE7C.1060907@v.loewis.de>
References: <5.1.1.6.0.20040802005132.02a64d00@mail.telecommunity.com>
	<5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com>
	<5.1.1.6.0.20040801230324.02170d80@mail.telecommunity.com>
	<5.1.1.6.0.20040802005132.02a64d00@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040802104550.035e4110@mail.telecommunity.com>

At 09:34 AM 8/2/04 +0200, Martin v. L?wis wrote:
>Phillip J. Eby wrote:
>>Yes, I saw that as a documented limitation.  Are there undocumented 
>>limitations as well?
>
>Might be. However, there is one more documented the limitation: The PEP
>311 extensions only work for a single interpreter state, as the PEP
>explains.

Maybe I'm misinterpreting it, but the "Design and Implementation" section 
sounds like the only issue would be that any automatically-allocated thread 
state would point to the primary interpreter.

For my intended use, that's not actually a problem, since import operations 
would have to switch the interpreter to the "correct" one, anyway.

From barry at python.org  Mon Aug  2 17:01:32 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug  2 17:01:29 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
References: <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
Message-ID: <1091458891.9134.29.camel@localhost>

On Mon, 2004-08-02 at 10:40, Phillip J. Eby wrote:
> At 04:32 PM 8/2/04 +1000, Anthony Baxter wrote:
> >The @decorator patch has been landed on the trunk, after
> >getting the go-ahead from the BDFL. I'll update PEP-0318
> >with the final syntax in the next day or so.
> 
> @decorator won?  When did that happen?

First I heard about it was when I saw Anthony's checkin messages.  Maybe
it was throwing that pie that inspired Guido -- an '@' does kind of look
like a pie.  I think Anthony's checkin message was accurate enough
though -- it's the syntax everyone can hate equally.  But I'm glad
/something/ made it in!

print>>-set-the-precedent-ly y'rs,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/1dad5c36/attachment.pgp
From skip at pobox.com  Mon Aug  2 17:04:20 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug  2 17:04:27 2004
Subject: [Python-Dev] refleak hunting season
In-Reply-To: <2mpt69bo47.fsf@starship.python.net>
References: <2mpt69bo47.fsf@starship.python.net>
Message-ID: <16654.22516.768093.314537@montanaro.dyndns.org>


    Michael> I've hacked regrtest again (patch attached).

Why not just check your regrtest changes into cvs?

Skip
From mwh at python.net  Mon Aug  2 17:11:36 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug  2 17:11:38 2004
Subject: [Python-Dev] refleak hunting season
In-Reply-To: <16654.22516.768093.314537@montanaro.dyndns.org> (Skip
	Montanaro's message of "Mon, 2 Aug 2004 10:04:20 -0500")
References: <2mpt69bo47.fsf@starship.python.net>
	<16654.22516.768093.314537@montanaro.dyndns.org>
Message-ID: <2mllgxbmmv.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

>     Michael> I've hacked regrtest again (patch attached).
>
> Why not just check your regrtest changes into cvs?

Because I'm not quite happy with them; I hope to make the changes I
want after the run I'm doing with -uall finishes (so, erm, Thursday or
so :-).

Also, there are entirely too many tests that fail when run with -R:

    test_builtin test_cgi test_decimal test_dircache test_doctest
    test_getopt test_logging test_multifile test_socket test_warnings
    test_zipimport

(I meant to mention this in the first mail).  I'll probably be filing
bug reports on or trying to fix some of these -- tests that muck with
the environment have a tendency to be a pain under any circumstances.

Oh, and test_threaded_import still deadlocks.  That's really annoying!

Cheers,
mwh

-- 
  MARVIN:  What a depressingly stupid machine.
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 7
From anthony at interlink.com.au  Mon Aug  2 17:14:50 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Aug  2 17:15:09 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <1091458891.9134.29.camel@localhost>
References: <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
Message-ID: <410E5A6A.4020408@interlink.com.au>

Barry Warsaw wrote:
> On Mon, 2004-08-02 at 10:40, Phillip J. Eby wrote:
>>@decorator won?  When did that happen?
> 
> 
> First I heard about it was when I saw Anthony's checkin messages.  Maybe
> it was throwing that pie that inspired Guido -- an '@' does kind of look
> like a pie.  I think Anthony's checkin message was accurate enough
> though -- it's the syntax everyone can hate equally.  But I'm glad
> /something/ made it in!
> 
> print>>-set-the-precedent-ly y'rs,
> -Barry

Hey, I was just going off Guido's decision (in email - looking back, it
wasn't cc'd to python-dev, which probably explains the lack of 400
followups <wink>)

Channelling the BDFL, I think the idea was to put it in 2.4a2, and see
how it works for people. If it turns out that it's really really really
hated, we can try something else in a3. Guido can of course follow up
to this (hint hint) and give his own answers.

I've been playing with it for most of the last week, on and off, and I
can say that I hate it less now. The @ makes it very obvious visually
what's going on, which the other syntaxes didn't.

But I like Barry's idea - I think it should be referred to at the
'pie decorator syntax'.

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From walter at livinglogic.de  Mon Aug  2 17:27:37 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon Aug  2 17:27:40 2004
Subject: [Python-Dev] refleak hunting season
In-Reply-To: <2mllgxbmmv.fsf@starship.python.net>
References: <2mpt69bo47.fsf@starship.python.net>	<16654.22516.768093.314537@montanaro.dyndns.org>
	<2mllgxbmmv.fsf@starship.python.net>
Message-ID: <410E5D69.3060009@livinglogic.de>

Michael Hudson wrote:

> Skip Montanaro <skip@pobox.com> writes:
> 
>>    Michael> I've hacked regrtest again (patch attached).
>>
>>Why not just check your regrtest changes into cvs?
> 
> Because I'm not quite happy with them; I hope to make the changes I
> want after the run I'm doing with -uall finishes (so, erm, Thursday or
> so :-).

I would prefer it, if the refcounting was done for each test method
not only for the complete test. This would help in finding the leak
but unfortunately would required changes to unittest.

Bye,
    Walter D?rwald


From barry at python.org  Mon Aug  2 17:33:01 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug  2 17:32:58 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410E5A6A.4020408@interlink.com.au>
References: <5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<410E5A6A.4020408@interlink.com.au>
Message-ID: <1091460781.9150.37.camel@localhost>

On Mon, 2004-08-02 at 11:14, Anthony Baxter wrote:

> But I like Barry's idea - I think it should be referred to at the
> 'pie decorator syntax'.

:)  I'm going to convert MM3 to use pie decorator syntax to play with
this (in my CST).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/c3338b6a/attachment.pgp
From mwh at python.net  Mon Aug  2 17:39:27 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug  2 17:39:28 2004
Subject: [Python-Dev] refleak hunting season
In-Reply-To: <410E5D69.3060009@livinglogic.de> (
	=?iso-8859-1?q?Walter_D=F6rwald's_message_of?= "Mon,
	02 Aug 2004 17:27:37 +0200")
References: <2mpt69bo47.fsf@starship.python.net>
	<16654.22516.768093.314537@montanaro.dyndns.org>
	<2mllgxbmmv.fsf@starship.python.net> <410E5D69.3060009@livinglogic.de>
Message-ID: <2mhdrlblcg.fsf@starship.python.net>

Walter D?rwald <walter@livinglogic.de> writes:

> Michael Hudson wrote:
>
>> Skip Montanaro <skip@pobox.com> writes:
>> 
>>>    Michael> I've hacked regrtest again (patch attached).
>>>
>>>Why not just check your regrtest changes into cvs?
>> Because I'm not quite happy with them; I hope to make the changes I
>> want after the run I'm doing with -uall finishes (so, erm, Thursday or
>> so :-).
>
> I would prefer it, if the refcounting was done for each test method
> not only for the complete test. This would help in finding the leak
> but unfortunately would required changes to unittest.

Certainly, this is a superior solution.  However, it also has the
problem that from regrtest you can't tell whether a given test is
using unittest, doctest or whatever.  It's probably possible to
redesign things so this is all elegant and natural, but I actually
wanted to get somewhere with this :-)

Cheers,
mwh

-- 
  I have a feeling that any simple problem can be made arbitrarily
  difficult by imposing a suitably heavy administrative process
  around the development.       -- Joe Armstrong, comp.lang.functional
From jim at zope.com  Mon Aug  2 18:32:56 2004
From: jim at zope.com (Jim Fulton)
Date: Mon Aug  2 18:33:01 2004
Subject: [Python-Dev] Proposed Python api for testing whether a module exists
Message-ID: <410E6CB8.3070001@zope.com>


I propose to add the following function to the imp module:

   exists(module_name)

      Test whether the named module can be found.

      This method should be used when you want to perform
      alternate actions depending on whether a module is available.

      Returns True of the module can be found and False otherwise.
      A True return value does not guarentee that the module will be
      importable.  The module will not be imported as a result of
      calling exists, but parent modules may be.

      If a dotted module name is given, then parent modules will be
      imported if they haven't already been imported.


If there are no objections, I'll try to get this in before Wednesday night.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Mon Aug  2 18:42:30 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  2 18:42:36 2004
Subject: [Python-Dev] Re: Can we limit the effects of module execution to
	sys.modules? (was Fix import errors to have data)
In-Reply-To: <410E5021.2020903@zope.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4107866C.7060609@zope.com>
	<1f7befae04073021175bc48767@mail.gmail.com> <410E5021.2020903@zope.com>
Message-ID: <1f7befae04080209424ee937a8@mail.gmail.com>

[Tim Peters]
>> ...I think it's already practical, based on that
>> virtually no Python application *intends* to catch errors from imports
>> other than ImportError, so that almost all "real bugs" in module
>> initialization are intended to stop execution.  In turn, in the cases
>> where ImportErrors are intentionally caught now, they generally occur
>> in "import blocks" near the starts of all modules in the failing
>> import chain, and so none of the modules involved have yet *done* any
>> non-trivial initialization -- they're all still trying to import the
>> stuff they need to *start* doing the meat of their initialization.

[Jim Fulton]
> Except in cases where imports are places later in a module to make
> circular imports work.

That's being discussed in "the other" thread (the now-massive "Fix
import errors to have data" thread).  I never do circular imports
intentionally, so my interest in that is unmeasurable -- on the low
end <wink>.

Note that I checked in changes last night so that a failing import in
2.4 *will* delete the failing importee from sys.modules.  This breaks
a couple Zope3 tests, BTW (explained in the other thread).

> It would be nice to disallow circular imports, although I don't know if that's
> possible.

I doubt it, because it would break existing code "gratuitously". 
Maybe in Python 3.  IMO the behavior of circular imports in CPython
today is about half implementation accident.

>  (Some time, when I have time, I'd like to see if we can get rid of them in Zope 3.

+1

> I'd like to have a tool to report circular imports, to keep them from creeping in,
> which they do so easily.)

Wasn't Fred working on such a tool?

> Having said that, you make a good point.  If all modules did their imports
> before making any changes outside the modules, then it would be pretty safe
> to clean up the imports.

I think most code already does.

>> If some modules happen to import successfully along the way, fine, they
>> should stay in sys.modules, and then importing them again later won't
>> run their initialization code again.

> This only works if all of the modules do all of their imports before doing other
> work.

It works in many more cases than just that.

   do a ton of work
   import A
   do more work
   blow up

A is almost certainly still fine in that case.  I understand we can
concoct pathological cases in which it isn't.

> If there are circular imports, you could have:
>
>   A defines class C
>   A imports B
>   B imports C from A
>   A imports D and gets an import error
>
> If we clean up A and D, then B has a class from a module that doesn't exist.

Yes, and that's unpleasant.  If C defines any methods, A.__dict__ will
still be alive (because methods of C refer to it via their
func_globals), but when the module object itself goes away its
__dict__ values get set to None.  So any method of C that refers to a
module global is likely to suffer a mysterious None-related exception.
 It's not going to segfault, though.

OTOH, to get into trouble, someone has to suppress the ImportError
raised by trying to import A.  You have to work pretty hard to get in
trouble.

> Hm. Suppose we have:
>
>   A imports B
>   B imports A
>   A imports D and gets an import error
> 
> We clean up A and D. What about the A that was imported into B?

I believe that works the same as before the patch, except that 'A' and
'D' are no longer in sys.modules.  The only "clean up" done by the
patch is, in case of error, to remove the module name from
sys.modules.  That removes a reference to the module object; it
doesn't do anything more than that directly.  The module object in
this case remains alive, and its __dict__ remains intact, because B
holds a reference to the module object.  If someone suppresses the
ImportError from importing A, then, then the primary difference is
that *another* attempt to import A will again raise ImportError now.

Over in the other thread, people are talking about schemes to do
better in circular import cases that suffer errors.  My patch ignored
those issues (beyond convincing myself segfaults wouldn't result),
because Guido's patch I built on ignored them <wink>.  That was enough
to allow removing some atrocious code in the Python libraries trying
to worm around damaged modules left behind in sys.modules, and appears
so far to be enough to allow "regrtest.py -r" (randomize test order)
to pass on Windows reliably for the first time in Python's history. 
So I think it's a real improvement, despite that it doesn't solve all
import-error problems.
From pje at telecommunity.com  Mon Aug  2 18:55:07 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 18:51:07 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410E5A6A.4020408@interlink.com.au>
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
Message-ID: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>

At 01:14 AM 8/3/04 +1000, Anthony Baxter wrote:
>Barry Warsaw wrote:
>>On Mon, 2004-08-02 at 10:40, Phillip J. Eby wrote:
>>>@decorator won?  When did that happen?
>>
>>First I heard about it was when I saw Anthony's checkin messages.  Maybe
>>it was throwing that pie that inspired Guido -- an '@' does kind of look
>>like a pie.  I think Anthony's checkin message was accurate enough
>>though -- it's the syntax everyone can hate equally.  But I'm glad
>>/something/ made it in!
>>print>>-set-the-precedent-ly y'rs,
>>-Barry
>
>Hey, I was just going off Guido's decision (in email - looking back, it
>wasn't cc'd to python-dev, which probably explains the lack of 400
>followups <wink>)
>
>Channelling the BDFL, I think the idea was to put it in 2.4a2, and see
>how it works for people. If it turns out that it's really really really
>hated, we can try something else in a3. Guido can of course follow up
>to this (hint hint) and give his own answers.

I would think the fact that the '[decorators]' syntax can be implemented in 
pure Python (no changes to the interpreter) for existing Python versions 
would give more weight to it.  That is, if someone wants to implement a 
decorator that's forwards and backwards-compatible, that's possible with 
the list syntax, but not the @ syntax.

From pje at telecommunity.com  Mon Aug  2 18:57:51 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 18:53:50 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <410E4938.80408@zope.com>
References: <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com>
	<5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org> <Your message of "Wed,
	28 Jul 2004 13:40:03
	EDT."	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org>
	<5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>
	<5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040802125556.05c5bc20@mail.telecommunity.com>

At 10:01 AM 8/2/04 -0400, Jim Fulton wrote:
>Phillip J. Eby wrote:
>>At 08:28 AM 7/30/04 -0400, Jim Fulton wrote:
>>
>>>The problem with Python's current package system is that it's not possible,
>>>in general, to write packages that can be moved around within the package
>>>system, because relative imports aren't reobust or flexible enough.
>>>PEP 328 would fix this. PEP 328 would allow creation of packages that
>>>worked regardless of where there were places or that worked relative
>>>to a containing package regardless of where that was placed. You could
>>>then have different module spaces expressed by adding a new layer of 
>>>top-level
>>>modules.  You wouldn't need any new machinery beyond PEP 328.
>>
>>Hm.  The concept I'm proposing would allow code to be used in a module 
>>space without needing to know about module spaces or be written to 
>>co-operate in such a fashion.  If you can make everybody conform to some 
>>sort of coding standard, you wouldn't even need PEP 328 to accomplish 
>>your goal.  ;)
>
>No, I don't think this is right.  PEP 328 should allow you to create
>module spaces using container packages.  For your example, we create
>two top-level packages, space1 and space 2.  You put version x of PIL
>in space1.  That package's absolute name is space1.PIL. You put version y
>of PIL in space2, creating space2.PIL.  Now you put the products that
>depend on version x of PIL in space1. You put the products that depend
>on version y of PIL on space 2.  The products mist use relative imports
>to import from PIL:
>
>   from ..PIL import somePILmodule
>
>For this to work, PIL also has to use relative imports to import it's own
>modules.

As I said, if you get to make PIL and everybody else rewrite their imports, 
you can do whatever you want to already.  :)  The point of my proposal is 
to make it possible *without* rewriting imports, in versions of Python from 
2.2 up.

From pje at telecommunity.com  Mon Aug  2 19:01:02 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 18:57:00 2004
Subject: [Python-Dev] Proposed Python api for testing whether a
	module exists
In-Reply-To: <410E6CB8.3070001@zope.com>
Message-ID: <5.1.1.6.0.20040802125815.05c5fea0@mail.telecommunity.com>

At 12:32 PM 8/2/04 -0400, Jim Fulton wrote:

>I propose to add the following function to the imp module:
>
>   exists(module_name)
>
>      Test whether the named module can be found.
>
>      This method should be used when you want to perform
>      alternate actions depending on whether a module is available.
>
>      Returns True of the module can be found and False otherwise.
>      A True return value does not guarentee that the module will be
>      importable.  The module will not be imported as a result of
>      calling exists, but parent modules may be.
>
>      If a dotted module name is given, then parent modules will be
>      imported if they haven't already been imported.
>
>
>If there are no objections, I'll try to get this in before Wednesday night.

Is this intended to work in the presence of import hooks?  In particular, 
all the hooks offered by PEP 302, but also systems like 'iu', 'imputil', etc.

Or, more specifically, will this be implemented in terms of the PEP 302 
importer/loader protocols, or will those protocols need extending in order 
to allow such loaders (e.g. zipimport) to work with this new function?

From pf_moore at yahoo.co.uk  Mon Aug  2 19:06:09 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Mon Aug  2 19:05:56 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
References: <410DDFE3.7010006@interlink.com.au>
Message-ID: <isc1jwqm.fsf@yahoo.co.uk>

Anthony Baxter <anthony@interlink.com.au> writes:

> We're aiming for 2.4a2 this Thursday - so can people please
> hold off on checking into the trunk from Wednesday evening
> onwards, please.

Can someone have a look at http://www.python.org/sf/870382 ? I'm not
at all sure how to address Martin's comments, and I'd like to see this
in Python 2.4, in some form or other.

Thanks,
Paul
-- 
Ooh, how Gothic. Barring the milk.

From pf_moore at yahoo.co.uk  Mon Aug  2 19:09:57 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Mon Aug  2 19:11:04 2004
Subject: [Python-Dev] Re: Proposed Python api for testing whether a module
	exists
References: <410E6CB8.3070001@zope.com>
Message-ID: <ekmpjwka.fsf@yahoo.co.uk>

Jim Fulton <jim@zope.com> writes:

> I propose to add the following function to the imp module:
>
>    exists(module_name)
[...]
> If there are no objections, I'll try to get this in before Wednesday night.

Does this work in the presence of PEP 302 style import hooks?
Specifically, the case where there's a zipfile on sys.path.

Paul.
-- 
A little inaccuracy sometimes saves tons of explanation -- Saki

From bob at redivi.com  Mon Aug  2 19:14:11 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug  2 19:14:23 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
Message-ID: <5F9B9E10-E4A7-11D8-B47B-000A95686CD8@redivi.com>


On Aug 2, 2004, at 12:55 PM, Phillip J. Eby wrote:

> At 01:14 AM 8/3/04 +1000, Anthony Baxter wrote:
>> Barry Warsaw wrote:
>>> On Mon, 2004-08-02 at 10:40, Phillip J. Eby wrote:
>>>> @decorator won?  When did that happen?
>>>
>>> First I heard about it was when I saw Anthony's checkin messages.  
>>> Maybe
>>> it was throwing that pie that inspired Guido -- an '@' does kind of 
>>> look
>>> like a pie.  I think Anthony's checkin message was accurate enough
>>> though -- it's the syntax everyone can hate equally.  But I'm glad
>>> /something/ made it in!
>>> print>>-set-the-precedent-ly y'rs,
>>> -Barry
>>
>> Hey, I was just going off Guido's decision (in email - looking back, 
>> it
>> wasn't cc'd to python-dev, which probably explains the lack of 400
>> followups <wink>)
>>
>> Channelling the BDFL, I think the idea was to put it in 2.4a2, and see
>> how it works for people. If it turns out that it's really really 
>> really
>> hated, we can try something else in a3. Guido can of course follow up
>> to this (hint hint) and give his own answers.
>
> I would think the fact that the '[decorators]' syntax can be 
> implemented in pure Python (no changes to the interpreter) for 
> existing Python versions would give more weight to it.  That is, if 
> someone wants to implement a decorator that's forwards and 
> backwards-compatible, that's possible with the list syntax, but not 
> the @ syntax.

... but that also means you can still make the [decorators] syntax work 
in 2.4, if you want compatibility or don't like @syntax.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3589 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/0b1718f6/smime.bin
From tim.peters at gmail.com  Mon Aug  2 19:15:20 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  2 19:15:23 2004
Subject: [Python-Dev] Proposed Python api for testing whether a module
	exists
In-Reply-To: <410E6CB8.3070001@zope.com>
References: <410E6CB8.3070001@zope.com>
Message-ID: <1f7befae0408021015624f0059@mail.gmail.com>

[Jim Fulton]
> I propose to add the following function to the imp module:
>
>   exists(module_name)
>
>      Test whether the named module can be found.

+1.  This can be much better than catching ImportError, for reasons
previously given.  site.py should be changed to use this to determine
whether sitecustomize exists (as is, if sitecustomize exists but
screws up by trying to import something that isn't on sys.path,
site.py suppresses the ImportError raised *by* sitecustomize; this is
bad).
From amk at amk.ca  Mon Aug  2 19:24:19 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Aug  2 19:24:23 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <isc1jwqm.fsf@yahoo.co.uk>
References: <410DDFE3.7010006@interlink.com.au> <isc1jwqm.fsf@yahoo.co.uk>
Message-ID: <20040802172419.GA9057@rogue.amk.ca>

On Mon, Aug 02, 2004 at 06:06:09PM +0100, Paul Moore wrote:
> Can someone have a look at http://www.python.org/sf/870382 ? I'm not
> at all sure how to address Martin's comments, and I'd like to see this
> in Python 2.4, in some form or other.

If you'd like to encourage examination of a bug or patch at the next
bug day, you can add them to the 'bugs for examination' section of
http://www.python.org/moin/PythonBugDayStatus.  No guarantees, of
course -- if the bug can only be reviewed by one person, and that
person isn't around for the bug day, nothing much can be done -- but
at least it'll draw some attention to them.

(Will send an announcement for the next bug day shortly...)

--amk
From pje at telecommunity.com  Mon Aug  2 19:30:03 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 19:26:13 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <5F9B9E10-E4A7-11D8-B47B-000A95686CD8@redivi.com>
References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>

At 01:14 PM 8/2/04 -0400, Bob Ippolito wrote:

>>I would think the fact that the '[decorators]' syntax can be implemented 
>>in pure Python (no changes to the interpreter) for existing Python 
>>versions would give more weight to it.  That is, if someone wants to 
>>implement a decorator that's forwards and backwards-compatible, that's 
>>possible with the list syntax, but not the @ syntax.
>
>.. but that also means you can still make the [decorators] syntax work in 
>2.4, if you want compatibility or don't like @syntax.

But then why not just make that the default syntax, so that no migration is 
necessary, and only one syntax has to be learned/explained to people?

From jim at zope.com  Mon Aug  2 19:32:12 2004
From: jim at zope.com (Jim Fulton)
Date: Mon Aug  2 19:32:16 2004
Subject: [Python-Dev] Re: Can we limit the effects of module execution
	to sys.modules? (was Fix import errors to have data)
In-Reply-To: <1f7befae04080209424ee937a8@mail.gmail.com>
References: <41065006.6020607@zope.com>	<200407271445.i6REjEK24454@guido.python.org>	<41067660.2070109@zope.com>	<1f7befae040727085715ce2536@mail.gmail.com>	<4107866C.7060609@zope.com>	<1f7befae04073021175bc48767@mail.gmail.com>
	<410E5021.2020903@zope.com>
	<1f7befae04080209424ee937a8@mail.gmail.com>
Message-ID: <410E7A9C.1060301@zope.com>

Tim Peters wrote:
> [Tim Peters]
> 
...

>>I'd like to have a tool to report circular imports, to keep them from creeping in,
>>which they do so easily.)
> 
> 
> Wasn't Fred working on such a tool?

He did, but I can't make sense of the output, so I can't use it. :)
I'm hoping that soon Fred and I can spend some time to make sense of
the output.

...

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From jim at zope.com  Mon Aug  2 19:37:10 2004
From: jim at zope.com (Jim Fulton)
Date: Mon Aug  2 19:37:15 2004
Subject: [Python-Dev] Proposed Python api for testing whether a module
	exists
In-Reply-To: <410E6F05.5040400@interlink.com.au>
References: <410E6CB8.3070001@zope.com> <410E6F05.5040400@interlink.com.au>
Message-ID: <410E7BC6.6040301@zope.com>

Anthony Baxter wrote:
> Jim Fulton wrote:
> 
>>      If a dotted module name is given, then parent modules will be
>>      imported if they haven't already been imported.
> 
> 
> So you might end up spelling something like:
> 
> if imp.exists('pkg') and imp.exists('pkg.subpkg') \
>      and imp.exists('pkg.subpkg.module'):
> 
> to avoid an importerror on the parent module?

No. If pkg doesn't exist then exists will report False.

Here's new wording:

    exists(module_name)

      Test whether the named module can be found.

      This method should be used when you want to perform
      alternate actions depending on whether a module is available.

      Returns True of the module can be found and False otherwise.
      A True return value does not guarentee that the module will be
      importable.  The module will not be imported as a result of
      calling exists, but parent modules may be.

      If a dotted module name is given and parent modules exist,
      then the parent modules will be imported if they haven't
      already been imported.

Thanks,

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From jhylton at gmail.com  Mon Aug  2 19:46:05 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Mon Aug  2 19:46:10 2004
Subject: httplib patch for 2.4 (was: Re: [Python-Dev] 2.4a2 ...)
In-Reply-To: <Pine.WNT.4.58.0408021425420.1720@vernon>
References: <410DDFE3.7010006@interlink.com.au>
	<Pine.WNT.4.58.0408021425420.1720@vernon>
Message-ID: <e8bf7a53040802104616e42b40@mail.gmail.com>

I'll take a look, probably tomorrow morning.

Jeremy

On Mon, 2 Aug 2004 14:30:39 +0100 (GMT Daylight Time), John J Lee
<jjl@pobox.com> wrote:
> On Mon, 2 Aug 2004, Anthony Baxter wrote:
> [...]
> > We're aiming for 2.4a2 this Thursday - so can people please
> > hold off on checking into the trunk from Wednesday evening
> > onwards, please.
> [...]
> 
> Would somebody mind checking this in:
> 
> http://www.python.org/sf/997626
> 
> It fixes a bug in httplib which badly breaks urllib2 now that urllib2 is
> using httplib.HTTPConnection instead of httplib.HTTP (POSTs don't work).
> 
> It includes patch and a new test, and does not require doc updates, but
> nobody has reviewed it yet.
From edloper at gradient.cis.upenn.edu  Mon Aug  2 19:50:42 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Mon Aug  2 19:49:41 2004
Subject: [Python-Dev] Proposed Python api for testing whether a module
	exists
In-Reply-To: <20040802164253.EBD151E4007@bag.python.org>
References: <20040802164253.EBD151E4007@bag.python.org>
Message-ID: <410E7EF2.9040707@gradient.cis.upenn.edu>

> I propose to add the following function to the imp module:
> 
>   exists(module_name)
> 
>      Test whether the named module can be found.  [...]
>      If a dotted module name is given, then parent modules will be
>      imported if they haven't already been imported.

If a parent module doesn't exist, will it return False or raise an 
ImportError?

-Edward
From barry at python.org  Mon Aug  2 20:01:23 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug  2 20:01:19 2004
Subject: [Python-Dev] Proposed Python api for testing whether a module
	exists
In-Reply-To: <410E6CB8.3070001@zope.com>
References: <410E6CB8.3070001@zope.com>
Message-ID: <1091469682.18326.13.camel@localhost>

On Mon, 2004-08-02 at 12:32, Jim Fulton wrote:
> I propose to add the following function to the imp module:
> 
>    exists(module_name)
> 
>       Test whether the named module can be found.
> 
>       This method should be used when you want to perform
>       alternate actions depending on whether a module is available.
> 
>       Returns True of the module can be found and False otherwise.
>       A True return value does not guarentee that the module will be
>       importable.  The module will not be imported as a result of
>       calling exists, but parent modules may be.
> 
>       If a dotted module name is given, then parent modules will be
>       imported if they haven't already been imported.

+1, but shouldn't "parent module" really be spelled "parent package"?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/a6edcd8c/attachment.pgp
From jim at zope.com  Mon Aug  2 20:03:16 2004
From: jim at zope.com (Jim Fulton)
Date: Mon Aug  2 20:03:21 2004
Subject: [Python-Dev] Proposed Python api for testing whether a module
	exists
In-Reply-To: <410E7EF2.9040707@gradient.cis.upenn.edu>
References: <20040802164253.EBD151E4007@bag.python.org>
	<410E7EF2.9040707@gradient.cis.upenn.edu>
Message-ID: <410E81E4.7090702@zope.com>

Edward Loper wrote:
>> I propose to add the following function to the imp module:
>>
>>   exists(module_name)
>>
>>      Test whether the named module can be found.  [...]
>>      If a dotted module name is given, then parent modules will be
>>      imported if they haven't already been imported.
> 
> 
> If a parent module doesn't exist, will it return False or raise an 
> ImportError?

Return false.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From python at rcn.com  Mon Aug  2 08:20:18 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug  2 20:21:53 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
Message-ID: <004101c47858$c8be8200$a2ae2c81@oemcomputer>

Is anyone else seeing a stack overflow assertion error in test_pyclbr.py
(WinMe, MSVC++6.0)?


Raymond

From guido at python.org  Mon Aug  2 20:40:45 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  2 20:40:57 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: Your message of "Mon, 02 Aug 2004 13:37:42 +1200."
	<200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> 
References: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408021840.i72Iejt28600@guido.python.org>

> > > Circular imports, to be made consistent, would need
> > > something like a final "commit" after all imports succeeded.
> > 
> > Silly idea.  Typically, circular imports aren't planned, they just
> > happen, and (usually) just happen to work.
> 
> I thought Christian meant this was something that would
> be done automatically, not something the user would have
> to do explicitly.

OK, I misunderstood then.  Apologies to Christian.

> Maybe it could be as simple as saving a snapshot of
> sys.modules whenever importing of a module is begun,
> and if execution of its body doesn't complete, restoring
> the snapshot?

That sounds like overkill -- any modules that don't participate in the
cycle but happen to be imported by one of the modules in the cycle
would also be deleted.

I still expect that the solution that Tim just checked in (delete any
module that failed to complete execution) will solve most of the
problems encountered in real life.  I'm aware of the case you describe
here:

> No, it's not. If A and B import each other, and A fails after
> importing B, then B ends up with a reference to a broken A
> even though it executed without error.

But I don't lose sleep over it; this would only be a problem if the
main code attempts to import A but suppresses the import error *and*
also uses B directly.  This doesn't seem to be a likely scenario
(modules that can be missed rarely participate in cycles).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Aug  2 20:49:17 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  2 20:49:25 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Mon, 02 Aug 2004 16:32:03 +1000."
	<410DDFE3.7010006@interlink.com.au> 
References: <410DDFE3.7010006@interlink.com.au> 
Message-ID: <200408021849.i72InHa28632@guido.python.org>

> The @decorator patch has been landed on the trunk, after
> getting the go-ahead from the BDFL. I'll update PEP-0318
> with the final syntax in the next day or so.

Thanks, Anthony!

Especially thanks for thinking of updating the PEP.

Jeremy reminded me last week that too often we're slacking on updating
PEPs.  The generator expressions PEP for example still doesn't reflect
the current state of affairs.

Maybe someone needs to put their foot down and require that PEPs are
complete before features are checked in?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Mon Aug  2 20:50:12 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  2 20:50:21 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Mon, 02 Aug 2004 16:32:03 +1000."
	<410DDFE3.7010006@interlink.com.au> 
References: <410DDFE3.7010006@interlink.com.au> 
Message-ID: <200408021850.i72IoCg28650@guido.python.org>

Did the relative import syntax that was approved ever get checked in?
I thought it was on the list of things to land in 2.4?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From martin at v.loewis.de  Mon Aug  2 20:53:29 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug  2 20:53:30 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
Message-ID: <410E8DA9.5020304@v.loewis.de>

Phillip J. Eby wrote:
> But then why not just make that the default syntax, so that no migration 
> is necessary, and only one syntax has to be learned/explained to people?

For a very simple reason: The time for discussion is now over. There is 
little point in going over all the same arguments again and again. For 
this alpha release, no amount of arguing will change what has been 
committed. You really have to organize a public outcry if you want the
syntax changed before 2.4.

Regards,
Martin
From martin at v.loewis.de  Mon Aug  2 20:58:09 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug  2 20:58:08 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <isc1jwqm.fsf@yahoo.co.uk>
References: <410DDFE3.7010006@interlink.com.au> <isc1jwqm.fsf@yahoo.co.uk>
Message-ID: <410E8EC1.2070401@v.loewis.de>

Paul Moore wrote:

> Can someone have a look at http://www.python.org/sf/870382 ? I'm not
> at all sure how to address Martin's comments, and I'd like to see this
> in Python 2.4, in some form or other.

It slipped through my attention because it is a bug report, not a patch.
I will look at it again soon.

That said, I should continue to require that anybody desiring 
incorporation of a patch should review 10 other patches first.

Regards,
Martin
From guido at python.org  Mon Aug  2 21:09:17 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  2 21:09:25 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> 
References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> 
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> 
Message-ID: <200408021909.i72J9IZ28757@guido.python.org>

> >>I would think the fact that the '[decorators]' syntax can be implemented 
> >>in pure Python (no changes to the interpreter) for existing Python 
> >>versions would give more weight to it.

Can it?  I must've missed that.  It sure sounds like an incredible
hack -- how to you prevent the default behavior that the list of
decorators is thrown away by the interpreter?

> >>That is, if someone wants to implement a decorator that's forwards
> >>and backwards-compatible, that's possible with the list syntax,
> >>but not the @ syntax.
> >
> >.. but that also means you can still make the [decorators] syntax
> >work in 2.4, if you want compatibility or don't like @syntax.
> 
> But then why not just make that the default syntax, so that no
> migration is necessary, and only one syntax has to be
> learned/explained to people?

Because that syntax received significant boohs when I presented it at
EuroPython.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From jim at zope.com  Mon Aug  2 21:16:33 2004
From: jim at zope.com (Jim Fulton)
Date: Mon Aug  2 21:16:35 2004
Subject: [Python-Dev] Proposed Python api for testing whether a  module
	exists
In-Reply-To: <5.1.1.6.0.20040802125815.05c5fea0@mail.telecommunity.com>
References: <5.1.1.6.0.20040802125815.05c5fea0@mail.telecommunity.com>
Message-ID: <410E9311.2090201@zope.com>

Phillip J. Eby wrote:
> At 12:32 PM 8/2/04 -0400, Jim Fulton wrote:
> 
>> I propose to add the following function to the imp module:

...

> Is this intended to work in the presence of import hooks?  In 
> particular, all the hooks offered by PEP 302, but also systems like 
> 'iu', 'imputil', etc.
> 
> Or, more specifically, will this be implemented in terms of the PEP 302 
> importer/loader protocols, or will those protocols need extending in 
> order to allow such loaders (e.g. zipimport) to work with this new 
> function?

Paul Moore wrote:
...
 > Does this work in the presence of PEP 302 style import hooks?
 > Specifically, the case where there's a zipfile on sys.path.

Good question. I was planning to implement this in terms of
imp.find_module, but that won't work in light of PEP 302.  Dang.
It's too bad that the imp.get_loader proposed in PEP 302 isn't
implemented.

Paul, do you have an implementation of what would have been
imp.get_loader?

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From amk at amk.ca  Mon Aug  2 21:18:15 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Aug  2 21:18:18 2004
Subject: [Python-Dev] Bug day coming up this Saturday
Message-ID: <20040802191815.GA9528@rogue.amk.ca>

The third Python Bug Day is coming up this Saturday, August 7 2004.

Having more participation from people with CVS access would be very
helpful.  On the two previous bug days, there have been several
non-developers contributing but only one or two people who can
actually check in code or close bugs.  You don't need to hang around
all day; an hour or two is enough.

This will be the last bug day before October.  I won't be running a
bug day in September because I'm getting married on September 19th and
will need every spare weekend possible.  If someone else wants to
organize a September bug day, please do.  (The tasks are pretty
simple: updating the Wiki pages before the chosen date, and then
staying on IRC for the entire day, to ensure that there's
*someone* around to answer questions and commit things.)

When?
=====

This Saturday, August 7, 2004, from 9AM to 5PM EDT (1PM to 9PM GMT).
You don't need to be around all day; feel free to stop by for a few hours
and contribute.

Where?
======

The #python-dev channel on irc.freenode.net.

More information
================

For instructions and more information, see
    http://www.python.org/cgi-bin/moinmoin/PythonBugDay

--amk
From jim at zope.com  Mon Aug  2 21:19:08 2004
From: jim at zope.com (Jim Fulton)
Date: Mon Aug  2 21:19:11 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410DDFE3.7010006@interlink.com.au>
References: <410DDFE3.7010006@interlink.com.au>
Message-ID: <410E93AC.9050807@zope.com>

Anthony Baxter wrote:
> The @decorator patch has been landed on the trunk, after
> getting the go-ahead from the BDFL. I'll update PEP-0318
> with the final syntax in the next day or so.

In the mean time, can anyone give a pointer to where this was proposed
or discussed?

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Mon Aug  2 21:31:32 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  2 21:31:35 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <004101c47858$c8be8200$a2ae2c81@oemcomputer>
References: <004101c47858$c8be8200$a2ae2c81@oemcomputer>
Message-ID: <1f7befae04080212312a1df319@mail.gmail.com>

[Raymond Hettinger]
> Is anyone else seeing a stack overflow assertion error in test_pyclbr.py
> (WinMe, MSVC++6.0)?

test_pyclbr passes on WinXP, VC7.1, release and debug builds, current CVS.
From pf_moore at yahoo.co.uk  Mon Aug  2 21:35:18 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Mon Aug  2 21:35:04 2004
Subject: [Python-Dev] Re: Proposed Python api for testing whether a module
	exists
References: <5.1.1.6.0.20040802125815.05c5fea0@mail.telecommunity.com>
	<410E9311.2090201@zope.com>
Message-ID: <1xipjpu1.fsf@yahoo.co.uk>

Jim Fulton <jim@zope.com> writes:

> Phillip J. Eby wrote:
>> At 12:32 PM 8/2/04 -0400, Jim Fulton wrote:
>>
>>> I propose to add the following function to the imp module:
>
> ...
>
>> Is this intended to work in the presence of import hooks?  In
>> particular, all the hooks offered by PEP 302, but also systems like
>> 'iu', 'imputil', etc.
>> Or, more specifically, will this be implemented in terms of the PEP
>> 302 importer/loader protocols, or will those protocols need
>> extending in order to allow such loaders (e.g. zipimport) to work
>> with this new function?
>
> Paul Moore wrote:
> ...
>  > Does this work in the presence of PEP 302 style import hooks?
>  > Specifically, the case where there's a zipfile on sys.path.
>
> Good question. I was planning to implement this in terms of
> imp.find_module, but that won't work in light of PEP 302.  Dang.
> It's too bad that the imp.get_loader proposed in PEP 302 isn't
> implemented.
>
> Paul, do you have an implementation of what would have been
> imp.get_loader?

Sorry, no. Just did the implementation, I mainly did some of the
write-up, testing, and cheerleading.

Paul.
-- 
Ooh, how Gothic. Barring the milk.

From pje at telecommunity.com  Mon Aug  2 21:49:41 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug  2 21:45:50 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408021909.i72J9IZ28757@guido.python.org>
References: <Your message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>

At 12:09 PM 8/2/04 -0700, Guido van Rossum wrote:
> > >>I would think the fact that the '[decorators]' syntax can be implemented
> > >>in pure Python (no changes to the interpreter) for existing Python
> > >>versions would give more weight to it.
>
>Can it?  I must've missed that.  It sure sounds like an incredible
>hack -- how to you prevent the default behavior that the list of
>decorators is thrown away by the interpreter?

By using sys.settrace (and a careful tracer implementation to avoid 
interfering with debuggers or other active tracers):

def as(*decorators):

     if len(decorators)>1:
         decorators = list(decorators)
         decorators.reverse()

     def callback(frame,k,v):
         for d in decorators:
             v = d(v)
         frame.f_locals[k] = v

     add_assignment_advisor(callback)


def add_assignment_advisor(callback,depth=2):
     """Invoke 'callback(frame,name,value)' on the next assignment in 'frame'

     The frame monitored is determined by the 'depth' argument, which gets
     passed to 'sys._getframe()'.  Note that when 'callback' is invoked, the
     frame state will be as though the assignment hasn't happened yet, so any
     previous value of the assigned variable will be available in the frame's
     locals.  (Unless there's no previous value, in which case there will be
     no such variable in the frame locals.)
     """

     frame = sys._getframe(depth)
     oldtrace = [frame.f_trace]
     old_locals = frame.f_locals.copy()

     def tracer(frm,event,arg):
         if event=='call':
             # We don't want to trace into any calls
             if oldtrace[0]:
                 # ...but give the previous tracer a chance to, if it wants
                 return oldtrace[0](frm,event,arg)
             else:
                 return None

         try:
             if frm is frame and event !='exception':
                 # Aha, time to check for an assignment...
                 for k,v in frm.f_locals.items():
                     if k not in old_locals:
                         del frm.f_locals[k]
                         break
                     elif old_locals[k] is not v:
                         frm.f_locals[k] = old_locals[k]
                         break
                 else:
                     # No luck, keep tracing
                     return tracer

                 # Got it, fire the callback, then get the heck outta here...
                 callback(frm,k,v)

         finally:
             # Give the previous tracer a chance to run before we return
             if oldtrace[0]:
                 # And allow it to replace our idea of the "previous" tracer
                 oldtrace[0] = oldtrace[0](frm,event,arg)

         # Unlink ourselves from the trace chain.
         frm.f_trace = oldtrace[0]
         sys.settrace(oldtrace[0])
         return oldtrace[0]

     # Install the trace function
     frame.f_trace = tracer
     sys.settrace(tracer)



class X(object):
      [as(classmethod)]
      def foo(cls):
          ....


Of course, 'as' is a hack to let you use arbitrary callables as decorators, 
rather than having to use add_assignment_advisor directly.



> > >>That is, if someone wants to implement a decorator that's forwards
> > >>and backwards-compatible, that's possible with the list syntax,
> > >>but not the @ syntax.
> > >
> > >.. but that also means you can still make the [decorators] syntax
> > >work in 2.4, if you want compatibility or don't like @syntax.
> >
> > But then why not just make that the default syntax, so that no
> > migration is necessary, and only one syntax has to be
> > learned/explained to people?
>
>Because that syntax received significant boohs when I presented it at
>EuroPython.

And "@" *didn't*???  Ah well.  C'est la vie.

From tim.peters at gmail.com  Mon Aug  2 21:45:27 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  2 21:46:04 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <004101c47858$c8be8200$a2ae2c81@oemcomputer>
References: <004101c47858$c8be8200$a2ae2c81@oemcomputer>
Message-ID: <1f7befae0408021245312d3c80@mail.gmail.com>

[Raymond Hettinger]
> Is anyone else seeing a stack overflow assertion error in test_pyclbr.py
> (WinMe, MSVC++6.0)?

BTW, which version of Python are you talking about?  Current CVS can't
compile under MSVC 6.0 anymore, because PC/pyconfig.h #defines
HAVE_UINTPTR_T and HAVE_INTPTR_T now, which aren't correct for VC6,
and then pyport.h goes nuts -- virtually nothing compiles anymore
unless you use VC7.1.
From jim at zope.com  Mon Aug  2 21:55:10 2004
From: jim at zope.com (Jim Fulton)
Date: Mon Aug  2 21:55:14 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
References: <Your message of "Mon, 02 Aug 2004 13:30:03
	EDT."	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
Message-ID: <410E9C1E.1010604@zope.com>

Phillip J. Eby wrote:
> At 12:09 PM 8/2/04 -0700, Guido van Rossum wrote:
> 

...

>> Because that syntax received significant boohs when I presented it at
>> EuroPython.
> 
> 
> And "@" *didn't*???  Ah well.  C'est la vie.

I don't remember it being presented.  Maybe I missed it, or forgot.

Jim


-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From skip at pobox.com  Mon Aug  2 21:58:01 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug  2 21:58:04 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
References: <Your message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
Message-ID: <16654.40137.30612.114797@montanaro.dyndns.org>


    >> Can it?  I must've missed that.  It sure sounds like an incredible
    >> hack -- how to you prevent the default behavior that the list of
    >> decorators is thrown away by the interpreter?

    Phillip> By using sys.settrace (and a careful tracer implementation to
    Phillip> avoid interfering with debuggers or other active tracers):

I suspect Guido might class that as an "incredible hack". ;-)

Skip
From dave at boost-consulting.com  Mon Aug  2 22:07:29 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Mon Aug  2 22:08:01 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
References: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<200408021909.i72J9IZ28757@guido.python.org>
Message-ID: <uvfg1fgn2.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

>> >>I would think the fact that the '[decorators]' syntax can be implemented 
>> >>in pure Python (no changes to the interpreter) for existing Python 
>> >>versions would give more weight to it.
>
> Can it?  I must've missed that.  It sure sounds like an incredible
> hack -- how to you prevent the default behavior that the list of
> decorators is thrown away by the interpreter?

See http://article.gmane.org/gmane.comp.python.devel/60542

I'm really surprised you missed it, and was likewise disappointed
that it wasn't included in the weekly (?) summary.  It seemed like a
pretty significant development to me.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From foom at fuhm.net  Mon Aug  2 23:13:22 2004
From: foom at fuhm.net (James Y Knight)
Date: Mon Aug  2 23:13:23 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
Message-ID: <C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>

On Aug 2, 2004, at 12:55 PM, Phillip J. Eby wrote:
> I would think the fact that the '[decorators]' syntax can be 
> implemented in pure Python (no changes to the interpreter) for 
> existing Python versions would give more weight to it.  That is, if 
> someone wants to implement a decorator that's forwards and 
> backwards-compatible, that's possible with the list syntax, but not 
> the @ syntax.

First off, let me just say I'm happy *one* proposal was selected for 
inclusion. The @foo on the line before the function was not my 
favorite, but that's okay. It'll definitely work.

However, I do think [decorator...] on the line before the function 
definition would be a far worse choice than @decorator. Decorators 
*are* special new syntax. At least @foo *looks* like new syntax. 
[decorators] looks like a expression creating a list, but instead is 
completely magical. Magical syntax seems very poor to me.

About the only advantage it has is that you can implement it via a 
completely horrible hack in Python < 2.4. But, that is only a temporary 
situation. In a few years everyone will have Python >= 2.4 and we'd 
still be stuck with strange weird syntax that doesn't look like syntax.

The one thing I do dislike about the new addition is the arbitrary (?) 
restriction on expression allowed in a decorator. From Grammar..
   decorator: '@' dotted_name [ '(' [arglist] ')' ]
That seems quite odd to me. Is there some reason it couldn't be:
   decorator: '@' test
?

I don't see how that would cause a problem, but maybe I've missed 
something.


Just testing out how this syntax looks, you can ignore the rest of this 
mail. :)

   @classmethod
   @arguments(string, int)
   @returns(list)
   def foo(s, i):
     return [s]*i

or perhaps the same, but with a list allowed:
   @classmethod, arguments(string, int), returns(list)
   def foo(s, i):
     return [s]*i

or original proposal:
   def foo(s, i) [classmethod, arguments(string, int), returns(list)]:
     return [s]*i

Ah well, they all have their advantages. :) This one is workable.

James

From goodger at python.org  Mon Aug  2 23:40:26 2004
From: goodger at python.org (David Goodger)
Date: Mon Aug  2 23:40:48 2004
Subject: [Python-Dev] Re: ConfigParser behavior change
In-Reply-To: <200408021033.03830.fdrake@acm.org>
References: <410E4C10.3080009@python.org> <200408021033.03830.fdrake@acm.org>
Message-ID: <410EB4CA.1030900@python.org>

Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040802/5b55cd2f/signature.pgp
From python at rcn.com  Mon Aug  2 14:10:04 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug  3 02:11:44 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <1f7befae0408021245312d3c80@mail.gmail.com>
Message-ID: <006201c47889$a5554ca0$a2ae2c81@oemcomputer>

> [Raymond Hettinger]
> > Is anyone else seeing a stack overflow assertion error in
test_pyclbr.py
> > (WinMe, MSVC++6.0)?
> 
> BTW, which version of Python are you talking about?  Current CVS can't
> compile under MSVC 6.0 anymore, because PC/pyconfig.h #defines
> HAVE_UINTPTR_T and HAVE_INTPTR_T now, which aren't correct for VC6,
> and then pyport.h goes nuts -- virtually nothing compiles anymore
> unless you use VC7.1.

I'm still with 6.0.  Had to hack a couple lines of pyport.h and
socketmodule.c which worked fine until yesterday.  

Too bad about 6.0.  I had thought the plan was to support both 6.0 and
7.1.  I'm not sure what we gained by dropping 6.0 right at the end :-(


Raymond

From tim.peters at gmail.com  Tue Aug  3 02:18:57 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug  3 02:18:59 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <006201c47889$a5554ca0$a2ae2c81@oemcomputer>
References: <006201c47889$a5554ca0$a2ae2c81@oemcomputer>
Message-ID: <1f7befae040802171858ab42d3@mail.gmail.com>

[Raymond Hettinger]
> Too bad about 6.0.  I had thought the plan was to support both 6.0 and
> 7.1.  I'm not sure what we gained by dropping 6.0 right at the end :-(

There's no plan to support MSVC 6.0.  Martin routinely checks things
in that break it, and that's fine.  The only reason it *has* worked
until recently is that people sometimes volunteer to patch up the 6.0
part of the tree.  I gave some days to that at the start, because I
didn't have a machine at the time that *could* run 7.1.  Now I do, so
I pay a lot less attention to 6.0 (and Win98SE) now.

IOW, 6.0 never had, and still doesn't have, a champion (== someone
willing to keep it working) for 2.4.

I would have volunteered, but your report of a stack overflow in
test_pyclbr scared me away <wink>.
From guido at python.org  Tue Aug  3 02:58:28 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 02:58:36 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Mon, 02 Aug 2004 15:49:41 EDT."
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> 
References: <Your message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com> 
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com> 
Message-ID: <200408030058.i730wSQ29267@guido.python.org>

> >Can it?  I must've missed that.  It sure sounds like an incredible
> >hack -- how to you prevent the default behavior that the list of
> >decorators is thrown away by the interpreter?
> 
> By using sys.settrace (and a careful tracer implementation to avoid 
> interfering with debuggers or other active tracers):

Ah, yuck.  Not an acceptable solution.  And it doesn't let you write

  [classmethod]

-- you have to wrap the 'classmethod' in something relatively ugly.

But let's move on.

> >Because that syntax received significant boohs when I presented it at
> >EuroPython.
> 
> And "@" *didn't*???  Ah well.  C'est la vie.

Indeed it didn't.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug  3 03:05:15 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 03:05:22 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Mon, 02 Aug 2004 17:13:22 EDT."
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net> 
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com> 
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net> 
Message-ID: <200408030105.i7315Fj29310@guido.python.org>

> The one thing I do dislike about the new addition is the arbitrary (?) 
> restriction on expression allowed in a decorator. From Grammar..
>    decorator: '@' dotted_name [ '(' [arglist] ')' ]
> That seems quite odd to me. Is there some reason it couldn't be:
>    decorator: '@' test
> ?

The first patch on SF actually had '@' test, and I requested that it
be changed.  Most things that are 'test' but not 'dotted_name'
optionally followed by an argument list don't make sense as
decorators; if you really have a need to write

    @ foo or bar
    def f(): ...

you can write

    deco = foo or bar
    @deco
    def f(): ...

--Guido van Rossum (home page: http://www.python.org/~guido/)
From greg at cosc.canterbury.ac.nz  Tue Aug  3 03:42:56 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug  3 03:43:01 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <1091458891.9134.29.camel@localhost>
Message-ID: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz>

> I think Anthony's checkin message was accurate enough though -- it's
> the syntax everyone can hate equally.  But I'm glad /something/ made
> it in!

Well, I'm not. It seems like a decision made in haste.

Maybe I'll get used to it in a few decades, but right
now I feel very disappointed. We were *so* close to
agreeing on something nice, and then for some reason
it all went pear-shaped. :-(

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From skip at pobox.com  Tue Aug  3 04:01:34 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug  3 04:01:36 2004
Subject: [Python-Dev] @decorators and PEP 318
In-Reply-To: <200408030105.i7315Fj29310@guido.python.org>
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
Message-ID: <16654.61950.278345.835840@montanaro.dyndns.org>


I'll be happy to update PEP 318 to conform to the pie decorator syntax.  Can
someone describe the application order?  If I have

    @ decorator1
    @ decorator2
    def func(...):
        ...

Which decorator is applied first to the function object, decorator1 or
decorator2?

Thx,

Skip
From greg at cosc.canterbury.ac.nz  Tue Aug  3 04:01:35 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug  3 04:01:46 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <410E02E6.9000205@stackless.com>
Message-ID: <200408030201.i7321ZRo028011@cosc353.cosc.canterbury.ac.nz>

> > Maybe it could be as simple as saving a snapshot of
> > sys.modules whenever importing of a module is begun,
> > and if execution of its body doesn't complete, restoring
> > the snapshot?
> 
> This is exactly what I thought of.
> It seems to be correct, also with nested imports,
> since this is a stack-like structure of undoable things.

What happens in the presence of threads, though?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From tismer at stackless.com  Tue Aug  3 04:16:16 2004
From: tismer at stackless.com (Christian Tismer)
Date: Tue Aug  3 04:13:50 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200408021840.i72Iejt28600@guido.python.org>
References: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz>
	<200408021840.i72Iejt28600@guido.python.org>
Message-ID: <410EF570.3010901@stackless.com>

Guido van Rossum wrote:

>>>>Circular imports, to be made consistent, would need
>>>>something like a final "commit" after all imports succeeded.
>>>
>>>Silly idea.  Typically, circular imports aren't planned, they just
>>>happen, and (usually) just happen to work.
>>
>>I thought Christian meant this was something that would
>>be done automatically, not something the user would have
>>to do explicitly.
> 
> OK, I misunderstood then.  Apologies to Christian.

I love you for that one.

>>Maybe it could be as simple as saving a snapshot of
>>sys.modules whenever importing of a module is begun,
>>and if execution of its body doesn't complete, restoring
>>the snapshot?
> 
> That sounds like overkill -- any modules that don't participate in the
> cycle but happen to be imported by one of the modules in the cycle
> would also be deleted.

Yes, but that is ok, methinks. Any module that gets imported just
on behalf of another module that *fails* to get imported, was
just imported on behalf of that module, *not* by the user.
And it is the user, finally, who has the say.
He has to accept the extra imports of the successful imports,
but by now means he is responsible of the side effects of
imported code that he didn't write.

If the user had imported that module before, he will still have it.
If he had not, he also will not expect to have it now.
I think it is just fine to remove what he didn't expect.

The user says "I import that module now, and all what it needs".
Nothing more, and nothing less. I know what I'm talking of,
having paid for foreign kids for half of my life :-)

> I still expect that the solution that Tim just checked in (delete any
> module that failed to complete execution) will solve most of the
> problems encountered in real life.  I'm aware of the case you describe
> here:

Yes, but I think you are almost an atom's distance apart off of
the final solution. Grab it!

ciao -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Tue Aug  3 04:21:14 2004
From: tismer at stackless.com (Christian Tismer)
Date: Tue Aug  3 04:19:03 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200408030201.i7321ZRo028011@cosc353.cosc.canterbury.ac.nz>
References: <200408030201.i7321ZRo028011@cosc353.cosc.canterbury.ac.nz>
Message-ID: <410EF69A.6050306@stackless.com>

Greg Ewing wrote:

>>>Maybe it could be as simple as saving a snapshot of
>>>sys.modules whenever importing of a module is begun,
>>>and if execution of its body doesn't complete, restoring
>>>the snapshot?
>>
>>This is exactly what I thought of.
>>It seems to be correct, also with nested imports,
>>since this is a stack-like structure of undoable things.
> 
> 
> What happens in the presence of threads, though?

To my memory, imports in the presense of threads *always*
have been a nightmare. My advice always goues this way:
If you must have threads, make sure to import everything
through the main thread, before starting new ones.

If you then still have problems, well, I have a different
solution, which is also more efficient,...
... but this is another story. Off topic.

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From anthony at interlink.com.au  Tue Aug  3 04:26:45 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug  3 04:28:02 2004
Subject: [Python-Dev] @decorators and PEP 318
In-Reply-To: <16654.61950.278345.835840@montanaro.dyndns.org>
References: <1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>	<200408030105.i7315Fj29310@guido.python.org>
	<16654.61950.278345.835840@montanaro.dyndns.org>
Message-ID: <410EF7E5.5000200@interlink.com.au>

Skip Montanaro wrote:
> I'll be happy to update PEP 318 to conform to the pie decorator syntax.  Can
> someone describe the application order?  If I have
> 
>     @ decorator1
>     @ decorator2
>     def func(...):
>         ...
> 
> Which decorator is applied first to the function object, decorator1 or
> decorator2?

func = decorator1(decorator2(func))

I hope to check in a revised 318 this afternoon.

Anthony
From anthony at interlink.com.au  Tue Aug  3 04:30:19 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug  3 04:30:47 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz>
References: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz>
Message-ID: <410EF8BB.7030605@interlink.com.au>

Greg Ewing wrote:
> Well, I'm not. It seems like a decision made in haste.

> Maybe I'll get used to it in a few decades, but right
> now I feel very disappointed. We were *so* close to
> agreeing on something nice, and then for some reason
> it all went pear-shaped. :-(

PEP 318 is now nearly 14 months old. There's been a truly
staggering number of posts on the subject - and every
time it's raised, the same points come up. The only things
to have happened recently were the new @ syntax, and PJE's
settrace hack.

If not now, when? This could drag on for another 14 months,
otherwise - and I really don't want to hold up 2.4 final
for that long.

Anthony


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From anthony at interlink.com.au  Tue Aug  3 04:33:07 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug  3 04:33:23 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408021850.i72IoCg28650@guido.python.org>
References: <410DDFE3.7010006@interlink.com.au>
	<200408021850.i72IoCg28650@guido.python.org>
Message-ID: <410EF963.40704@interlink.com.au>

Guido van Rossum wrote:
> Did the relative import syntax that was approved ever get checked in?
> I thought it was on the list of things to land in 2.4?

Nope. The 2.4 release schedule PEP now lists this as lacking someone
to drive it. I won't/can't spend time on it. So, if one of the people
who wants it could step forward, that'd be excellent. Even if it's
just to do the multi-line import syntax of

  from foo import ( bar, baz, bar2,
                    baz2, bar3, baz3 )



-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From skip at pobox.com  Tue Aug  3 04:46:11 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug  3 04:46:32 2004
Subject: [Python-Dev] @decorators and PEP 318
In-Reply-To: <410EF7E5.5000200@interlink.com.au>
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
	<16654.61950.278345.835840@montanaro.dyndns.org>
	<410EF7E5.5000200@interlink.com.au>
Message-ID: <16654.64627.453369.967660@montanaro.dyndns.org>

    Anthony> I hope to check in a revised 318 this afternoon.

Cool.  Let me know if you want any help/input/proofing/whatever...

Skip

From tim.peters at gmail.com  Tue Aug  3 05:14:12 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug  3 05:14:14 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <410EF570.3010901@stackless.com>
References: <200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz>
	<200408021840.i72Iejt28600@guido.python.org>
	<410EF570.3010901@stackless.com>
Message-ID: <1f7befae040802201470e3d4c@mail.gmail.com>

[Christian Tismer]
...
> Yes, but I think you are almost an atom's distance apart off of
> the final solution. Grab it!

If you mean this:

>>> Maybe it could be as simple as saving a snapshot of
>>> sys.modules whenever importing of a module is begun,
>>> and if execution of its body doesn't complete, restoring
>>> the snapshot?

then no, that's a long way off in CPython.  There's no choke point for
when importing begins, or for when importing ends, and even
__builtin__.__import__ is routinely replaced.  Guido latched on to the
only choke point there is:  sooner or later, every import gimmick
worthy of the name has to execute "the module's" code, whether it be
direct import from Python, directly via C API calls, implicit package
__init__.py imports, imports arranged via magical importers (like the
.zip importer), etc.  So that's what the patch targeted:  there's one
routine that executes a module's initialization code, all imports go
thru it eventually, and that's the routine that now removes the
module's name from sys.modules if the initialization code barfs.

"The rest" of import logic is sprayed all over creation.  To hook
"begin import" and "end import" first requires that all importers be
changed to have a formal notion of those events.  The easiest start
would be to use bytecodehacks to inject "call begin_import" and "call
end_import" opcode sequences around the code generated for import
statements <0.8 wink>.
From pje at telecommunity.com  Tue Aug  3 05:22:04 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug  3 05:18:04 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae040802201470e3d4c@mail.gmail.com>
References: <410EF570.3010901@stackless.com>
	<200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz>
	<200408021840.i72Iejt28600@guido.python.org>
	<410EF570.3010901@stackless.com>
Message-ID: <5.1.1.6.0.20040802231932.0318c6a0@mail.telecommunity.com>

At 11:14 PM 8/2/04 -0400, Tim Peters wrote:
>[Christian Tismer]
>..
> > Yes, but I think you are almost an atom's distance apart off of
> > the final solution. Grab it!
>
>If you mean this:
>
> >>> Maybe it could be as simple as saving a snapshot of
> >>> sys.modules whenever importing of a module is begun,
> >>> and if execution of its body doesn't complete, restoring
> >>> the snapshot?
>
>then no, that's a long way off in CPython.  There's no choke point for
>when importing begins, or for when importing ends, and even
>__builtin__.__import__ is routinely replaced.  Guido latched on to the
>only choke point there is:  sooner or later, every import gimmick
>worthy of the name has to execute "the module's" code, whether it be
>direct import from Python, directly via C API calls, implicit package
>__init__.py imports, imports arranged via magical importers (like the
>.zip importer), etc.  So that's what the patch targeted:  there's one
>routine that executes a module's initialization code, all imports go
>thru it eventually, and that's the routine that now removes the
>module's name from sys.modules if the initialization code barfs.

It's worse than that...  'ihooks' and 'imputil', for example, both do:

     exec code in module.__dict__

so they'd need to be changed to support this fix as well.  (Not to mention 
any third-party import hooks...)

From tim.peters at gmail.com  Tue Aug  3 05:27:59 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug  3 05:28:01 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <5.1.1.6.0.20040802231932.0318c6a0@mail.telecommunity.com>
References: <410EF570.3010901@stackless.com>
	<200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz>
	<200408021840.i72Iejt28600@guido.python.org>
	<410EF570.3010901@stackless.com>
	<5.1.1.6.0.20040802231932.0318c6a0@mail.telecommunity.com>
Message-ID: <1f7befae0408022027465a8789@mail.gmail.com>

[Phillip J. Eby]
> It's worse than that...  'ihooks' and 'imputil', for example, both do:
> 
>     exec code in module.__dict__
> 
> so they'd need to be changed to support this fix as well.  (Not to mention
> any third-party import hooks...)

Ouch.  For that matter, they also avoid the "choke point" Sunday
night's patch relied on.  So there are actually no choke points for
any part of the import process now.  Maybe after each bytecode we
could make the eval loop look to see whether sys.modules had been
changed <wink>.
From guido at python.org  Tue Aug  3 05:51:38 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 05:51:47 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Tue, 03 Aug 2004 13:42:56 +1200."
	<200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz> 
References: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408030351.i733pcO29560@guido.python.org>

> Well, I'm not. It seems like a decision made in haste.

Hardly.

> Maybe I'll get used to it in a few decades, but right
> now I feel very disappointed. We were *so* close to
> agreeing on something nice, and then for some reason
> it all went pear-shaped. :-(

Not really.  The contenders were:

- [decorators] after the arguments.  I really hate this, for good
  reasons which I won't repeat here.

- [decorators] in front of the 'def'.  For a while I really loved
  this, but several killer arguments were made against it, most having
  to do with the ambiguity, which confuses beginners.  (A couple of
  people in the audience of my EuroPython keynote stood up and argued
  this with passion.)

- Any number of variations on [decorators] before 'def', adding
  arbitrary mark-up to make it unambiguous.  Including the 'as'
  keyword (which it isn't, by the way).  All of which strike me as
  totally arbitrary.  The 'as' keyword has a history of uses for
  renaming (e.g. in SQL and Python's import) but decorators are
  everything but renaming.

- @decorators in front of 'def'.  This is borrowed from Java (just
  like [decorators] was borrowed from C#) so minimizes invention.  It
  isn't ambiguous, and as long as we're going to use arbitrary syntax
  we might as well use something that's familiar to at least one huge
  group of potential Python users.

And yes, I'm not 100% comfortable with it either.  But I know one
thing: in this particular case, now is better than never.  So I'm
picking a solution everybody can hate equally.

(What's wrong with pear-shaped, anyway?  It's one of my favorite
shapes. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From foom at fuhm.net  Tue Aug  3 06:17:43 2004
From: foom at fuhm.net (James Y Knight)
Date: Tue Aug  3 06:17:47 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408030105.i7315Fj29310@guido.python.org>
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
Message-ID: <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>

On Aug 2, 2004, at 9:05 PM, Guido van Rossum wrote:
> The first patch on SF actually had '@' test, and I requested that it
> be changed.  Most things that are 'test' but not 'dotted_name'
> optionally followed by an argument list don't make sense as
> decorators; if you really have a need to write
>
>     @ foo or bar
>     def f(): ...
>
> you can write
>
>     deco = foo or bar
>     @deco
>     def f(): ...

An even better workaround is to write:
   def d(arg): return arg

   @d(foo or bar)
   def f(): ...

However, it seems as if this restriction creates a new class of 
expression for no gain. It is true enough that *most* python 
expressions aren't useful to write after a @, but that's also true of a 
lot of other places an expression can be used (e.g. before a () of a 
function call. A list comprehension can never result in a callable 
object. An arithmetic operation usually won't.).

The only real necessary restriction on the @ operator is that its 
argument be callable and take a single argument. Many expressions could 
return a callable object. Why not let them? Is it really worth having a 
special case just to SyntaxError expressions that sometimes won't 
result in an appropriate callable?

Things someone might want to do, ordered roughly from most reasonable 
to least reasonable ;)
@foo().bar()
@foo or bar
@mydecorators['foo']
@lambda f: foo(f) or bar(f)

Why disallow these forms? It seems quite difficult, especially, to 
explain why the first one does not, or should not, work.

James

From neal at metaslash.com  Tue Aug  3 06:20:18 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Tue Aug  3 06:20:21 2004
Subject: [Python-Dev] refleak hunting season
In-Reply-To: <2mpt69bo47.fsf@starship.python.net>
References: <2mpt69bo47.fsf@starship.python.net>
Message-ID: <20040803042017.GN7708@epoch.metaslash.com>

On Mon, Aug 02, 2004 at 03:39:36PM +0100, Michael Hudson wrote:
> 
> test_sax leaked [1899, 1899, 1899, 1899] references
>     no idea -- scary numbers, though!

It looks like the worst (only?) leakers are:

        test_expat_incomplete
        test_expat_inpsource_location

Both of these cause exceptions to be raised.  I don't know the pyexpat
code at all.  But I think I fixed a few leaks with the attached patch.
Some leaks were caught by the tests, but I doubt all were.  The patch
is a starting point if anyone else wants to give it a shot.  If not,
I'll add it to SF or review carefully and check in later.

Neal
-------------- next part --------------
Index: Modules/pyexpat.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/pyexpat.c,v
retrieving revision 2.84
diff -u -w -r2.84 pyexpat.c
--- Modules/pyexpat.c	12 Oct 2003 19:09:37 -0000	2.84
+++ Modules/pyexpat.c	3 Aug 2004 04:14:27 -0000
@@ -104,12 +104,12 @@
 set_error_attr(PyObject *err, char *name, int value)
 {
     PyObject *v = PyInt_FromLong(value);
+    int rc = 0;
 
-    if (v != NULL && PyObject_SetAttrString(err, name, v) == -1) {
-        Py_DECREF(v);
-        return 0;
-    }
-    return 1;
+    if (v != NULL)
+        rc = PyObject_SetAttrString(err, name, v) != -1;
+    Py_XDECREF(v);
+    return rc;
 }
 
 /* Build and set an Expat exception, including positioning
@@ -359,7 +359,7 @@
 {
     PyThreadState *tstate = PyThreadState_GET();
     PyFrameObject *f;
-    PyObject *res;
+    PyObject *res = NULL;
 
     if (c == NULL)
         return NULL;
@@ -370,7 +370,7 @@
     tstate->frame = f;
 #ifdef FIX_TRACE
     if (trace_frame(tstate, f, PyTrace_CALL, Py_None) < 0) {
-	return NULL;
+	goto cleanup;
     }
 #endif
     res = PyEval_CallObject(func, args);
@@ -379,7 +379,7 @@
 	    PyTraceBack_Here(f);
 #ifdef FIX_TRACE
 	if (trace_frame_exc(tstate, f) < 0) {
-	    return NULL;
+	    goto cleanup;
 	}
     }
     else {
@@ -391,6 +391,7 @@
 #else
     }
 #endif
+cleanup:
     tstate->frame = f->f_back;
     Py_DECREF(f);
     return res;
@@ -996,6 +997,7 @@
         if (fp) {
             bytes_read = fread(buf, sizeof(char), BUF_SIZE, fp);
             if (bytes_read < 0) {
+                Py_XDECREF(readmethod);
                 PyErr_SetFromErrno(PyExc_IOError);
                 return NULL;
             }
From eppstein at ics.uci.edu  Tue Aug  3 06:28:52 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Tue Aug  3 06:28:56 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
Message-ID: <eppstein-CA7893.21285202082004@sea.gmane.org>

In article <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>,
 James Y Knight <foom@fuhm.net> wrote:

> >     @ foo or bar
> >     def f(): ...
> >
> > you can write
> >
> >     deco = foo or bar
> >     @deco
> >     def f(): ...
> 
> An even better workaround is to write:
>    def d(arg): return arg
> 
>    @d(foo or bar)
>    def f(): ...
> 
> However, it seems as if this restriction creates a new class of 
> expression for no gain. It is true enough that *most* python 
> expressions aren't useful to write after a @, but that's also true of a 
> lot of other places an expression can be used (e.g. before a () of a 
> function call. A list comprehension can never result in a callable 
> object. An arithmetic operation usually won't.).

As you say, there aren't a lot of types of Python expression that return 
callables.  The main thing this restriction seems to prevent is @lambda,
and I think preventing that is a good thing.  The other possibility that 
comes to mind is @functable[index], but I'd have to see a use case 
before worrying too much that it's not allowed.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From exarkun at divmod.com  Tue Aug  3 06:50:56 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Tue Aug  3 06:51:00 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
References: <1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>	<200408030105.i7315Fj29310@guido.python.org>
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
Message-ID: <410F19B0.2090809@divmod.com>

James Y Knight wrote:
> 
> [snip]
> 
> 
> An even better workaround is to write:
>   def d(arg): return arg
> 
>   @d(foo or bar)
>   def f(): ...
> 
> However, it seems as if this restriction creates a new class of 
> expression for no gain. It is true enough that *most* python expressions 
> aren't useful to write after a @, but that's also true of a lot of other 
> places an expression can be used (e.g. before a () of a function call. A 
> list comprehension can never result in a callable object. An arithmetic 
> operation usually won't.).
> 
> The only real necessary restriction on the @ operator is that its 
> argument be callable and take a single argument. Many expressions could 
> return a callable object. Why not let them? Is it really worth having a 
> special case just to SyntaxError expressions that sometimes won't result 
> in an appropriate callable?
> 
> Things someone might want to do, ordered roughly from most reasonable to 
> least reasonable ;)
> @foo().bar()
> @foo or bar
> @mydecorators['foo']
> @lambda f: foo(f) or bar(f)
> 
> Why disallow these forms? It seems quite difficult, especially, to 
> explain why the first one does not, or should not, work.
> 
> James
> 

   For what it's worth, I agree with James completely.

   Jp
From python at rcn.com  Mon Aug  2 18:54:35 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug  3 06:56:12 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410F19B0.2090809@divmod.com>
Message-ID: <002201c478b1$6461ebe0$eb23c797@oemcomputer>

> > The only real necessary restriction on the @ operator is that its
> > argument be callable and take a single argument. Many expressions
could
> > return a callable object. Why not let them? Is it really worth
having a
> > special case just to SyntaxError expressions that sometimes won't
result
> > in an appropriate callable?
> >
> > Things someone might want to do, ordered roughly from most
reasonable to
> > least reasonable ;)
> > @foo().bar()
> > @foo or bar
> > @mydecorators['foo']
> > @lambda f: foo(f) or bar(f)
> >
> > Why disallow these forms? It seems quite difficult, especially, to
> > explain why the first one does not, or should not, work.
> >
> > James
> >
> 
>    For what it's worth, I agree with James completely.
> 
>    Jp

I concur.


Raymond

From python at rcn.com  Mon Aug  2 19:11:49 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug  3 07:13:25 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects
	listobject.c, 2.218, 2.219
In-Reply-To: <E1BrrIq-0002nF-2x@sc8-pr-cvs1.sourceforge.net>
Message-ID: <002b01c478b3$cc9c7020$eb23c797@oemcomputer>

>   	 * complain about the unused name.
>   	 */
> ! 	return status, v;
>   }
> 
> --- 863,867 ----
>   	 * complain about the unused name.
>   	 */
> ! 	return status++, v;

Does anyone else see this as perverse?  I have to think hard about
whether all the compilers are smart enough to optimize this away
(probably, but maybe not).  More importantly, it is obfuscated C and
makes my stomach churn.  This particular case introduces several lines
for an assertion that is probably not worth the surrounding mess.  Does
anyone have a better way?


Raymond

From bob at redivi.com  Tue Aug  3 07:28:00 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Aug  3 07:28:02 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <002201c478b1$6461ebe0$eb23c797@oemcomputer>
References: <002201c478b1$6461ebe0$eb23c797@oemcomputer>
Message-ID: <E2CC0B32-E50D-11D8-AA20-000A95686CD8@redivi.com>


On Aug 2, 2004, at 12:54 PM, Raymond Hettinger wrote:

>>> The only real necessary restriction on the @ operator is that its
>>> argument be callable and take a single argument. Many expressions
> could
>>> return a callable object. Why not let them? Is it really worth
> having a
>>> special case just to SyntaxError expressions that sometimes won't
> result
>>> in an appropriate callable?
>>>
>>> Things someone might want to do, ordered roughly from most
> reasonable to
>>> least reasonable ;)
>>> @foo().bar()
>>> @foo or bar
>>> @mydecorators['foo']
>>> @lambda f: foo(f) or bar(f)
>>>
>>> Why disallow these forms? It seems quite difficult, especially, to
>>> explain why the first one does not, or should not, work.
>>>
>>> James
>>>
>>
>>    For what it's worth, I agree with James completely.
>>
>>    Jp
>
> I concur.
>
>
> Raymond

Ditto.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3589 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040803/9809f219/smime.bin
From exarkun at divmod.com  Tue Aug  3 07:36:19 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Tue Aug  3 07:36:25 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408030058.i730wSQ29267@guido.python.org>
References: <Your message of "Mon, 02 Aug 2004 13:30:03
	EDT."	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
Message-ID: <410F2453.7000506@divmod.com>

Guido van Rossum wrote:
>>>Can it?  I must've missed that.  It sure sounds like an incredible
>>>hack -- how to you prevent the default behavior that the list of
>>>decorators is thrown away by the interpreter?
>>
>>By using sys.settrace (and a careful tracer implementation to avoid 
>>interfering with debuggers or other active tracers):
> 
> 
> Ah, yuck.  Not an acceptable solution.  And it doesn't let you write
> 
>   [classmethod]
> 
> -- you have to wrap the 'classmethod' in something relatively ugly.
> 

     Here's a brief test for a syntax-change-less implementation of this 
feature, not as complete as test_decorators, but a good start, I believe:

def test():
     try:
         from test import test_decorators
     except ImportError:
         test_decorators = None

     class DecoratorTest(object):
         __metaclass__ = DecoratableType

         def foo(self):
             print 'undecorated foo'

         decorate(staticmethod)
         def bar(x):
             print x

         decorate(classmethod)
         def baz(cls, y):
             print cls, y

         if test_decorators:
             counts = {}
             decorate(test_decorators.countcalls(counts),
                      test_decorators.memoize)
             def quux(self):
                 print 'quux called'

     o = DecoratorTest()
     o.foo()
     o.bar('static method on instance object')
     o.baz('class method on the instance')
     DecoratorTest.bar('static method on the class')
     DecoratorTest.baz('class method on the class')

     if test_decorators:
         print 'Calling quux once'
         o.quux()
         print 'Calling quux twice'
         o.quux()
         print 'Called quux', DecoratorTest.counts['quux'], 'times'

     And here's the implementation, without using settrace, and without 
requiring wrappers for individual decorators:

import inspect

MAGIC_NAME = '__internal_decorators_list__'

class Decorator(object):
     def __init__(self, firstCallable, *callables):
         cf = inspect.currentframe()
         self.callables = [firstCallable] + list(callables)
         self.decoratesLine = cf.f_back.f_lineno + 1
         cf.f_back.f_locals.setdefault(MAGIC_NAME, []).append(self)

     def __call__(self, f):
         i = iter(self.callables)
         f = i.next()(f)
         for c in i:
             f = c(f)
         return f
decorate = Decorator

class DecoratableType(type):
     def __new__(cls, name, bases, attrs):
         decorators = attrs.get(MAGIC_NAME, [])
         if decorators:
             del attrs[MAGIC_NAME]
             lines = {}
             for (k, v) in attrs.items():
                 try:
                     source, lineno = inspect.getsourcelines(v)
                 except:
                     pass
                 else:
                     lines[lineno] = k
             for d in decorators:
                 if d.decoratesLine in lines:
                     k = lines[d.decoratesLine]
                     attrs[k] = d(attrs[k])
         return super(DecoratableType, cls).__new__(
             cls, name, bases, attrs)

     There are clear drawbacks to this approach.  Metaclass required, no 
obvious ways to support free functions, and it depends _slightly_ 
hackishly on the inspect module.  I think at least one of these problems 
  can be solved (and line number handling can be made smarter to deal 
with intervening comments, whitespace, etc).

     What are the advantages?

   * It works with Python 2.2 and Python 2.3.

   * It requires no interpreter changes.  It can be distributed with the 
standard library, as a cookbook recipe, or in the official documentation 
as a hint regarding more "advanced" decorator usage.

   * It introduces no new syntax and uses up no operator character.

   * It supports arbitrary expressions.

   * It's pure python.

   I realize there is little or no chance of '@decorator' being pulled 
from 2.4a2.  I hope that something along the lines of the above will be 
considered, instead, for the next alpha, unless there is widespread 
community support for '@decorator', as opposed to the ridiculously faint 
support ("it's better than nothing") currently behind it.

   Jean-Paul Calderone
From martin at v.loewis.de  Tue Aug  3 08:00:57 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug  3 08:00:57 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <006201c47889$a5554ca0$a2ae2c81@oemcomputer>
References: <006201c47889$a5554ca0$a2ae2c81@oemcomputer>
Message-ID: <410F2A19.1020209@v.loewis.de>

Raymond Hettinger wrote:
> I'm still with 6.0.  Had to hack a couple lines of pyport.h and
> socketmodule.c which worked fine until yesterday.  
> 
> Too bad about 6.0.  I had thought the plan was to support both 6.0 and
> 7.1.  I'm not sure what we gained by dropping 6.0 right at the end :-(

Someone please read the comment in PC/pyconfig.h:

/* Atleast VC 7.1 has them. If some compiler does not provide them,
    #ifdef appropriately .*/
#define HAVE_UINTPTR_T 1
#define HAVE_INTPTR_T 1

I did not bother checking whether VC6 has uintptr_t - that does not
mean that somebody else couldn't.

Hacking a couple of lines is probably the wrong approach. Properly
maintaining the VC6 port (i.e. sharing the well-designed changes
to a couple of lines) would be a good idea.

Regards,
Martin
From fdrake at acm.org  Tue Aug  3 08:06:48 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Aug  3 08:07:02 2004
Subject: [Python-Dev] refleak hunting season
In-Reply-To: <20040803042017.GN7708@epoch.metaslash.com>
References: <2mpt69bo47.fsf@starship.python.net>
	<20040803042017.GN7708@epoch.metaslash.com>
Message-ID: <200408030206.48728.fdrake@acm.org>

On Tuesday 03 August 2004 12:20 am, Neal Norwitz wrote:
 > It looks like the worst (only?) leakers are:
 >
 >         test_expat_incomplete
 >         test_expat_inpsource_location
 >
 > Both of these cause exceptions to be raised.  I don't know the pyexpat
 > code at all.  But I think I fixed a few leaks with the attached patch.
 > Some leaks were caught by the tests, but I doubt all were.  The patch
 > is a starting point if anyone else wants to give it a shot.  If not,
 > I'll add it to SF or review carefully and check in later.

Please post to SF and assign to me; I'll take a look sometime tomorrow.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From fperez528 at yahoo.com  Tue Aug  3 08:15:16 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Tue Aug  3 08:15:17 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
References: <5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<410F2453.7000506@divmod.com>
Message-ID: <cenahf$ks9$1@sea.gmane.org>

Jp Calderone wrote:

>      Here's a brief test for a syntax-change-less implementation of this
> feature, not as complete as test_decorators, but a good start, I believe:

[...]

> from 2.4a2.  I hope that something along the lines of the above will be
> considered, instead, for the next alpha, unless there is widespread
> community support for '@decorator', as opposed to the ridiculously faint
> support ("it's better than nothing") currently behind it.

FWIW, I'd be very much +1 on something like this, which feels much more
'pythonic' and elegant to me than the @ syntax.  I know I'm biased because the
@ syntax will force me to change ipython and it will break current ipython for
anyone wanting to use 2.4, but I'm honestly trying to assess it beyond ipython. 
And I still don't like it, while JP's solution seems nice and clean to me.

Cheers,

f

From djc at object-craft.com.au  Tue Aug  3 08:44:53 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Tue Aug  3 08:45:03 2004
Subject: [Python-Dev] Tuple/list assignment question
Message-ID: <410F3465.9000005@object-craft.com.au>

Is there any reason why something like this would not be a good idea?

 >>> a_list = [1, 2, 3, 4, 5]
 >>> a, b, *c = a_list

You could then do things like this:

 >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]]
 >>> for a, b *c in lol:
...

- Dave

-- 
http://www.object-craft.com.au
From ncoghlan at iinet.net.au  Tue Aug  3 09:02:53 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue Aug  3 09:03:21 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <410F3465.9000005@object-craft.com.au>
References: <410F3465.9000005@object-craft.com.au>
Message-ID: <410F389D.6090609@iinet.net.au>

Dave Cole wrote:

> Is there any reason why something like this would not be a good idea?
> 
>  >>> a_list = [1, 2, 3, 4, 5]
>  >>> a, b, *c = a_list
> 
> You could then do things like this:
> 
>  >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]]
>  >>> for a, b *c in lol:
> ...
> 
> - Dave
> 

As opposed to:

 >>> for a, b, c in ((x[0], x[1], x[2:]) for x in lol): print a, b, c


With generator expressions around, I don't know that this case is common 
enough for special casing. . .

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From arigo at tunes.org  Tue Aug  3 10:54:01 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue Aug  3 10:58:44 2004
Subject: [Python-Dev] Optimized string concatenation
Message-ID: <20040803085401.GA2798@vicky.ecs.soton.ac.uk>

Hello,

The SF patch http://www.python.org/sf/980695 about making repeated string
concatenations efficient has been reviewed and is acceptable on technical
grounds.  This is about avoiding the quadratic behavior of

s = ''
for x in y:
  s += some_string(x)

This leaves open the policy questions:

* first, is that an implementation detail or a published feature?
  The question is important because the difference in performance is enormous
  -- we are not talking about 2x or even 10x faster but roughly Nx faster
  where N is the size of the input data set.

* if it is a published feature, what about Jython?

* The patch would encourage a coding style that gives program that essentially
  don't scale with Jython -- nor, for that matter, with 2.3 or older -- and
  worse, the programs would *appear* to work on Jython or 2.3 when tested with
  small or medium-sized data sets, but just appear to hang when run on larger
  data sets.  Obviously, this is problem that has always been here, but if we 
  fix it in 2.4 we can be sure that people will develop and test with 2.4,
  and less thoroughly on 2.3, and when they deploy on 2.3 platforms it will
  unexpectedly not scale.

* discussed on SF too is whether we should remove the 'a=a+b' acceleration
  from the patch, keeping only 'a+=b'; see the SF tracker.

This seems overkill, but should the acceleration be there but disabled by
default?

from __future__ import string_concatenate?


A bient?t,

Armin.
From arigo at tunes.org  Tue Aug  3 11:12:33 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue Aug  3 11:17:17 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects
	listobject.c, 2.218, 2.219
In-Reply-To: <002b01c478b3$cc9c7020$eb23c797@oemcomputer>
References: <E1BrrIq-0002nF-2x@sc8-pr-cvs1.sourceforge.net>
	<002b01c478b3$cc9c7020$eb23c797@oemcomputer>
Message-ID: <20040803091233.GB2798@vicky.ecs.soton.ac.uk>

Hello Raymond,

On Mon, Aug 02, 2004 at 01:11:49PM -0400, Raymond Hettinger wrote:
> >   	 * complain about the unused name.
> >   	 */
> > ! 	return status, v;
> >   }
> > 
> > --- 863,867 ----
> >   	 * complain about the unused name.
> >   	 */
> > ! 	return status++, v;
> 
> Does anyone else see this as perverse?

Yes!  And obfuscated definitely.  A general solution to this problem would be
helpful... well in this case I can think about a specific solution:

  assert(!PyErr_Occurred());

Patch below.

Armin

*** listobject.c        3 Aug 2004 04:53:29 -0000       2.219
--- listobject.c        3 Aug 2004 09:15:45 -0000
***************
*** 829,835 ****
  {
        int i = -1;
        PyObject *v, *arg = NULL;
-       int status;
  
        if (!PyArg_UnpackTuple(args, "pop", 0, 1, &arg))
                return NULL;
--- 829,834 ----
***************
*** 852,868 ****
        }
        v = self->ob_item[i];
        if (i == self->ob_size - 1) {
!               status = list_resize(self, self->ob_size - 1);
!               assert(status >= 0);
                return v; /* and v now owns the reference the list had */
        }
        Py_INCREF(v);
!       status = list_ass_slice(self, i, i+1, (PyObject *)NULL);
!       assert(status >= 0);
!       /* Use status, so that in a release build compilers don't
!        * complain about the unused name.
!        */
!       return status++, v;
  }
  
  /* Reverse a slice of a list in place, from lo up to (exclusive) hi. */
--- 851,863 ----
        }
        v = self->ob_item[i];
        if (i == self->ob_size - 1) {
!               list_resize(self, self->ob_size - 1);
                return v; /* and v now owns the reference the list had */
        }
        Py_INCREF(v);
!       list_ass_slice(self, i, i+1, (PyObject *)NULL);
!       assert(!PyErr_Occurred());
!       return v;
  }
  
  /* Reverse a slice of a list in place, from lo up to (exclusive) hi. */
From sjoerd at acm.org  Tue Aug  3 11:48:24 2004
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Tue Aug  3 11:48:30 2004
Subject: [Python-Dev] RE: [Python-checkins]
	python/dist/src/Objects	listobject.c, 2.218, 2.219
In-Reply-To: <20040803091233.GB2798@vicky.ecs.soton.ac.uk>
References: <E1BrrIq-0002nF-2x@sc8-pr-cvs1.sourceforge.net>	<002b01c478b3$cc9c7020$eb23c797@oemcomputer>
	<20040803091233.GB2798@vicky.ecs.soton.ac.uk>
Message-ID: <410F5F68.9030704@acm.org>

Armin Rigo wrote:
> Hello Raymond,
> 
> On Mon, Aug 02, 2004 at 01:11:49PM -0400, Raymond Hettinger wrote:
> 
>>>  	 * complain about the unused name.
>>>  	 */
>>>! 	return status, v;
>>>  }
>>>
>>>--- 863,867 ----
>>>  	 * complain about the unused name.
>>>  	 */
>>>! 	return status++, v;
>>
>>Does anyone else see this as perverse?
> 
> 
> Yes!  And obfuscated definitely.  A general solution to this problem would be
> helpful... well in this case I can think about a specific solution:

This is indeed perverted.  Another option is to add a statement
	(void) status;
somewhere to the function and to remove the status++ from the return. 
This seems to shut up all compilers I've dealt with.

>   assert(!PyErr_Occurred());
> 
> Patch below.
> 
> Armin
> 
> *** listobject.c        3 Aug 2004 04:53:29 -0000       2.219
> --- listobject.c        3 Aug 2004 09:15:45 -0000
> ***************
> *** 829,835 ****
>   {
>         int i = -1;
>         PyObject *v, *arg = NULL;
> -       int status;
>   
>         if (!PyArg_UnpackTuple(args, "pop", 0, 1, &arg))
>                 return NULL;
> --- 829,834 ----
> ***************
> *** 852,868 ****
>         }
>         v = self->ob_item[i];
>         if (i == self->ob_size - 1) {
> !               status = list_resize(self, self->ob_size - 1);
> !               assert(status >= 0);
>                 return v; /* and v now owns the reference the list had */
>         }
>         Py_INCREF(v);
> !       status = list_ass_slice(self, i, i+1, (PyObject *)NULL);
> !       assert(status >= 0);
> !       /* Use status, so that in a release build compilers don't
> !        * complain about the unused name.
> !        */
> !       return status++, v;
>   }
>   
>   /* Reverse a slice of a list in place, from lo up to (exclusive) hi. */
> --- 851,863 ----
>         }
>         v = self->ob_item[i];
>         if (i == self->ob_size - 1) {
> !               list_resize(self, self->ob_size - 1);
>                 return v; /* and v now owns the reference the list had */
>         }
>         Py_INCREF(v);
> !       list_ass_slice(self, i, i+1, (PyObject *)NULL);
> !       assert(!PyErr_Occurred());
> !       return v;
>   }
>   
>   /* Reverse a slice of a list in place, from lo up to (exclusive) hi. */
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/sjoerd%40acm.org


-- 
Sjoerd Mullender <sjoerd@acm.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 374 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040803/abc2c638/signature.pgp
From mwh at python.net  Tue Aug  3 12:03:54 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug  3 12:03:55 2004
Subject: [Python-Dev] refleak hunting season
In-Reply-To: <20040803042017.GN7708@epoch.metaslash.com> (Neal Norwitz's
	message of "Tue, 3 Aug 2004 00:20:18 -0400")
References: <2mpt69bo47.fsf@starship.python.net>
	<20040803042017.GN7708@epoch.metaslash.com>
Message-ID: <2m4qnkbks5.fsf@starship.python.net>

Neal Norwitz <neal@metaslash.com> writes:

> On Mon, Aug 02, 2004 at 03:39:36PM +0100, Michael Hudson wrote:
>> 
>> test_sax leaked [1899, 1899, 1899, 1899] references
>>     no idea -- scary numbers, though!
>
> It looks like the worst (only?) leakers are:
>
>         test_expat_incomplete
>         test_expat_inpsource_location

Yes.  The rest are all caused by the module level variable items
containing it's old value...

> Both of these cause exceptions to be raised.  I don't know the pyexpat
> code at all.  But I think I fixed a few leaks with the attached patch.

I certainly agree with your patch to set_error_attr (I had the same).
Also, I think one needs to decref 'err' in set_error (PyErr_SetObject
increfs its arguments).

> Some leaks were caught by the tests, but I doubt all were. 

Probably not :-/

> The patch is a starting point if anyone else wants to give it a
> shot.  If not, I'll add it to SF or review carefully and check in
> later.

Well, it seems Fred wants a look.  I'll attach my diffs to pyexpat.c
to the report (there's another fairly obvious leak exposed by
test_minidom).

Cheers,
mwh

-- 
  > I'm a little confused.
  That's because you're Australian!  So all the blood flows to
  your head, away from the organ most normal guys think with.
                        -- Mark Hammond & Tim Peters, comp.lang.python
From mwh at python.net  Tue Aug  3 12:33:13 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug  3 12:33:16 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410E93AC.9050807@zope.com> (Jim Fulton's message of "Mon, 02
	Aug 2004 15:19:08 -0400")
References: <410DDFE3.7010006@interlink.com.au> <410E93AC.9050807@zope.com>
Message-ID: <2mzn5ca4uu.fsf@starship.python.net>

Jim Fulton <jim@zope.com> writes:

> Anthony Baxter wrote:
>> The @decorator patch has been landed on the trunk, after
>> getting the go-ahead from the BDFL. I'll update PEP-0318
>> with the final syntax in the next day or so.
>
> In the mean time, can anyone give a pointer to where this was proposed
> or discussed?
>

AFAIK, in offline email (and occasionally on IRC, but that was mostly
about implementation), presumably to avoid the thousands of emails
cascade this subject inevitably seems to trigger.

bike-shed-the-ultimate-ly y'rs
mwh

-- 
  If a train station is a place where a train stops, what's a
  workstation?                            -- unknown (to me, at least)
From mwh at python.net  Tue Aug  3 12:36:31 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug  3 12:36:33 2004
Subject: [Python-Dev] Bug day coming up this Saturday
In-Reply-To: <20040802191815.GA9528@rogue.amk.ca> (A. M. Kuchling's message
	of "Mon, 2 Aug 2004 15:18:15 -0400")
References: <20040802191815.GA9528@rogue.amk.ca>
Message-ID: <2mvfg0a4pc.fsf@starship.python.net>

"A.M. Kuchling" <amk@amk.ca> writes:

> When?
> =====
>
> This Saturday, August 7, 2004, from 9AM to 5PM EDT (1PM to 9PM GMT).
> You don't need to be around all day; feel free to stop by for a few hours
> and contribute.

I should be able to attend this one.

Cheers,
mwh

-- 
 (Of course SML does have its weaknesses, but by comparison, a
  discussion of C++'s strengths and flaws always sounds like an
  argument about whether one should face north or east when one
  is sacrificing one's goat to the rain god.)         -- Thant Tessman
From jim at zope.com  Tue Aug  3 12:43:14 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Aug  3 12:43:57 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
References: <1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>	<200408030105.i7315Fj29310@guido.python.org>
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
Message-ID: <410F6C42.8010001@zope.com>

James Y Knight wrote:
...
> Things someone might want to do, ordered roughly from most reasonable to 
> least reasonable ;)
> @foo().bar()
> @foo or bar
> @mydecorators['foo']
> @lambda f: foo(f) or bar(f)
> 
> Why disallow these forms? It seems quite difficult, especially, to 
> explain why the first one does not, or should not, work.

+1



-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From mwh at python.net  Tue Aug  3 13:36:26 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug  3 13:36:28 2004
Subject: [Python-Dev] refleak hunting season
In-Reply-To: <2mpt69bo47.fsf@starship.python.net> (Michael Hudson's message
	of "Mon, 02 Aug 2004 15:39:36 +0100")
References: <2mpt69bo47.fsf@starship.python.net>
Message-ID: <2mr7qoa1xh.fsf@starship.python.net>

Michael Hudson <mwh@python.net> writes:

> Last time round I found a bunch of reference counting bugs in Python
> just *after* a major release; I'm trying to do it all before 2.4.0
> this time.

I think I've now accounted for all the apparent and real refleaks
revealed by the test suite (not with -uall, will try that soon).  I've
checked my regrtest patches in.  I've fixed a bunch of tests that
failed when repeated and I've filed bugs against those I couldn't
figure out.

Hopefully we can keep on top of these things!

Cheers,
mwh

-- 
  Also, remember to put the galaxy back when you've finished, or an
  angry mob of astronomers will come round and kneecap you with a
  small telescope for littering. 
       -- Simon Tatham, ucam.chat, from Owen Dunn's review of the year
From marktrussell at btopenworld.com  Tue Aug  3 13:51:14 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Tue Aug  3 13:51:17 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410F2453.7000506@divmod.com>
References: <Your message of "Mon, 02 Aug 2004 13:30:03
	EDT."	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<410F2453.7000506@divmod.com>
Message-ID: <1091533874.3797.0.camel@localhost>

On Tue, 2004-08-03 at 06:36, Jp Calderone wrote:
>    I realize there is little or no chance of '@decorator' being pulled 
> from 2.4a2.  I hope that something along the lines of the above will be 
> considered, instead, for the next alpha, unless there is widespread 
> community support for '@decorator', as opposed to the ridiculously faint 
> support ("it's better than nothing") currently behind it.

Personally I think it's not just better than nothing, it's better than
any of the alternatives I've seen, including the do-nothing option. 

I added a comment to your blog post in defence of @decorator --
http://www.livejournal.com/users/jcalderone/3913.html?thread=2121#t2121
but lets not restart a syntax debate here.

My guess is that the syntax will grow on people as they use it.  If not
there is the alternative of using one of the pure-python versions or
just carrying on using the existing way of doing things.

Mark


From mcherm at mcherm.com  Tue Aug  3 14:30:04 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Aug  3 14:29:30 2004
Subject: [Python-Dev] Optimized string concatenation
Message-ID: <1091536204.410f854c18727@mcherm.com>

Armin writes:
> The SF patch http://www.python.org/sf/980695 about making
> repeated string
> concatenations efficient has been reviewed and is acceptable
> on technical grounds.
    [...]
> This leaves open the policy questions:
>
> * first, is that an implementation detail or a published feature?

IMHO, an implementation detail. Specifically, a published feature
of CPython, but an implementation detail of the language Python.

> * if it is a published feature, what about Jython?

And I guess we should ask about Iron Python too. The answer is
that they add this optimization if and only if the maintainers of
those versions get around to it. Whether it is easily done given
the underlying platform is another important consideration. But
if we leave it OUT of the language spec then we avoid unnecessarily
breaking other implementations.

That being said, I'll note that both Jython and Iron Python already
use unicode as the basic string type, which is a much bigger
change (except to those who manage to use only 7-bit ASCII).

> * The patch would encourage a coding style that gives program
> that essentially
>   don't scale with Jython -- nor, for that matter, with 2.3
> or older

Yes, but we ALREADY see lots of programs that use that coding
style, even though every web page talking about Python optimization
lists that as the #1 issue. Whether we like it or not, it seems
that especially for novice programmers, the "s = s + x" idiom for
accumulating a string is the "obvious" way to do it. Encouraging
that may not be good, but going along rather than fighting it
seems like a wise idea.

> * discussed on SF too is whether we should remove the 'a=a+b'
> acceleration
>   from the patch, keeping only 'a+=b'; see the SF tracker.

Hmm... I couldn't think of any reason to limit the optimization
to += until I actually went and read the comments in the SF
tracker. What I took away from that discussion was that it's
possible to optimize "a=a+b", but NOT possible to optimize
"a=a+b+c". This is a subtle distinction that is harder to
explain to people than simply saying "it only works with +=, not
with +".

From mcherm at mcherm.com  Tue Aug  3 14:33:29 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Aug  3 14:32:56 2004
Subject: [Python-Dev] Optimized string concatenation
Message-ID: <1091536409.410f8619be6f0@mcherm.com>

PLEASE IGNORE PREVIOUS MESSAGE.
(I really ought to stop using a mailer which is one accidental click
away from sending an unfinished email)


Armin writes:
> The SF patch http://www.python.org/sf/980695 about making
> repeated string
> concatenations efficient has been reviewed and is acceptable
> on technical grounds.
    [...]
> This leaves open the policy questions:
>
> * first, is that an implementation detail or a published feature?

IMHO, an implementation detail. Specifically, a published feature
of CPython, but an implementation detail of the language Python.

> * if it is a published feature, what about Jython?

And I guess we should ask about Iron Python too. The answer is
that they add this optimization if and only if the maintainers of
those versions get around to it. Whether it is easily done given
the underlying platform is another important consideration. But
if we leave it OUT of the language spec then we avoid unnecessarily
breaking other implementations.

That being said, I'll note that both Jython and Iron Python already
use unicode as the basic string type, which is a much bigger
change (except to those who manage to use only 7-bit ASCII).

> * The patch would encourage a coding style that gives program
> that essentially
>   don't scale with Jython -- nor, for that matter, with 2.3
> or older

Yes, but we ALREADY see lots of programs that use that coding
style, even though every web page talking about Python optimization
lists that as the #1 issue. Whether we like it or not, it seems
that especially for novice programmers, the "s = s + x" idiom for
accumulating a string is the "obvious" way to do it. Encouraging
that may not be good, but going along rather than fighting it
seems like a wise idea.

> * discussed on SF too is whether we should remove the 'a=a+b'
> acceleration
>   from the patch, keeping only 'a+=b'; see the SF tracker.

Hmm... I couldn't think of any reason to limit the optimization
to += until I actually went and read the comments in the SF
tracker. What I took away from that discussion was that it's
possible to optimize "a=a+b", but NOT possible to optimize
"a=a+b+c". This is a subtle distinction that is harder to
explain to people than simply saying "it only works with +=, not
with +".

That's a fairly convincing point, so I guess I'm on the fence
on this one.

> This seems overkill, but should the acceleration be there but
> disabled by
> default?
>
> from __future__ import string_concatenate?

Absolutely, unconditionally NOT. I'd rather just leave it out.

-- Michael Chermside

From jhylton at gmail.com  Tue Aug  3 14:47:41 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Tue Aug  3 14:47:44 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410F2453.7000506@divmod.com>
References: <Your message of "Mon, 02 Aug 2004 13:30:03
	EDT."	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<410F2453.7000506@divmod.com>
Message-ID: <e8bf7a5304080305475a9aecb@mail.gmail.com>

On Tue, 03 Aug 2004 01:36:19 -0400, Jp Calderone <exarkun@divmod.com> wrote:
[JP provides a partial implementation of a variant decorator approach]
>    I realize there is little or no chance of '@decorator' being pulled
> from 2.4a2.  I hope that something along the lines of the above will be
> considered, instead, for the next alpha, unless there is widespread
> community support for '@decorator', as opposed to the ridiculously faint
> support ("it's better than nothing") currently behind it.

I'm not sure how to take your post.  In particular, I don't see the
implementation as the limiting factor in decorator design.  We can
probably implement any design that we want to.  The question is
actually about the preferred design.  You seem to be suggesting a
design where decorate() is used as a pseudo-keyword; assuming, for
example, that you'd want it to work for functions as well as methods. 
I suspect it's futile to propose alternate designs at this point.

Jeremy
From jhylton at gmail.com  Tue Aug  3 14:51:36 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Tue Aug  3 14:51:38 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects
	listobject.c, 2.218, 2.219
In-Reply-To: <410F5F68.9030704@acm.org>
References: <E1BrrIq-0002nF-2x@sc8-pr-cvs1.sourceforge.net>	<002b01c478b3$cc9c7020$eb23c797@oemcomputer>
	<20040803091233.GB2798@vicky.ecs.soton.ac.uk>
	<410F5F68.9030704@acm.org>
Message-ID: <e8bf7a530408030551f9210a@mail.gmail.com>

On Tue, 03 Aug 2004 11:48:24 +0200, Sjoerd Mullender <sjoerd@acm.org> wrote:
> This is indeed perverted.  Another option is to add a statement
>         (void) status;
> somewhere to the function and to remove the status++ from the return.
> This seems to shut up all compilers I've dealt with.

I believe we use this idiom elsewhere in the Python codebase.  If it's
combined with a comment like /* avoid compile warning about unused
variable */ it's quite clear.

Jeremy
From FBatista at uniFON.com.ar  Tue Aug  3 15:08:53 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Aug  3 15:12:08 2004
Subject: [Python-Dev] Optimized string concatenation
Message-ID: <A128D751272CD411BC9200508BC2194D053C773A@escpl.tcp.com.ar>

[Armin Rigo]

#- s = ''
#- for x in y:
#-   s += some_string(x)
#- 
#- This leaves open the policy questions:
#- 
#- * first, is that an implementation detail or a published feature?

Depending on how much slower (if still) is now this idiom against...

  l = [some_string(x) for x in y]
  s = ''.join(l)

...maybe we should stop concatenating strings. One or two points of the FAQs
should be reviewed, but only after a speed comparison.

.	Facundo





. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
ADVERTENCIA  

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. 

Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo. 

Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada. 

Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje. 

Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040803/357dd29e/attachment.html
From pje at telecommunity.com  Tue Aug  3 16:04:27 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug  3 16:00:26 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410F2453.7000506@divmod.com>
References: <200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03
	EDT."	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
Message-ID: <5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>

At 01:36 AM 8/3/04 -0400, Jp Calderone wrote:
>     There are clear drawbacks to this approach.  Metaclass required, no 
> obvious ways to support free functions, and it depends _slightly_ 
> hackishly on the inspect module.  I think at least one of these 
> problems  can be solved (and line number handling can be made smarter to 
> deal with intervening comments, whitespace, etc).

But at least *two* of those drawbacks are critical, IMO: the metaclass 
requirement and the absence of free function support.


>     What are the advantages?
>
>   * It works with Python 2.2 and Python 2.3.
>
>   * It requires no interpreter changes.  It can be distributed with the 
> standard library, as a cookbook recipe, or in the official documentation 
> as a hint regarding more "advanced" decorator usage.
>
>   * It introduces no new syntax and uses up no operator character.
>
>   * It supports arbitrary expressions.
>
>   * It's pure python.

If those are the only advantages, one might as well use the settrace-based 
hack, which doesn't have the problems that this one does.  Note, by the 
way, that your 'decorate' function can be implemented by renaming my 'as' 
function to 'decorate' and dropping the 'decorators.reverse()' line.  (i.e. 
'as(x,y,z)' is the same as 'decorate(z,y,x)').  Although my examples 
wrapped such calls in brackets, there's no actual need for them.  (Note 
also that the settrace hack works for class decorations as well as functions.)

(FYI: 'as' and 'add_assignment_advisor' are available in the PyProtocols 
CVS head, in the 'protocols.advice' module.)


>   I realize there is little or no chance of '@decorator' being pulled 
> from 2.4a2.  I hope that something along the lines of the above will be 
> considered, instead, for the next alpha, unless there is widespread 
> community support for '@decorator', as opposed to the ridiculously faint 
> support ("it's better than nothing") currently behind it.

Actually, I'm not sure I agree that it's better than nothing.  For one 
thing, I don't think that there's any ambiguity of interpretation when 
decorators are written as function calls, be it 'as(classmethod)' or 
'decorate(spam)' -- and I doubt that the point was argued at EuroPython, 
given that Guido missed the thread about it.

But...  this is moot, since it clearly has been pronounced upon.  That 
pretty much means folks who want decorators right away are left with using 
the hacks, and not having a syntax-compatible upgrade path.  However, 
that's *still* better than where they were before, because none of the 
original syntax proposals were expected to be backward-compatible.

In practical terms, all this means is that I'll just use the hacked syntax 
until I no longer need to support Python 2.3.

From guido at python.org  Tue Aug  3 16:04:38 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 16:04:42 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Tue, 03 Aug 2004 01:36:19 EDT."
	<410F2453.7000506@divmod.com> 
References: <Your message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org> 
	<410F2453.7000506@divmod.com> 
Message-ID: <200408031404.i73E4cS30629@guido.python.org>

>      Here's a brief test for a syntax-change-less implementation of this 
> feature, not as complete as test_decorators, but a good start, I believe:

[fast forward to syntax example]

>          decorate(staticmethod)
>          def bar(x):
>              print x
> 
>          decorate(classmethod)
>          def baz(cls, y):
>              print cls, y

I'm speechless.  If the ambiguous

    [classmethod]
    def foo(x):
        ...

is rejected because it doesn't look like it does something to foo, how
come there's suddenly a crop of solutions that have the same problem
being proposed?  What you write looks like a call to the function
decorate(), followed by a function method definition.  The
"action-at-a-distance" that is presumed by the decorate() call is
difficult to explain and a precedent for other worse hacks.  Its only
point in favor seems to be that it doesn't use '@'.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug  3 16:13:30 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 16:13:40 2004
Subject: [Python-Dev] Optimized string concatenation
In-Reply-To: Your message of "Tue, 03 Aug 2004 09:54:01 BST."
	<20040803085401.GA2798@vicky.ecs.soton.ac.uk> 
References: <20040803085401.GA2798@vicky.ecs.soton.ac.uk> 
Message-ID: <200408031413.i73EDUu30665@guido.python.org>

> The SF patch http://www.python.org/sf/980695 about making repeated string
> concatenations efficient has been reviewed and is acceptable on technical
> grounds.  This is about avoiding the quadratic behavior of
> 
> s = ''
> for x in y:
>   s += some_string(x)

FWIW, I didn't approve it, Raymond did, IMO prematurely. :-(

In the past similar schemes based on refcount were rejected because we
couldn't be sure that there wasn't C code with an uncounted
reference.  Is that ruled out in this scheme?

Also, I believe that even Java doesn't optimize this case in general
-- only the much simpler case where you concatenate a number of
strings in a single expression.

> This leaves open the policy questions:
> 
> * first, is that an implementation detail or a published feature?
>   The question is important because the difference in performance is
>   enormous -- we are not talking about 2x or even 10x faster but
>   roughly Nx faster where N is the size of the input data set.

Like tail recursion, it will change the way people write code.  I am
going to make up a new rule here, and state that implementation
features that affect not just the running speed but the O() rate for
certain algorithms should be considered language features, because
any implementation will be required to implement them in order to
ensure code portability.

> * if it is a published feature, what about Jython?
> 
> * The patch would encourage a coding style that gives program that
>   essentially don't scale with Jython -- nor, for that matter, with
>   2.3 or older -- and worse, the programs would *appear* to work on
>   Jython or 2.3 when tested with small or medium-sized data sets,
>   but just appear to hang when run on larger data sets.  Obviously,
>   this is problem that has always been here, but if we fix it in 2.4
>   we can be sure that people will develop and test with 2.4, and
>   less thoroughly on 2.3, and when they deploy on 2.3 platforms it
>   will unexpectedly not scale.
> 
> * discussed on SF too is whether we should remove the 'a=a+b'
>   acceleration from the patch, keeping only 'a+=b'; see the SF
>   tracker.
> 
> This seems overkill, but should the acceleration be there but
> disabled by default?
> 
> from __future__ import string_concatenate?

Why do people want this so badly?  Please leave well enough alone.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug  3 16:18:15 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 16:18:20 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/dist/src/Python compile.c, 2.311, 2.312
In-Reply-To: Your message of "Tue, 03 Aug 2004 03:24:07 PDT."
	<E1BrwSl-0005lx-7A@sc8-pr-cvs1.sourceforge.net> 
References: <E1BrwSl-0005lx-7A@sc8-pr-cvs1.sourceforge.net> 
Message-ID: <200408031418.i73EIFR30699@guido.python.org>

> Log Message:
> Don't intern the filename of a file being compiled.
> Hard to believe it ever helped anything, and it hurts finding reference
> leaks.

The intention was to introduce sharing of the filename object between
code objects compiled from the same file (remember that every method
is a separate code object).

But I believe the sharing won't happen when the code is loaded from a
bytecoded file instead, so it is indeed wasted efficiency.

Good catch.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug  3 16:22:25 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 16:22:47 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Tue, 03 Aug 2004 11:33:13 BST."
	<2mzn5ca4uu.fsf@starship.python.net> 
References: <410DDFE3.7010006@interlink.com.au> <410E93AC.9050807@zope.com> 
	<2mzn5ca4uu.fsf@starship.python.net> 
Message-ID: <200408031422.i73EMPX30737@guido.python.org>

> Jim Fulton <jim@zope.com> writes:
> > In the mean time, can anyone give a pointer to where this was
> > proposed or discussed?

There are two slides in my EuroPython keynote

(http://www.python.org/doc/essays/ppt/euro2004/euro2004.pdf)

(pages 10 and 11) that mention it.

It was discussion during and after this keynote that made me change my
mind -- first I accepted that [deco] before def was unacceptable, then
I found that @deco was not universally hated (Fredrik Lundh even liked
it).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From python at rcn.com  Tue Aug  3 04:24:03 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug  3 16:25:41 2004
Subject: [Python-Dev] Optimized string concatenation
In-Reply-To: <1091536409.410f8619be6f0@mcherm.com>
Message-ID: <003101c47900$f23161e0$df30cb97@oemcomputer>

> > * discussed on SF too is whether we should remove the 'a=a+b'
> > acceleration
> >   from the patch, keeping only 'a+=b'; see the SF tracker.


> Hmm... I couldn't think of any reason to limit the optimization
> to += until I actually went and read the comments in the SF
> tracker. What I took away from that discussion was that it's
> possible to optimize "a=a+b", but NOT possible to optimize
> "a=a+b+c". This is a subtle distinction that is harder to
> explain to people than simply saying "it only works with +=, not
> with +".
> 
> That's a fairly convincing point, so I guess I'm on the fence
> on this one.

I'm not.  Skipping a=a+b breaks symmetry with a+=b.  More importantly,
skipping a=a+b misses most of the potential benefits (see sre_parse.py
for an example).  PyBench, ParrotBench, and my other benchmarks all show
gains when a=a+b is done inplace.


The explanation is not hard.  The CPython implementation can concatenate
inplace two term expressions of the form a=a+b or a+=b.  Expressions
with more terms are not eligible for inplace concatenation.


Raymond 

From martin at v.loewis.de  Tue Aug  3 16:25:55 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug  3 16:25:58 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.311, 2.312
In-Reply-To: <200408031418.i73EIFR30699@guido.python.org>
References: <E1BrwSl-0005lx-7A@sc8-pr-cvs1.sourceforge.net>
	<200408031418.i73EIFR30699@guido.python.org>
Message-ID: <410FA073.5070105@v.loewis.de>

Guido van Rossum wrote:
> The intention was to introduce sharing of the filename object between
> code objects compiled from the same file (remember that every method
> is a separate code object).
> 
> But I believe the sharing won't happen when the code is loaded from a
> bytecoded file instead, so it is indeed wasted efficiency.

In 2.4, it would: strings that are interned when being marshalled will
get interned on unmarshalling.

Regards,
Martin
From guido at python.org  Tue Aug  3 16:27:49 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 16:27:54 2004
Subject: [Python-Dev] PEP 324 (process module)
In-Reply-To: Your message of "Tue, 03 Aug 2004 06:13:59 PDT."
	<E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net> 
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net> 
Message-ID: <200408031427.i73ERn330790@guido.python.org>

I believe ActiveState's Trent Mick has released something along these
lines.  Does your proposed API match theirs?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Aug  3 16:32:32 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 16:32:39 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Tue, 03 Aug 2004 10:04:27 EDT."
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> 
References: <200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org> 
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> 
Message-ID: <200408031432.i73EWWV30829@guido.python.org>

> In practical terms, all this means is that I'll just use the hacked
> syntax until I no longer need to support Python 2.3.

I don't understand why you prefer your hack over the established way
to do decorators pre-2.4, which is

  def foo(): ...
  foo = staticmethod(foo)

This works across releases (including IronPython), doesn't require any
magic, is documented, etc.  So if your main constraint is that it be
implementable pre-2.4, you already have a solution.  Isn't that much
better than spending effort on hacks based on sys.settrace (which are
surely going to produce bafflement from users who aren't familiar with
that implementation hack)?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Aug  3 16:34:57 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 16:35:07 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.311, 2.312
In-Reply-To: Your message of "Tue, 03 Aug 2004 16:25:55 +0200."
	<410FA073.5070105@v.loewis.de> 
References: <E1BrwSl-0005lx-7A@sc8-pr-cvs1.sourceforge.net>
	<200408031418.i73EIFR30699@guido.python.org> 
	<410FA073.5070105@v.loewis.de> 
Message-ID: <200408031434.i73EYvv30846@guido.python.org>

> > The intention was to introduce sharing of the filename object between
> > code objects compiled from the same file (remember that every method
> > is a separate code object).
> > 
> > But I believe the sharing won't happen when the code is loaded from a
> > bytecoded file instead, so it is indeed wasted efficiency.
> 
> In 2.4, it would: strings that are interned when being marshalled will
> get interned on unmarshalling.

Aha!  So maybe we should reconsider whether mwh's removal of the
filename interning in the compiler should be reverted.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From pje at telecommunity.com  Tue Aug  3 16:41:15 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug  3 16:37:13 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408031404.i73E4cS30629@guido.python.org>
References: <Your message of "Tue,
	03 Aug 2004 01:36:19 EDT." <410F2453.7000506@divmod.com>
	<Your message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<410F2453.7000506@divmod.com>
Message-ID: <5.1.1.6.0.20040803101242.02a0b750@mail.telecommunity.com>

At 07:04 AM 8/3/04 -0700, Guido van Rossum wrote:
> >      Here's a brief test for a syntax-change-less implementation of this
> > feature, not as complete as test_decorators, but a good start, I believe:
>
>[fast forward to syntax example]
>
> >          decorate(staticmethod)
> >          def bar(x):
> >              print x
> >
> >          decorate(classmethod)
> >          def baz(cls, y):
> >              print cls, y
>
>I'm speechless.  If the ambiguous
>
>     [classmethod]
>     def foo(x):
>         ...
>
>is rejected because it doesn't look like it does something to foo, how
>come there's suddenly a crop of solutions that have the same problem
>being proposed?

Because '[foo]' is absolutely a list with no side effects in today's 
Python.  But '[foo()]' clearly *can* have some side effect, so if you're 
reading the code you have to look up 'foo' to understand what's happening 
there.

This isn't at all sudden, btw: David Abrahams made the first 'decorate' 
proposal in June, and suggested the idea of using the debugger hook to 
implement it.  I thought that one of the best parts of David Abrahams' idea 
was that requiring a function call made the [] syntax less ambiguous.


>   What you write looks like a call to the function
>decorate(), followed by a function method definition.  The
>"action-at-a-distance" that is presumed by the decorate() call is
>difficult to explain and a precedent for other worse hacks.

I agree, which is why I wrap my decorators in [], even though it has no 
effect on the actual operation.  It's pure "eye candy" sprinkled with 
syntactic sugar.  :)


>  Its only
>point in favor seems to be that it doesn't use '@'.

Well, the other point is that it allows you to use C# syntax.  Note that if 
it were spelled:

     [classmethod()]
     def foo(x):
         ...

this would also resolve the ambiguity (IMO).

From guido at python.org  Tue Aug  3 16:40:05 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 16:40:19 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Tue, 03 Aug 2004 00:17:43 EDT."
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> 
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org> 
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net> 
Message-ID: <200408031440.i73Ee6V30875@guido.python.org>

> The only real necessary restriction on the @ operator is that its
> argument be callable and take a single argument. Many expressions
> could return a callable object. Why not let them? Is it really worth
> having a special case just to SyntaxError expressions that sometimes
> won't result in an appropriate callable?
> 
> Things someone might want to do, ordered roughly from most reasonable 
> to least reasonable ;)
> @foo().bar()
> @foo or bar
> @mydecorators['foo']
> @lambda f: foo(f) or bar(f)
> 
> Why disallow these forms? It seems quite difficult, especially, to 
> explain why the first one does not, or should not, work.

I have a gut feeling about this one.  I'm not sure where it comes
from, but I have it.  It may be that I want the compiler to be able to
recognize certain decorators.

So while it would be quite easy to change the syntax to @test in the
future, I'd like to stick with the more restricted form unless a real
use case is presented where allowing @test would increase readability.
(@foo().bar() doesn't count because I don't expect you'll ever need
that).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug  3 16:50:00 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 16:50:06 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Tue, 03 Aug 2004 10:41:15 EDT."
	<5.1.1.6.0.20040803101242.02a0b750@mail.telecommunity.com> 
References: <Your message of "Tue,
	03 Aug 2004 01:36:19 EDT." <410F2453.7000506@divmod.com> <Your
	message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<410F2453.7000506@divmod.com> 
	<5.1.1.6.0.20040803101242.02a0b750@mail.telecommunity.com> 
Message-ID: <200408031450.i73Eo0e30956@guido.python.org>

> Because '[foo]' is absolutely a list with no side effects in today's
> Python.  But '[foo()]' clearly *can* have some side effect, so if
> you're reading the code you have to look up 'foo' to understand
> what's happening there.

Too subtle for the folks who argued against [foo] on grounds of
ambiguity I think.

> This isn't at all sudden, btw: David Abrahams made the first
> 'decorate' proposal in June, and suggested the idea of using the
> debugger hook to implement it.  I thought that one of the best parts
> of David Abrahams' idea was that requiring a function call made the
> [] syntax less ambiguous.

And I think it is one of the worst.  sys.settrace is an implementation
feature not a language feature.

> Well, the other point is that it allows you to use C# syntax.  Note
> that if it were spelled:
> 
>      [classmethod()]
>      def foo(x):
>          ...
> 
> this would also resolve the ambiguity (IMO).

Not IMO, that would just add more mystery (and would require yet
another hack in classmethod to allow it to be called without args).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From python at rcn.com  Tue Aug  3 04:48:31 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug  3 16:50:08 2004
Subject: [Python-Dev] Optimized string concatenation
In-Reply-To: <200408031413.i73EDUu30665@guido.python.org>
Message-ID: <003201c47904$5cd353c0$df30cb97@oemcomputer>

> > The SF patch http://www.python.org/sf/980695 about making repeated
> string
> > concatenations efficient has been reviewed and is acceptable on
> technical
> > grounds.  This is about avoiding the quadratic behavior of
> >
> > s = ''
> > for x in y:
> >   s += some_string(x)
> 
> FWIW, I didn't approve it, Raymond did, IMO prematurely. :-(
> 
> In the past similar schemes based on refcount were rejected because we
> couldn't be sure that there wasn't C code with an uncounted
> reference.  Is that ruled out in this scheme?

Yes it is.  Unlike the refcount check proposed for PyList_Sort, Armin's
patch only applies to the eval loop and cannot be called directly.

I spent a *long* time reviewing and testing this patch.  The code is
clean.  The concept works.  It delivers significant, measurable
benefits.  The standard library itself has existing use cases.  Every
benchmark I tried showed results.  The patch was not approved
prematurely!

On the off chance that Armin, Michael, and I missed a case, then
including the patch in the second alpha is the best way to find it.



>  I am
> going to make up a new rule here, and state that implementation
> features that affect not just the running speed but the O() rate for
> certain algorithms should be considered language features, because
> any implementation will be required to implement them in order to
> ensure code portability.

Even list.sort() does not guarantee specific O() rates.  Currently, the
fastest way to write a merge(a,b) is sort(a+b).  That relies on the
current implementation recognizing that a and b are already sorted and
doing a linear time concatenation.



> Why do people want this so badly? 

Because the a=a+b form occurs everywhere, not just in newbie code.  It
is in the standard library, it shows up naturally in SAX, and it is
often the most clear way to express an idea.



Raymond

From tim.peters at gmail.com  Tue Aug  3 16:53:12 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug  3 16:53:15 2004
Subject: [Python-Dev] RE: [Python-checkins]
	python/dist/src/Objects listobject.c, 2.218, 2.219
In-Reply-To: <e8bf7a530408030551f9210a@mail.gmail.com>
References: <E1BrrIq-0002nF-2x@sc8-pr-cvs1.sourceforge.net>	<002b01c478b3$cc9c7020$eb23c797@oemcomputer>
	<20040803091233.GB2798@vicky.ecs.soton.ac.uk>
	<410F5F68.9030704@acm.org> <e8bf7a530408030551f9210a@mail.gmail.com>
Message-ID: <1f7befae04080307536a8e3c5c@mail.gmail.com>

[Sjoerd Mullender]
> This is indeed perverted.  Another option is to add a statement
>         (void) status;
> somewhere to the function and to remove the status++ from the return.
> This seems to shut up all compilers I've dealt with.

If that's enough to shut compilers up, +1 from me.  MSVC 7.1 doesn't
whine even with no tricks.

I do want to keep the two optimizations in list.pop(), which consist
of not test+branch'ing the return values of two calls that have been
proved safe.  But things change over time, and in a debug build I do
want to assert that they are in fact safe.

I originally did this with a pile of #ifdef Py_DEBUG blocks, but (as
the original checkin comment said) that was even uglier than the
comma-expression hack.
From barry at python.org  Tue Aug  3 16:53:36 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug  3 16:53:32 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408031432.i73EWWV30829@guido.python.org>
References: <200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408031432.i73EWWV30829@guido.python.org>
Message-ID: <1091544810.8416.83.camel@localhost>

On Tue, 2004-08-03 at 10:32, Guido van Rossum wrote:

> I don't understand why you prefer your hack over the established way
> to do decorators pre-2.4, which is
> 
>   def foo(): ...
>   foo = staticmethod(foo)
> 
> This works across releases (including IronPython), doesn't require any
> magic, is documented, etc.  So if your main constraint is that it be
> implementable pre-2.4, you already have a solution.  Isn't that much
> better than spending effort on hacks based on sys.settrace (which are
> surely going to produce bafflement from users who aren't familiar with
> that implementation hack)?

+1

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040803/72c063c2/attachment.pgp
From marktrussell at btopenworld.com  Tue Aug  3 13:06:54 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Tue Aug  3 16:56:37 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
Message-ID: <1091531214.1502.20.camel@localhost>

On Tue, 2004-08-03 at 05:17, James Y Knight wrote:
> Things someone might want to do, ordered roughly from most reasonable 
> to least reasonable ;)
> @foo().bar()
> @foo or bar
> @mydecorators['foo']
> @lambda f: foo(f) or bar(f)
> 
> Why disallow these forms? It seems quite difficult, especially, to 
> explain why the first one does not, or should not, work.

I don't think any of these seem that likely in practice, and if you do
want them, it is not hard to say:

	# Hopefully some comment here explaining the reason why you're
	# doing this.  And a more meaningful name than foo_or_bar!
	foo_or_bar = foo or bar

	@foo_or_bar
	def func(): pass

I would say leave it like this for now.  It is easy to relax the
restriction later if there's a demand.

Mark

From pje at telecommunity.com  Tue Aug  3 17:00:41 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug  3 16:56:40 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408031432.i73EWWV30829@guido.python.org>
References: <Your message of "Tue, 03 Aug 2004 10:04:27 EDT."
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040803104400.01ea5590@mail.telecommunity.com>

At 07:32 AM 8/3/04 -0700, Guido van Rossum wrote:
> > In practical terms, all this means is that I'll just use the hacked
> > syntax until I no longer need to support Python 2.3.
>
>I don't understand why you prefer your hack over the established way
>to do decorators pre-2.4, which is
>
>   def foo(): ...
>   foo = staticmethod(foo)

For the same reason that PEP 318 was written in the first place: to improve 
readability.


>This works across releases (including IronPython), doesn't require any
>magic, is documented, etc.

True; and most of my existing decorator-using code works this way, and I 
don't plan on rushing out to change any of it any time soon.


>   Isn't that much
>better than spending effort on hacks based on sys.settrace (which are
>surely going to produce bafflement from users who aren't familiar with
>that implementation hack)?

Well, up until recently, I thought the [] syntax was still a shoo-in and 
that the settrace hack was just going to be a way to write 
backwards-compatible code.

In the meantime, I've implemented a predicate-dispatch generic function 
system using the settrace hack for PyProtocols 1.0, and don't want to have 
to put off using it until 2.4.  (FWIW, I still support 2.2.2, so "until 
2.4" means maybe 1-2 years for me.)  It looks like this:

     [when("isinstance(x,Foo) and y>23")]
     def do_something(x,y):
          # code for when x is a Foo and y>23

     [when("isinstance(x,Bar) and y<99")]
     def do_something(x,y):
          # code for when x is a Bar and y<99

Rendered without any decorator syntax, it would have to be something like:

     def do_something_x_Foo_y_gt_23(x,y):
          # code for when x is a Foo and y>23

     do_something.add("isinstance(x,Foo) and y>23", do_something_x_Foo_y_gt_23)

     def do_something_x_Bar_y_lt_99(x,y):
          # code for when x is a Bar and y>99

     do_something.add("isinstance(x,Bar) and y>23", do_something_x_Bar_y_lt_99)

...which is a significant loss in readability.

Now, it's true that this will just become '@when("whatever")' in 2.4, and 
I'm *fine* with that, just disappointed that the syntax won't be forward 
compatible.

Indeed, what I'd rather focus on is making sure that the *semantics* will 
be forward compatible.  Which I think they will be.  What I'll probably do 
is make 'when()' and other decorators return a callable that deactivates 
the trace hack as a side effect of performing the decoration.  That way, 
[when()] and @when() can coexist in a code base during the migration period.

From jim at zope.com  Tue Aug  3 16:58:53 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Aug  3 16:58:56 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408031432.i73EWWV30829@guido.python.org>
References: <200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03
	EDT."	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408031432.i73EWWV30829@guido.python.org>
Message-ID: <410FA82D.8080502@zope.com>

Guido van Rossum wrote:
>>In practical terms, all this means is that I'll just use the hacked
>>syntax until I no longer need to support Python 2.3.
> 
> 
> I don't understand why you prefer your hack over the established way
> to do decorators pre-2.4, which is
> 
>   def foo(): ...
>   foo = staticmethod(foo)
> 
> This works across releases (including IronPython), doesn't require any
> magic, is documented, etc.  So if your main constraint is that it be
> implementable pre-2.4, you already have a solution.  Isn't that much
> better than spending effort on hacks based on sys.settrace (which are
> surely going to produce bafflement from users who aren't familiar with
> that implementation hack)?

I think this is a great point.  It makes me wonder if:

   @staticmethod
   def foo():
       ...

if sufficiently better than:

   def foo():
       ...
   foo = staticmethod(foo)

to justify the language change. FWIW, It isn't to me.
The new syntax is yet another rule that people have to know to
understand Python code they read. That's OK if it produces enough
value to justify the burden.  I question whether that's the case here.

Perhsps the difficulty in pickling an acceptable syntax should be taken as
a warning sign that there's a problem with the feature.

I wonder if perhaps the feature is too powerful.  In particular, the need to
support passing arguments to the descriptors seemes to add a lot of syntactic
burden. I think that the most compeling common applications of decorators don't
need to have decorator expressions.

For example, for the common cases that this is trying to solve, I'd be happy
to be able to say:

   def staticmethod foo():
       ....

and allow only names as decorators.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From arigo at tunes.org  Tue Aug  3 16:58:56 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue Aug  3 17:03:41 2004
Subject: [Python-Dev] Optimized string concatenation
In-Reply-To: <003101c47900$f23161e0$df30cb97@oemcomputer>
References: <1091536409.410f8619be6f0@mcherm.com>
	<003101c47900$f23161e0$df30cb97@oemcomputer>
Message-ID: <20040803145856.GA7070@vicky.ecs.soton.ac.uk>

Hello Raymond,

On Mon, Aug 02, 2004 at 10:24:03PM -0400, Raymond Hettinger wrote:
> The explanation is not hard.  The CPython implementation can concatenate
> inplace two term expressions of the form a=a+b or a+=b.  Expressions
> with more terms are not eligible for inplace concatenation.

No: a+=b+c is eligible.  That's my problem.


Armin
From guido at python.org  Tue Aug  3 17:23:16 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 17:23:20 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Tue, 03 Aug 2004 11:00:41 EDT."
	<5.1.1.6.0.20040803104400.01ea5590@mail.telecommunity.com> 
References: <Your message of "Tue, 03 Aug 2004 10:04:27 EDT."
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com> 
	<5.1.1.6.0.20040803104400.01ea5590@mail.telecommunity.com> 
Message-ID: <200408031523.i73FNGq31069@guido.python.org>

> Now, it's true that this will just become '@when("whatever")' in
> 2.4, and I'm *fine* with that, just disappointed that the syntax
> won't be forward compatible.

Implementability in 2.2 and 2.3 was never on the list of requirements
for decorators.  You're adding this constraint yourself.  I know it
was brought up before but I never agreed with that constraint (and the
previous community favorite, [...]-after-args, wasn't compatible
either).

I presume your framework is already full of hacks, which is fine by
me, but I don't want to use it as a deciding factor for the decorator
syntax.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug  3 17:24:46 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 17:24:52 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Tue, 03 Aug 2004 10:58:53 EDT."
	<410FA82D.8080502@zope.com> 
References: <200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408031432.i73EWWV30829@guido.python.org> 
	<410FA82D.8080502@zope.com> 
Message-ID: <200408031524.i73FOkM31082@guido.python.org>

> I think this is a great point.  It makes me wonder if:
> 
>    @staticmethod
>    def foo():
>        ...
> 
> if sufficiently better than:
> 
>    def foo():
>        ...
>    foo = staticmethod(foo)
> 
> to justify the language change. FWIW, It isn't to me.

Sigh.  This discussion is going around in pointless circles; we're
*months* past that point.  You're wasting your time (and mine).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug  3 17:27:26 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 17:27:31 2004
Subject: [Python-Dev] Optimized string concatenation
In-Reply-To: Your message of "Mon, 02 Aug 2004 22:48:31 EDT."
	<003201c47904$5cd353c0$df30cb97@oemcomputer> 
References: <003201c47904$5cd353c0$df30cb97@oemcomputer> 
Message-ID: <200408031527.i73FRQb31100@guido.python.org>

> I spent a *long* time reviewing and testing this patch.  The code is
> clean.  The concept works.  It delivers significant, measurable
> benefits.  The standard library itself has existing use cases.
> Every benchmark I tried showed results.  The patch was not approved
> prematurely!

Yes it was.  You know I am interested in this kind of decision
(otherwise you'd have just checked it in) so you should have left the
honor to me.

> > I am going to make up a new rule here, and state that
> > implementation features that affect not just the running speed but
> > the O() rate for certain algorithms should be considered language
> > features, because any implementation will be required to implement
> > them in order to ensure code portability.
> 
> Even list.sort() does not guarantee specific O() rates.  Currently,
> the fastest way to write a merge(a,b) is sort(a+b).  That relies on
> the current implementation recognizing that a and b are already
> sorted and doing a linear time concatenation.

That's a much more obscure case.  The string concatenation hack will
affect a significant fraction of all code.

> > Why do people want this so badly? 
> 
> Because the a=a+b form occurs everywhere, not just in newbie code.
> It is in the standard library, it shows up naturally in SAX, and it
> is often the most clear way to express an idea.

And in 99% of those cases nobody cares about performance.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From martin at v.loewis.de  Tue Aug  3 17:37:16 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug  3 17:37:17 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.311, 2.312
In-Reply-To: <200408031434.i73EYvv30846@guido.python.org>
References: <E1BrwSl-0005lx-7A@sc8-pr-cvs1.sourceforge.net>
	<200408031418.i73EIFR30699@guido.python.org>
	<410FA073.5070105@v.loewis.de>
	<200408031434.i73EYvv30846@guido.python.org>
Message-ID: <410FB12C.4010105@v.loewis.de>

Guido van Rossum wrote:
> Aha!  So maybe we should reconsider whether mwh's removal of the
> filename interning in the compiler should be reverted.

I just looked at the sizes of the four largest .pyc files wrt. to this
change (ie. the one generated before 2.312, and the one generated
after):

                  before            after
pydoc            84711             91263
cookielib        57353             57353
pickletools      55862             57038
tarfile          54074             58974

So the patch does cause a size increase (and likely also a slowdown
because of the extra memory allocations at startup). Whether this is
relevant or not, I don't know.

I think I would prefer if a different mechanism was found to account
for the change in references during an import, e.g. by taking the
number of interned strings before and after the import operation.

Regards,
Martin
From dave at boost-consulting.com  Tue Aug  3 17:44:51 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Tue Aug  3 17:45:21 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
References: <200408030058.i730wSQ29267@guido.python.org> <Your message of
	"Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408031432.i73EWWV30829@guido.python.org>
	<410FA82D.8080502@zope.com>
Message-ID: <u7jsgcjkc.fsf@boost-consulting.com>

Jim Fulton <jim@zope.com> writes:

> to justify the language change. FWIW, It isn't to me.  The new
> syntax is yet another rule that people have to know to understand
> Python code they read. That's OK if it produces enough value to
> justify the burden.  I question whether that's the case here.
>
> Perhsps the difficulty in pickling an acceptable syntax should be
> taken as a warning sign that there's a problem with the feature.

I just want to be clear that I know implementing my proposal involves
an ugly hack, and I'm not fond of it.  Nonetheless, I think it's
important.

I made the proposal in response to exactly the same instinct that Jim
is reflecting here.  If people really need a prefix syntax for
decorators it would be best to at least _start_ with an implementation
that doesn't involve core language changes, because the whole area
looks quite perilous from a language design point of view.

I'm saying without any desire to change Guido's mind about '@', but
just to explain the reasons that I brought up the idea in the first
place.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From jim at zope.com  Tue Aug  3 17:51:12 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Aug  3 17:51:15 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408031524.i73FOkM31082@guido.python.org>
References: <200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408031432.i73EWWV30829@guido.python.org>
	<410FA82D.8080502@zope.com>
	<200408031524.i73FOkM31082@guido.python.org>
Message-ID: <410FB470.7080709@zope.com>

Guido van Rossum wrote:
>>I think this is a great point.  It makes me wonder if:
>>
>>   @staticmethod
>>   def foo():
>>       ...
>>
>>if sufficiently better than:
>>
>>   def foo():
>>       ...
>>   foo = staticmethod(foo)
>>
>>to justify the language change. FWIW, It isn't to me.
> 
> 
> Sigh.  This discussion is going around in pointless circles; we're
> *months* past that point.  You're wasting your time (and mine).

Sorry.  I think you'd be wise to consider the months without consensus
as evidence of a problem. Don't bother wasting time with a reply. ;)

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From mwh at python.net  Tue Aug  3 18:01:18 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug  3 18:01:19 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.311, 2.312
In-Reply-To: <410FB12C.4010105@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Tue,
	03 Aug 2004 17:37:16 +0200")
References: <E1BrwSl-0005lx-7A@sc8-pr-cvs1.sourceforge.net>
	<200408031418.i73EIFR30699@guido.python.org>
	<410FA073.5070105@v.loewis.de>
	<200408031434.i73EYvv30846@guido.python.org>
	<410FB12C.4010105@v.loewis.de>
Message-ID: <2misc09po1.fsf@starship.python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> Guido van Rossum wrote:
>> Aha!  So maybe we should reconsider whether mwh's removal of the
>> filename interning in the compiler should be reverted.
>
> I just looked at the sizes of the four largest .pyc files wrt. to this
> change (ie. the one generated before 2.312, and the one generated
> after):
>
>                   before            after
> pydoc            84711             91263
> cookielib        57353             57353
> pickletools      55862             57038
> tarfile          54074             58974
>
> So the patch does cause a size increase (and likely also a slowdown
> because of the extra memory allocations at startup). Whether this is
> relevant or not, I don't know.

Me neither.  Are you sure you regenerated cookielib?

What would it cost to check if all strings could be stored via
TYPE_STRINGREF?  (Almost nothing in code terms...)

> I think I would prefer if a different mechanism was found to account
> for the change in references during an import, e.g. by taking the
> number of interned strings before and after the import operation.

Well, this isn't about import, it's more about execfile.  I had
entirely failed to twig that interning the filename might save space
in the .pyc, so didn't see any downsides to this particular
workaround.  I'd be happy with any other (I mentioned not doing
refcounting on dictobject.c's dummy; maybe I should actually see if
that's possible :-).

Cheers,
mwh

-- 
  The gripping hand is really that there are morons everywhere, it's
  just that the Americon morons are funnier than average.
                              -- Pim van Riezen, alt.sysadmin.recovery
From pinard at iro.umontreal.ca  Tue Aug  3 16:53:45 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Tue Aug  3 18:03:17 2004
Subject: [Python-Dev] Python in Unicode context
Message-ID: <20040803145345.GA6250@titan.progiciels-bpi.ca>

Hello, people.

I'm switching from ISO-8859-1 to UTF-8 in my locale, knowing it may
take a while before everything gets fully adapted.  Of course, I am
prepared to do whatever it means.  On my side at least, the perception
of a meaning is an evolving process. :-)

So, my goal here is to share some of difficulties I see with the current
setup of Python in Unicode context, under the hypothesis that Python
should ideally be designed to alleviate the pain of migration.  I hope
this is not out of context on the Python development list.

Converting a Python source file from ISO-8859-1 to UTF-8, back and forth
at the charset level, is a snap within Vim, and I would like if it was
(almost) a snap in the Python code as well.  There is some amount of
trickery that I could put in to achieve this, but too much trickery does
not fit well in usual Python elegance.

As Martin once put it, the ultimate goal is to convert data to Unicode
as early as possible in a Python program, and back to the locale as late
as possible.  While it's very OK with me, we should not loose sight that
people might adopt different approaches.

One thing is that a Python module should have some way to know the
encoding used in its source file, maybe some kind of `module.__coding__'
next to `module.__file__', saving the coding effectively used while
compilation was going on.  When a Python module is compiled, per PEP
0263 as I understand it, strings are logically converted to UTF-8 before
scanning, and produced str-strings (but not unicode-strings), converted
back to the original file coding.  When later, at runtime, the string
has to be converted back to Unicode, it would help if the programmer did
not have to hardwire the encoding in the program, and edit more than the
`coding:' cookie at the beginning if s/he ever switches file charset.
That same `module.__coding__' could also be used for other things, like
for example, to decide at run-time whether codecs streawriters should be
used or not.

Another solution would of course be to edit all strings, or at least
those containing non-ASCII characters, to prepend a `u' and turn
them into Unicode strings.  This is what I intend to do in practice.
However, all this editing is cumbersome, especially until it is
definitive.  I wonder if some other cookie, next to the `coding:'
cookie, could not be used to declare that all strings _in this module
only_ should be interpreted as Unicode by default, but without the need
of resorting to `u' prefix all over.  That would be weaker than the
`-U' switch on a Python call, but likely much more convenient as well.
As a corollary, maybe that some `s' prefix could force `str' type in a
Unicodized module.  Another way of saying it would be that an unadorned
string would have `s' or `u' implied, depending if the Unicode cookie is
missing or given at the start of a module.

I have the intuition, still unverified, but to be confirmed over time
and maybe discussions, that the above would alleviate transition to
Unicode, back and forth.


P.S. - Should I say and confess, one thing I do not like much about
Unicode is how proponents often perceive it, like a religion, and all
the fanatism going with it.  Unicode should be seen and implemented as
a choice, more than a life commitment :-). Right now, my feeling is that
Python asks a bit too much of a programmer, in terms of commitment, if
we only consider the editing work required on sources to use it, or not.

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard
From exarkun at divmod.com  Tue Aug  3 18:06:12 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Tue Aug  3 18:06:16 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408031404.i73E4cS30629@guido.python.org>
References: <Your message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<410F2453.7000506@divmod.com>
	<200408031404.i73E4cS30629@guido.python.org>
Message-ID: <410FB7F4.2070309@divmod.com>

Guido van Rossum wrote:
> [snip]
> 
> 
> I'm speechless.  If the ambiguous
> 
>     [classmethod]
>     def foo(x):
>         ...
> 
> is rejected because it doesn't look like it does something to foo, how
> come there's suddenly a crop of solutions that have the same problem
> being proposed?  What you write looks like a call to the function
> decorate(), followed by a function method definition.  The
> "action-at-a-distance" that is presumed by the decorate() call is
> difficult to explain and a precedent for other worse hacks.  Its only
> point in favor seems to be that it doesn't use '@'.

   In my view, the strongest point in favor of a solution that involves 
calling functions rather than changing syntax is that the functions 
involved can be placed in the standard library rather than the 
interpreter.

   I believe a widely held view is that features can be supported by the 
stdlib do not merit language changes?

   Moreover, I have the impression that many people are clamoring for 
this feature, no matter how it ends up looking, because they simply 
*must have it*.  Well, if they must have it, why wait for 2.4, when 
clearly they can have it right now?

   Jp
From jhylton at gmail.com  Tue Aug  3 18:09:40 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Tue Aug  3 18:09:44 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408031524.i73FOkM31082@guido.python.org>
References: <200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408031432.i73EWWV30829@guido.python.org> 
	<410FA82D.8080502@zope.com>
	<200408031524.i73FOkM31082@guido.python.org>
Message-ID: <e8bf7a5304080309094f3a44f7@mail.gmail.com>

On Tue, 03 Aug 2004 08:24:46 -0700, Guido van Rossum <guido@python.org> wrote:
> Sigh.  This discussion is going around in pointless circles; we're
> *months* past that point.  You're wasting your time (and mine).

Is there a PEP that summarizes the current state of the decorators
design? One of the goals of PEPs was to prevent design discussions
from going in circles.  Another goal was to keep core developers
abreast of changes without having to read every message on python-dev.

As you mentioned in an earlier email, I think the work on 2.4 would
have been smoother if we had used the PEP process more effectively. 
We never had a decent PEP for the decorator syntax discussion.  PEP
318 describes a particular design choice, but it's weak on rationale. 
There's no PEP that summarizes the various syntax options and the
reason for the eventual choice.

Facundo's PEP for decimal arithmetic (PEP 327) looks like an example
of the right thing.  I haven't followed the work closely, but it looks
like the PEP describes the current state of the design and the
rationale for it.  The generator PEP is in better shape than it used
to be.

Jeremy
From tim.peters at gmail.com  Tue Aug  3 18:15:07 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug  3 18:15:09 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410FA82D.8080502@zope.com>
References: <200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03
	EDT."	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408031432.i73EWWV30829@guido.python.org>
	<410FA82D.8080502@zope.com>
Message-ID: <1f7befae04080309153c7ac193@mail.gmail.com>

I want to jump into this debate, but have stayed out of it so far and
just don't have time for it.  But I can't let this one pass!

[Jim Fulton]
> ...
> Perhsps the difficulty in pickling an acceptable syntax should be taken as
> a warning sign that there's a problem with the feature.

There is, in fact, no such difficulty with any of these syntaxes: 
I've tried pickling all of them, under both pickle.py and cPickle, and
they both pickle and unpickle fine.  I think this has to do with that
they're really all just strings <wink>.
From guido at python.org  Tue Aug  3 18:23:44 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 18:23:58 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.311, 2.312
In-Reply-To: Your message of "Tue, 03 Aug 2004 17:37:16 +0200."
	<410FB12C.4010105@v.loewis.de> 
References: <E1BrwSl-0005lx-7A@sc8-pr-cvs1.sourceforge.net>
	<200408031418.i73EIFR30699@guido.python.org>
	<410FA073.5070105@v.loewis.de>
	<200408031434.i73EYvv30846@guido.python.org> 
	<410FB12C.4010105@v.loewis.de> 
Message-ID: <200408031623.i73GNiL31284@guido.python.org>

> > Aha!  So maybe we should reconsider whether mwh's removal of the
> > filename interning in the compiler should be reverted.
> 
> I just looked at the sizes of the four largest .pyc files wrt. to this
> change (ie. the one generated before 2.312, and the one generated
> after):
> 
>                   before            after
> pydoc            84711             91263
> cookielib        57353             57353
> pickletools      55862             57038
> tarfile          54074             58974

Are you sure the cookielib numbers are correct?  They show no
difference, which would only make sense if there was only a single
code object.

> So the patch does cause a size increase (and likely also a slowdown
> because of the extra memory allocations at startup). Whether this is
> relevant or not, I don't know.

Based on this data I'd like to see the change reverted; the motivation
for the change was debuggability of leaks, not a real problem with the
changed code.

> I think I would prefer if a different mechanism was found to account
> for the change in references during an import, e.g. by taking the
> number of interned strings before and after the import operation.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From astrand at lysator.liu.se  Tue Aug  3 17:22:35 2004
From: astrand at lysator.liu.se (Peter Astrand)
Date: Tue Aug  3 18:25:59 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <200408031427.i73ERn330790@guido.python.org>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net> 
	<200408031427.i73ERn330790@guido.python.org>
Message-ID: <Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>


> I believe ActiveState's Trent Mick has released something along these
> lines.  Does your proposed API match theirs?

No, not at all. There are some issues with this module, both regards the
API and the implementation, which I'm not happy with. I tried to establish
a discussion with Trent about this, but if I remember correctly, I never
recieved a response.

Here's some things which I'm not happy with:

* There are three different "factory" classes: Process, ProcessOpen,
ProcessProxy. I have only one, covering all cases. I think three different
classes are confusing to users. Even I have trouble understanding the
difference between them.

* The code size is very large. The code is complex.

* Trent's module always executes things through the shell, and deals with
every ugly cornercase that comes from this.

* The modules uses destructors, which I'm usually avoiding.


So, the chances of getting our modules API-compatible are very small.


/Peter ?strand <astrand@lysator.liu.se>

From mwh at python.net  Tue Aug  3 18:29:17 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug  3 18:29:19 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test
 test_threadsignals.py, 1.2, 1.3
In-Reply-To: <E1Bs1vc-0004pz-9R@sc8-pr-cvs1.sourceforge.net>
	(fdrake@users.sourceforge.net's
	message of "Tue, 03 Aug 2004 09:14:16 -0700")
References: <E1Bs1vc-0004pz-9R@sc8-pr-cvs1.sourceforge.net>
Message-ID: <2mekmo9ode.fsf@starship.python.net>

fdrake@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Lib/test
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18587
>
> Modified Files:
> 	test_threadsignals.py 
> Log Message:
> add missing import!

Oops, sorry!

Cheers,
mwh

-- 
  But maybe I've just programmed in enough different languages to
  assume that they are, in fact, different.
     -- Tony J Ibbs explains why Python isn't Java on comp.lang.python
From pje at telecommunity.com  Tue Aug  3 18:34:16 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug  3 18:30:19 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408031523.i73FNGq31069@guido.python.org>
References: <Your message of "Tue, 03 Aug 2004 11:00:41 EDT."
	<5.1.1.6.0.20040803104400.01ea5590@mail.telecommunity.com>
	<Your message of "Tue, 03 Aug 2004 10:04:27 EDT."
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<5.1.1.6.0.20040803104400.01ea5590@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040803113843.03190e60@mail.telecommunity.com>

At 08:23 AM 8/3/04 -0700, Guido van Rossum wrote:
> > Now, it's true that this will just become '@when("whatever")' in
> > 2.4, and I'm *fine* with that, just disappointed that the syntax
> > won't be forward compatible.
>
>Implementability in 2.2 and 2.3 was never on the list of requirements
>for decorators.  You're adding this constraint yourself.  I know it
>was brought up before but I never agreed with that constraint (and the
>previous community favorite, [...]-after-args, wasn't compatible
>either).

All understood and agreed.  I was only trying to answer the question that 
you asked; I was emphatically *not* trying to reopen the discussion.  My 
"disappointment" here is on a par with being disappointed that a restaurant 
is out of my favorite dish: a trifling matter that doesn't deserve the time 
I've already spent on it, except that I hate being judged poorly for 
something that I didn't actually *do*.  You seem to think that I'm saying 
"@" is a bad thing, or that "[]" is better for the language or community 
than "@".  But I'm most emphatically *not* saying those things, and don't 
believe I ever have.


>I presume your framework is already full of hacks,

Well, I don't know if I'd call 4 hacks "full", especially since two of 
those are to implement decorators (class level and function level).  The 
other two are to implement 'call-next-method' for generic functions, and to 
implement microthreads with generators.  Out of nearly ninety thousand of 
lines of Python, I don't think that "full" is an accurate characterization 
here.


>  which is fine by
>me, but I don't want to use it as a deciding factor for the decorator
>syntax.

I'm not asking you to, and I don't believe I ever have.  I only replied to 
J.P. Calderone's suggestion of a less-functional hack (i.e. if you're going 
to hack, you might as well make it do the whole thing), and to *your* 
emails inquiring why I and other people thought a call-oriented syntax was 
better than bare "[]".  With regard to the actual language change, I 
conceded the point when you said "let's move on" the other day.

I understand your frustration with this issue, and I can also understand 
that you are probably feeling pretty second-guessed right now, but *I'm* 
not second-guessing you!  So there's no call for the Spanish Inquisition 
here, and certainly not for personal insults because I answered questions 
that *you* asked.  (The Python standard library actually has *six* modules 
or packages that use sys._getframe(), not including tests; shall we call it 
"full of hacks", too?)

From guido at python.org  Tue Aug  3 18:26:57 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 18:37:09 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: Your message of "Tue, 03 Aug 2004 09:44:51 MDT."
	<u7jsgcjkc.fsf@boost-consulting.com> 
References: <200408030058.i730wSQ29267@guido.python.org> <Your message of "Mon,
	02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<5.1.1.6.0.20040803094749.01ea3020@mail.telecommunity.com>
	<200408031432.i73EWWV30829@guido.python.org>
	<410FA82D.8080502@zope.com> <u7jsgcjkc.fsf@boost-consulting.com> 
Message-ID: <200408031626.i73GQvO31298@guido.python.org>

> I made the proposal in response to exactly the same instinct that Jim
> is reflecting here.  If people really need a prefix syntax for
> decorators it would be best to at least _start_ with an implementation
> that doesn't involve core language changes, because the whole area
> looks quite perilous from a language design point of view.

We *had* an implementation without core language changes, but it was a
postfix syntax.

A prefix syntax should be designed with future usability in mind
exclusively, unconstrained by "implementability without language
changes".  That's a much more stifling requirement than the
requirement that existing code shouldn't be broken (which is stifling
enough in itself but IMO unavoidable).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tismer at stackless.com  Tue Aug  3 18:51:12 2004
From: tismer at stackless.com (Christian Tismer)
Date: Tue Aug  3 18:51:27 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae0408022027465a8789@mail.gmail.com>
References: <410EF570.3010901@stackless.com>
	<200408020137.i721bgvc026118@cosc353.cosc.canterbury.ac.nz>
	<200408021840.i72Iejt28600@guido.python.org>
	<410EF570.3010901@stackless.com>
	<5.1.1.6.0.20040802231932.0318c6a0@mail.telecommunity.com>
	<1f7befae0408022027465a8789@mail.gmail.com>
Message-ID: <410FC280.9060008@stackless.com>

Tim Peters wrote:

> [Phillip J. Eby]
> 
>>It's worse than that...  'ihooks' and 'imputil', for example, both do:
>>
>>    exec code in module.__dict__
>>
>>so they'd need to be changed to support this fix as well.  (Not to mention
>>any third-party import hooks...)
> 
> 
> Ouch.  For that matter, they also avoid the "choke point" Sunday
> night's patch relied on.  So there are actually no choke points for
> any part of the import process now.  Maybe after each bytecode we
> could make the eval loop look to see whether sys.modules had been
> changed <wink>.

:-(

Life is so cruel...

I agree it is a little harder, ahem :-)

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From martin at v.loewis.de  Tue Aug  3 19:00:17 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug  3 19:00:15 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.311, 2.312
In-Reply-To: <200408031623.i73GNiL31284@guido.python.org>
References: <E1BrwSl-0005lx-7A@sc8-pr-cvs1.sourceforge.net>
	<200408031418.i73EIFR30699@guido.python.org>
	<410FA073.5070105@v.loewis.de>
	<200408031434.i73EYvv30846@guido.python.org>
	<410FB12C.4010105@v.loewis.de>
	<200408031623.i73GNiL31284@guido.python.org>
Message-ID: <410FC4A1.7000409@v.loewis.de>

Guido van Rossum wrote:
> Are you sure the cookielib numbers are correct?  They show no
> difference, which would only make sense if there was only a single
> code object.

No. Turns out I did not have "old" bytecode for cookielib.

Regards,
Martin
From guido at python.org  Tue Aug  3 19:00:22 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 19:00:28 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: Your message of "Tue, 03 Aug 2004 12:06:12 EDT."
	<410FB7F4.2070309@divmod.com> 
References: <Your message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<410F2453.7000506@divmod.com>
	<200408031404.i73E4cS30629@guido.python.org> 
	<410FB7F4.2070309@divmod.com> 
Message-ID: <200408031700.i73H0MY31391@guido.python.org>

> In my view, the strongest point in favor of a solution that involves
> calling functions rather than changing syntax is that the functions
> involved can be placed in the standard library rather than the
> interpreter.
> 
> I believe a widely held view is that features can be supported by
> the stdlib do not merit language changes?

No viewpoint ought to be held to the exclusion of all others.  Using
functions with "action-at-a-distance" through sys.settraceback may be
okay for an obscure feature that can't be had any other way yet
doesn't merit changes to the language, but that's not the situation
for decorators.  The widely held view here is that decorators need to
be added as a syntactic feature to avoid the problems with the postfix
notation used in 2.2 and 2.3.  Decorators are slated to be an
important new language feature and their design needs to be
forward-looking, not constrained by what can be implemented in 2.3.

One important feature of @deco (and of [deco] prefix as I proposed it)
is that it is a compile-time feature.  This has all sorts of
advantages, since future compilers and tools will be able to know more
about methods.  Solutions based on sys.settrace fail badly here.

> Moreover, I have the impression that many people are clamoring for
> this feature, no matter how it ends up looking, because they simply
> *must have it*.  Well, if they must have it, why wait for 2.4, when
> clearly they can have it right now?

Sure, but that's not an argument against improving their situation in
2.4.

If I had any hope that something better than @deco was possible but
hasn't been discovered yet, I'd be holding off too.  But I don't think
that's the case.  Every solution proposed has its own set of
drawbacks, and the drawbacks aren't getting resolved by waiting.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From mcherm at mcherm.com  Tue Aug  3 19:07:45 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Aug  3 19:07:09 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
Message-ID: <1091552865.410fc661b4abd@mcherm.com>

Guido writes:
> I believe ActiveState's Trent Mick has released something
> along these lines.  Does your proposed API match theirs?

Peter replies:
   [...]
> So, the chances of getting our modules API-compatible are very small.

I'll note that my impression is that Peter's module is not aiming to
be particularly innovative in terms of implementation, it's primary
"feature" is having a decent API. And my impression is that it has
been well received on c.l.py (for instance, I often see it recommended
in response to newbie questions). He got a lot of input during the PEP
process and tried to incorporate it.

Last time I tried it there were still a few windows issues (docs say
these have been resolved), but I thought the API was well-designed.
And popen2 isn't.

-- Michael Chermside

From martin at v.loewis.de  Tue Aug  3 19:12:44 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug  3 19:12:43 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Python	compile.c, 2.311, 2.312
In-Reply-To: <2misc09po1.fsf@starship.python.net>
References: <E1BrwSl-0005lx-7A@sc8-pr-cvs1.sourceforge.net>	<200408031418.i73EIFR30699@guido.python.org>	<410FA073.5070105@v.loewis.de>	<200408031434.i73EYvv30846@guido.python.org>	<410FB12C.4010105@v.loewis.de>
	<2misc09po1.fsf@starship.python.net>
Message-ID: <410FC78C.4010609@v.loewis.de>

Michael Hudson wrote:
> Me neither.  Are you sure you regenerated cookielib?

Oops, no. Or, rather, yes - even the first number is already
regenerated.

> What would it cost to check if all strings could be stored via
> TYPE_STRINGREF?  (Almost nothing in code terms...)

It would cause additional dictionary lookups on marshalling, which
in turn would cause hash computation for all strings. Currently,
as only interned strings are marshalled through TYPE_STRINGREF,
we know that hash values are already computed for all of them,
so we only pay a dict lookup per string (and perhaps an allocation
if the string is new and the dict needs resizing).

In addition, there is a slight semantical change, in that equal
strings become shared on unmarshalling. I don't think this is relevant,
though, since marshal never worried about shared strings in the first
place.

> I'd be happy with any other (I mentioned not doing
> refcounting on dictobject.c's dummy; maybe I should actually see if
> that's possible :-).

Feel free to change the marshalling to try sharing all strings;
it might be easier to revert the change to compile.c, though.

Regards,
Martin
From mcherm at mcherm.com  Tue Aug  3 19:22:42 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Aug  3 19:22:06 2004
Subject: [Python-Dev] 2.4a2, and @decorators
Message-ID: <1091553762.410fc9e2ef458@mcherm.com>

James Knight writes:
> However, it seems as if this restriction creates a new class of
> expression for no gain. It is true enough that *most* python
> expressions aren't useful to write after a @, but that's also
> true of a
> lot of other places an expression can be used (e.g. before a () of a
> function call. A list comprehension can never result in a callable
> object. An arithmetic operation usually won't.).
>
> The only real necessary restriction on the @ operator is that its
> argument be callable and take a single argument. Many
> expressions could
> return a callable object. Why not let them? Is it really
> worth having a
> special case just to SyntaxError expressions that sometimes won't
> result in an appropriate callable?
>
> Things someone might want to do, ordered roughly from most reasonable
> to least reasonable ;)
> @foo().bar()
> @foo or bar
> @mydecorators['foo']
> @lambda f: foo(f) or bar(f)
>
> Why disallow these forms? It seems quite difficult, especially, to
> explain why the first one does not, or should not, work.


Yeah... what he said.

Seriously, a well-designed programming language has very strong
orthogonality. A few simple concepts should suffice. "an expression"
is an existing production which is well-known to users and well
understood. I agree that using complex expressions in a decorator
leads to difficult-to-read code, but I would STILL like it to be
possible, just so I don't have to remember (or teach) decorators as
a special case.

-- Michael Chermside
From exarkun at divmod.com  Tue Aug  3 19:23:10 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Tue Aug  3 19:23:15 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408031700.i73H0MY31391@guido.python.org>
References: <Your message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<410F2453.7000506@divmod.com>
	<200408031404.i73E4cS30629@guido.python.org>
	<410FB7F4.2070309@divmod.com>
	<200408031700.i73H0MY31391@guido.python.org>
Message-ID: <410FC9FE.6060305@divmod.com>

Guido van Rossum wrote:
> 
> No viewpoint ought to be held to the exclusion of all others.  Using
> functions with "action-at-a-distance" through sys.settraceback may be
> okay for an obscure feature that can't be had any other way yet
> doesn't merit changes to the language, but that's not the situation
> for decorators.  The widely held view here is that decorators need to
> be added as a syntactic feature to avoid the problems with the postfix
> notation used in 2.2 and 2.3.  Decorators are slated to be an
> important new language feature and their design needs to be
> forward-looking, not constrained by what can be implemented in 2.3.

   Just to be clear, the implementation I shared did not use 
sys.settrace().  It did rely on inspect.currentframe(), but so do a 
handful of other stdlib modules (as PJE pointed out).

> 
> One important feature of @deco (and of [deco] prefix as I proposed it)
> is that it is a compile-time feature.  This has all sorts of
> advantages, since future compilers and tools will be able to know more
> about methods.  Solutions based on sys.settrace fail badly here.
> 
> 
>>Moreover, I have the impression that many people are clamoring for
>>this feature, no matter how it ends up looking, because they simply
>>*must have it*.  Well, if they must have it, why wait for 2.4, when
>>clearly they can have it right now?
> 
> 
> Sure, but that's not an argument against improving their situation in
> 2.4.

   If it is truly an improvement.  Here we (and many others) disagree.

   I like Python(*) because it is simple and generally unsurprising.  I 
think that '@deco' adds complexity, both for beginners and experienced 
programmers alike, so I do not see it as an improvement.  I am content 
with "foo = deco(foo)" and various metaclass-based automatic 
decorations, and see no need for them to be moved into the core.  I may 
be free to not use the '@deco' syntax in 2.4, but that will not prevent 
me from having to read it in the code of others, or from explaining it 
to other Python programmers.

   None of these are new points, so I don't expect a response to them.

> 
> If I had any hope that something better than @deco was possible but
> hasn't been discovered yet, I'd be holding off too.  But I don't think
> that's the case.  Every solution proposed has its own set of
> drawbacks, and the drawbacks aren't getting resolved by waiting.

   How many features that everyone hates can Python support?

   The Zen of Python might state that practicality beats purity, but if 
it continues to do so, time after time, at some point we're going to 
look up and wonder where that clean, readable, maintainable programming 
language we all loved has gone.

   Jp

  (*) And I really do like Python.  A lot.  Of the software I now write, 
at least 99.9% of it is in Python. The developers of this language 
_almost_ always show more good sense, intelligence, and skill than most 
other groups I can think of.  You all deserve more praise than you're given.
From martin at v.loewis.de  Tue Aug  3 19:24:11 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug  3 19:24:11 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <20040803145345.GA6250@titan.progiciels-bpi.ca>
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
Message-ID: <410FCA3B.1020002@v.loewis.de>

Fran?ois Pinard wrote:
> One thing is that a Python module should have some way to know the
> encoding used in its source file, maybe some kind of `module.__coding__'
> next to `module.__file__', saving the coding effectively used while
> compilation was going on. 

That would be possible to implement. Feel free to create a patch.

> I wonder if some other cookie, next to the `coding:'
> cookie, could not be used to declare that all strings _in this module
> only_ should be interpreted as Unicode by default, but without the need
> of resorting to `u' prefix all over.

This could be a starting point of another syntax debate. For example,

from __future__ import string_literals_are_unicode

would be possible to implement. If PEP 244 would have been adapted, I
would have proposed

directive unicode_strings

Other syntax forms would also be possible. Again, if you know a syntax
which you like, propose a patch. Be prepared to also write a PEP 
defending that syntax.

> P.S. - Should I say and confess, one thing I do not like much about
> Unicode is how proponents often perceive it, like a religion, and all
> the fanatism going with it.  Unicode should be seen and implemented as
> a choice, more than a life commitment :-). Right now, my feeling is that
> Python asks a bit too much of a programmer, in terms of commitment, if
> we only consider the editing work required on sources to use it, or not.

Not sure what you are referring here to. You do have the choice of
source encodings, and, in fact, "Unicode" is not a valid source
encoding. "UTF-8" is, and from a Python point of view, there is
absolutely no difference between that and, say, "ISO-8859-15" -
you can choose whatever source encoding you like, and Python does
not favour any of them (strictly speaking, it favour ASCII, then
ISO-8859-1, then the rest).

Choice of source encoding is different from the choice of string
literals. You can use Unicode strings, or byte strings, or mix them.
It really is your choice.

Regards,
Martin
From DavidA at ActiveState.com  Tue Aug  3 19:27:56 2004
From: DavidA at ActiveState.com (David Ascher)
Date: Tue Aug  3 19:28:02 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
Message-ID: <410FCB1C.3080009@ActiveState.com>

Peter Astrand wrote:

>>I believe ActiveState's Trent Mick has released something along these
>>lines.  Does your proposed API match theirs?

[...]

> So, the chances of getting our modules API-compatible are very small.

I really don't want to get into a religious war about the relative 
benefits of the two modules (I don't know Peter's well enough to 
comment), but I think it would be nice for users if there weren't two 
modules with the same goal, different interfaces and the same _name_.  I 
understand from Trent that Twisted has started to use his process.py, so 
having another process.py in the stdlib at some point would cause at 
least some users pain.

--david

From mal at egenix.com  Tue Aug  3 19:35:59 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue Aug  3 19:36:17 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <410FCA3B.1020002@v.loewis.de>
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de>
Message-ID: <410FCCFF.4050806@egenix.com>

Martin v. L?wis wrote:
> Fran?ois Pinard wrote:
> 
>> One thing is that a Python module should have some way to know the
>> encoding used in its source file, maybe some kind of `module.__coding__'
>> next to `module.__file__', saving the coding effectively used while
>> compilation was going on. 
> 
> That would be possible to implement. Feel free to create a patch.

+1

>> I wonder if some other cookie, next to the `coding:'
>> cookie, could not be used to declare that all strings _in this module
>> only_ should be interpreted as Unicode by default, but without the need
>> of resorting to `u' prefix all over.
> 
> 
> This could be a starting point of another syntax debate. For example,
> 
> from __future__ import string_literals_are_unicode
> 
> would be possible to implement. If PEP 244 would have been adapted, I
> would have proposed
> 
> directive unicode_strings
> 
> Other syntax forms would also be possible. Again, if you know a syntax
> which you like, propose a patch. Be prepared to also write a PEP 
> defending that syntax.

+1

Things that have been proposed earlier on, extended a bit:

b'xxx' - return a buffer to hold binary data; same
          as buffer(s'abc')

s'abc' - (forced) 8-bit string literal in source code encoding

u'abc' - (forced) Unicode literal

'abc'  - maps to s'abc' per default, can map to u'abc' based
          on the command line switch -U or a module switch

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 03 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From adamsz at gmail.com  Tue Aug  3 19:46:49 2004
From: adamsz at gmail.com (Adam Souzis)
Date: Tue Aug  3 19:46:55 2004
Subject: [Python-Dev] Optimized string concatenation
In-Reply-To: <200408031527.i73FRQb31100@guido.python.org>
References: <003201c47904$5cd353c0$df30cb97@oemcomputer>
	<200408031527.i73FRQb31100@guido.python.org>
Message-ID: <d7cee4a70408031046ebfbf2b@mail.gmail.com>

> > > Why do people want this so badly?

As someone relatively new to python, it struck me as a language wart
that i had to learn the form '"".join() as the proper way to do string
concatenation. It violates the principles of OOWTI and its certainly
not the obvious way to do string concatenation. This patch does not
cover all the cases so we're still stuck with join(), but as long as
it is not a documentated "feature" it will harmlessly improve the
performance of countless lines of code where the coder is either
untrained or too lazy to use join().  If its documented it'd just
muddy the waters vis a vis join(), besides the issues with other
Python implementation mentioned here.

-- adam

On Tue, 03 Aug 2004 08:27:26 -0700, Guido van Rossum <guido@python.org> wrote:
> > I spent a *long* time reviewing and testing this patch.  The code is
> > clean.  The concept works.  It delivers significant, measurable
> > benefits.  The standard library itself has existing use cases.
> > Every benchmark I tried showed results.  The patch was not approved
> > prematurely!
> 
> Yes it was.  You know I am interested in this kind of decision
> (otherwise you'd have just checked it in) so you should have left the
> honor to me.
> 
> > > I am going to make up a new rule here, and state that
> > > implementation features that affect not just the running speed but
> > > the O() rate for certain algorithms should be considered language
> > > features, because any implementation will be required to implement
> > > them in order to ensure code portability.
> >
> > Even list.sort() does not guarantee specific O() rates.  Currently,
> > the fastest way to write a merge(a,b) is sort(a+b).  That relies on
> > the current implementation recognizing that a and b are already
> > sorted and doing a linear time concatenation.
> 
> That's a much more obscure case.  The string concatenation hack will
> affect a significant fraction of all code.
> 
> > > Why do people want this so badly?
> >
> > Because the a=a+b form occurs everywhere, not just in newbie code.
> > It is in the standard library, it shows up naturally in SAX, and it
> > is often the most clear way to express an idea.
> 
> And in 99% of those cases nobody cares about performance.
> 
> 
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/adamsz%40gmail.com
>
From astrand at lysator.liu.se  Tue Aug  3 20:01:23 2004
From: astrand at lysator.liu.se (Peter Astrand)
Date: Tue Aug  3 20:01:34 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <410FCB1C.3080009@ActiveState.com>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net> 
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<410FCB1C.3080009@ActiveState.com>
Message-ID: <Pine.GSO.4.51L2.0408031947451.9769@koeberg.lysator.liu.se>

> > So, the chances of getting our modules API-compatible are very small.
>
> I really don't want to get into a religious war about the relative
> benefits of the two modules (I don't know Peter's well enough to
> comment), but I think it would be nice for users if there weren't two
> modules with the same goal, different interfaces and the same _name_.  I

Yes, this could be a problem, but no-one has been able to come up with a
solution: After 6 months, "process" and "popen" was the only suggested
names.

How about "subprocess"?

/Peter ?strand <astrand@lysator.liu.se>

From barry at python.org  Tue Aug  3 20:01:42 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug  3 20:01:39 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408030351.i733pcO29560@guido.python.org>
References: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz>
	<200408030351.i733pcO29560@guido.python.org>
Message-ID: <1091556102.7597.26.camel@localhost>

On Mon, 2004-08-02 at 23:51, Guido van Rossum wrote:

> And yes, I'm not 100% comfortable with it either.  But I know one
> thing: in this particular case, now is better than never.  So I'm
> picking a solution everybody can hate equally.
> 
> (What's wrong with pear-shaped, anyway?  It's one of my favorite
> shapes. :-)

So I conducted a little experiment, converting a bunch of modules that
were using Python 2.3 compatible "post-processing" decorator syntax, to
pie decorator syntax, so that I could get a more visceral reaction to
the decision.  For me, the short answer is: +1, and not just because
"it's better than nothing".

For longer functions, it's a clear win; the '@' stands out nicely,
especially with the python-mode.el font-locking support I just added. :)
It certainly exhibits all the benefits that motivated adding special
syntax for decorators in the first place.

For shorter functions, e.g. read-only accessors, my only (very minor)
lament is that I can't compact the enter thing down to a single line. 
My fingers want to do something like:

	@property def get_bacon(self): return self._bacon

but then I tell my fingers to do this instead:

	@property
	def get_bacon(self):
		return self._bacon

Or, we'll compromise, my fingers and I, on:

	get_bacon = property(lambda self: self._bacon)

So it's at least a wash there.  (Really, pie decorators are better here
too, IMO.)

My only other minor lament has nothing to do with this specific syntax. 
I still wish there was a way to decorate multi-arg property()'s, but oh
well.

All in all, I'm quite happy with this syntax.  Thanks Guido!
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040803/d88a456e/attachment.pgp
From pje at telecommunity.com  Tue Aug  3 20:11:26 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug  3 20:07:41 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <1091556102.7597.26.camel@localhost>
References: <200408030351.i733pcO29560@guido.python.org>
	<200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz>
	<200408030351.i733pcO29560@guido.python.org>
Message-ID: <5.1.1.6.0.20040803140853.034c46f0@mail.telecommunity.com>

At 02:01 PM 8/3/04 -0400, Barry Warsaw wrote:

>My only other minor lament has nothing to do with this specific syntax.
>I still wish there was a way to decorate multi-arg property()'s, but oh
>well.

     @property_getter
     def foo(self):
         ...

     @property_setter
     def foo(self,value):
         ...


See Ed Loper's post at:

    http://mail.python.org/pipermail/python-dev/2004-April/043902.html

and implementation at:

    http://www.cis.upenn.edu/~edloper/pydecorators.html

From astrand at lysator.liu.se  Tue Aug  3 20:09:29 2004
From: astrand at lysator.liu.se (Peter Astrand)
Date: Tue Aug  3 20:11:06 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <1091552865.410fc661b4abd@mcherm.com>
References: <1091552865.410fc661b4abd@mcherm.com>
Message-ID: <Pine.GSO.4.51L2.0408031935450.9769@koeberg.lysator.liu.se>


> Last time I tried it there were still a few windows issues (docs say
> these have been resolved), but I thought the API was well-designed.

The details: The module works great on Windows currently, but requires the
win32all extensions. The idea is to get rid of this dependency by writing
a glue module, pretty much as _winreg.

Work on such a _process module has been done on two fronts:

* A friendly Python-developer has gotten quite far, but was experiencing
some strange hangs. I haven't been able to look at this work yet.

* I've started working on an implementation myself. I was using _winreg as
some sort of template. The problem is: I've never written extension
modules before :-) It was quite easy to export the numeric constants, but
when I came to creating Python objects, I was lost. I'll guess I need to
read some documentation.

This is what remains:

win32api:
 GetStdHandle
 GetCurrentProcess, DuplicateHandle

win32pipe:
 CreatePipe

win32process:
 CreateProcess, STARTUPINFO, GetExitCodeProcess

win32event:
 WaitForSingleObject


Here's the module so far:


/*
  _process.c
*/

#include "windows.h"
#include "Python.h"
#include "structmember.h"
#include "malloc.h" /* for alloca */

/* The win32api module reports the function name that failed,
   but this concept is not in the Python core.
   Hopefully it will one day, and in the meantime I dont
   want to lose this info...
*/
#define PyErr_SetFromWindowsErrWithFunction(rc, fnname) \
	PyErr_SetFromWindowsErr(rc)

/* Forward declares */

/* Doc strings */
// FIXME
PyDoc_STRVAR(module_doc,
"This module provides...\n");


static struct PyMethodDef process_methods[] = {
	NULL,
};

static void
insint(PyObject * d, char * name, long value)
{
	PyObject *v = PyInt_FromLong(value);
	if (!v || PyDict_SetItemString(d, name, v))
		PyErr_Clear();
	Py_XDECREF(v);
}

#define ADD_INT(val) insint(d, #val, val)


PyMODINIT_FUNC init_process(void)
{
	PyObject *m, *d;
	m = Py_InitModule3("_process", process_methods, module_doc);
	d = PyModule_GetDict(m);
	Py_INCREF(PyExc_WindowsError);
	if (PyDict_SetItemString(d, "error",
				 PyExc_WindowsError) != 0)
		return;

	/* Add the relevant constants */
	ADD_INT(DUPLICATE_SAME_ACCESS);
	ADD_INT(STARTF_USESTDHANDLES);
	ADD_INT(STD_INPUT_HANDLE);
	ADD_INT(STD_OUTPUT_HANDLE);
        ADD_INT(STD_ERROR_HANDLE);
	ADD_INT(INFINITE);
	ADD_INT(WAIT_OBJECT_0);
}

(I will be away for a couple of days.)

/Peter ?strand <astrand@lysator.liu.se>

From barry at python.org  Tue Aug  3 20:22:43 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug  3 20:22:38 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <1091531214.1502.20.camel@localhost>
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
	<1091531214.1502.20.camel@localhost>
Message-ID: <1091557363.7562.38.camel@localhost>

On Tue, 2004-08-03 at 07:06, Mark Russell wrote:

> I would say leave it like this for now.  It is easy to relax the
> restriction later if there's a demand.

+1
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040803/4e411a24/attachment-0001.pgp
From eppstein at ics.uci.edu  Tue Aug  3 20:52:22 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Tue Aug  3 20:52:29 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
	<1091531214.1502.20.camel@localhost>
	<1091557363.7562.38.camel@localhost>
Message-ID: <eppstein-EE1115.11522203082004@sea.gmane.org>

BTW, here's another use case I haven't seen suggested yet:

    def substitution_for(regexp):
        sub = re.compile(regexp).sub
        return lambda func: lambda text: sub(func,text)

e.g. (example from some recent code of mine, redone as a decorator)

    @substitution_for('[a-z][A-Z]')
    def camelspace(m):
        """Insert space into camelcase regexp match."""
        return m.group(0)[0]+' '+m.group(0)[1:]

    ...

    info.title = camelspace(info.basefilename)

Instead my current code defines a separate variable for the 
re.compile().sub object, then later has to put this variable together 
correctly with the undecorated camelspace function to perform the 
substitution.  Using decorators allows the sub and replacement function 
to become merged into a single object with a more intuitive API.
(Also, my actual code uses more than one of these substitutions...)

I don't generally prefer to use explicit lambdas but expanding them out 
with names makes the definition of substitution_for a lot longer without 
much readability gain in my eyes.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From foom at fuhm.net  Tue Aug  3 21:08:43 2004
From: foom at fuhm.net (James Y Knight)
Date: Tue Aug  3 21:08:48 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408031440.i73Ee6V30875@guido.python.org>
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408031440.i73Ee6V30875@guido.python.org>
Message-ID: <89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net>

On Aug 3, 2004, at 10:40 AM, Guido van Rossum wrote:
> I have a gut feeling about this one.  I'm not sure where it comes
> from, but I have it.

I guess I have the opposite gut feeling: special classes of expressions 
in the grammar are always bad. Michael Chermside's message said this 
better than I can.

Heck, even 'except' can take an arbitrary expression as an argument! 
Now I'm damn sure I'd never write code that says
   except FooError and f() or MyErrors[5]:
because that would be insane, but I'm glad python doesn't restrict the 
argument to some strange subset of expressions that someone decided 
were approved! Consistency of grammar is a key feature, even when it 
allows users to write insane programs.

>  It may be that I want the compiler to be able to
> recognize certain decorators.

If I understand what you're saying, it seems that you're thinking the @ 
form should be more a static declaration, not a dynamically evaluated 
expression, so that the compiler can reason about it without running 
code. However, that isn't the case even now, and if it was, it wouldn't 
fit in with the rest of the python language very well. Definitely an 
interesting idea, but I think it would require the @'s argument to be a 
compile-time macro, instead of a run-time expression. And python 
doesn't have compile-time macros. With @ taking a runtime expression, I 
don't see how the compiler could possibly do anything with its 
argument. Nothing can be guaranteed.



As a postscript, I must say I am alarmed by some comments others have 
used to justify the current restricted state. In particular 
(paraphrased), "it's good because it keeps people from using lambda", 
and "it's good because it keeps people from doing '@1+1' by mistake. 
For the first: you hate lambdas so much? They are in python -- 
arbitrarily forbidding their use in certain expressions because you 
don't like them is pretty strange. For the second: You want to do type 
checking via syntax error?! *That* is insanity. And it doesn't even 
work well (of course).

James

From ncoghlan at iinet.net.au  Tue Aug  3 21:19:37 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue Aug  3 21:20:03 2004
Subject: [Python-Dev] Optimized string concatenation
In-Reply-To: <d7cee4a70408031046ebfbf2b@mail.gmail.com>
References: <003201c47904$5cd353c0$df30cb97@oemcomputer>	<200408031527.i73FRQb31100@guido.python.org>
	<d7cee4a70408031046ebfbf2b@mail.gmail.com>
Message-ID: <410FE549.1030407@iinet.net.au>

Adam Souzis wrote:
> As someone relatively new to python, it struck me as a language wart
> that i had to learn the form '"".join() as the proper way to do string
> concatenation. It violates the principles of OOWTI and its certainly
> not the obvious way to do string concatenation. This patch does not
> cover all the cases so we're still stuck with join(), but as long as
> it is not a documentated "feature" it will harmlessly improve the
> performance of countless lines of code where the coder is either
> untrained or too lazy to use join().  If its documented it'd just
> muddy the waters vis a vis join(), besides the issues with other
> Python implementation mentioned here.

If I understand correctly, you're suggesting that ''.join(strings) 
continue to be the recommended, portable, non-quadratic method for 
concatenating strings.

And this patch be seen merely as a CPython optimisation for code which 
doesn't use the recommended string concatenation hack. . .

Seems like a sensible way of looking at it to me.

Cheers,
Nick.


-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From pinard at iro.umontreal.ca  Tue Aug  3 21:47:24 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Tue Aug  3 21:54:38 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <410FCCFF.4050806@egenix.com>
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com>
Message-ID: <20040803194724.GB9031@titan.progiciels-bpi.ca>

[M.-A. Lemburg]
> Martin v. L?wis wrote:

> Things that have been proposed earlier on, extended a bit:

> b'xxx' - return a buffer to hold binary data; same as buffer(s'abc')
> s'abc' - (forced) 8-bit string literal in source code encoding
> u'abc' - (forced) Unicode literal

I currently do not see the need of a fine distinction between `b' or
`s' as a prefix. `s' and `u' are the first letter of the type (`str' or
`unicode') and that makes them natural.

> 'abc' - maps to s'abc' per default, can map to u'abc' based on the
> command line switch -U or a module switch

The idea would be, indeed, to create some kind of per-module switch.
I'm less sure that `-U' is any useful in practice, as long as all of the
library does not become "Unicode-aware", whatever that would imply...


P.S. - Command line switch for command line switch :-), a switch for
fully turning on the newer type system would be more productive than
`-U', and put some pressure for refreshening the library in this area.
Just curious, as I do not intend to volunteer in this area, is there
something else than Exception in the Python internals that rely on
old-style classes?

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard
From pinard at iro.umontreal.ca  Tue Aug  3 21:35:16 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Tue Aug  3 21:54:39 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <410FCA3B.1020002@v.loewis.de>
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de>
Message-ID: <20040803193516.GA9031@titan.progiciels-bpi.ca>

[Martin von L?wis]

> Fran?ois Pinard wrote:

> > maybe some kind of `module.__coding__' next to `module.__file__',
> >saving the coding effectively used while compilation was going on.

> That would be possible to implement.  Feel free to create a patch.

I might try, and it would be my first Python patch.  But please, please
tell me if the idea is not welcome, as my free time is rather short and
I already have a lot of things waiting for me! :-).

> >I wonder if some other cookie, next to the `coding:' cookie, could
> >not be used to declare that all strings _in this module only_ should
> >be interpreted as Unicode by default, but without the need of
> >resorting to `u' prefix all over.

> [...] if you know a syntax which you like, propose a patch.  Be
> prepared to also write a PEP defending that syntax.

Surely no particular syntax that I like enough for defending it.
Anything reasonable would do as far as I am concerned, so I might
propose a reasonable patch without involving myself into a crusade.
Yet I may try to assemble and edit together the ideas of others, if it
serves a purpose.

> >Right now, my feeling is that Python asks a bit too much of a
> >programmer, in terms of commitment, if we only consider the editing
> >work required on sources to use it, or not.

> Not sure what you are referring here to.

There is currently a lot of effort involved in Python so Unicode strings
and usual strings inter-operate correctly and automatically, also hiding
as much as reasonable to the unwilling user whether if characters are
large or narrow: s/he uses about the same code no matter what.  The way
Python does is rather lovely, in fact. :-)

I'm going to transform a flurry of Latin-1 Python scripts to UTF-8, but
not all of them, as I'm not going to impose Unicode in our team where
it is not wanted.  For French, and German and many others, we have
been lucky enough for having one codepoint per character in Unicode,
so we can hope that programs assuming that S[N] addresses the N'th
(0-based) character of string S will work the same way irrelevant of if
strings are narrow or wide.  However, and I shall have the honesty to
state it, this is *not* respectful of the general Unicode spirit: the
Python implementation allows for independently addressable surrogate
halves, combining zero-width diacritics, normal _and_ decomposed forms,
directional marks, linguistic marks and various other such complexities.

But in our case, where applications already work in Latin-1, abusing our
Unicode luck, UTF-8 may _not_ be used as is, we ought to use Unicode or
wide strings as well, for preserving S[N] addressability.  So changing
source encodings may be intimately tied to going Unicode whenever UTF-8
(or any other variable-length encoding) gets into the picture.

> You do have the choice of source encodings, and, in fact, "Unicode"
> is not a valid source encoding.  "UTF-8" is [...]

Guess that I know! :-) :-)

> [...] from a Python point of view, there is absolutely no difference
> between [UTF-8] and, say, "ISO-8859-15".  Choice of source encoding
> is different from the choice of string literals.  You can use Unicode
> strings, or byte strings, or mix them.  It really is your choice.

I hope that my explanation above helps at seeing that source encoding
and choice of string literals are not as independent as one may think.
A choice that I surely do _not_ have is to see bugs appear in programs
merely because I changed the source encoding.  Going from ISO 8859-1 to
ISO 8859-15 for a Python source is probably fairly safe, because there
is no need for switching the narrowness of strings.  Going from ISO
8859-1 to UTF-8 is very unsafe, and editing all literal strings from
narrow to wide, using `u' prefixes, becomes almost unavoidable.

There ought to be a way to maintain a single Python source that would
work dependably through re-encoding of the source, but not uselessly
relying on wide strings when there is no need for them.  That is,
without marking all literal strings as being Unicode.  Changing encoding
from ISO 8859-1 to UTF-8 should not be a one-way, no-return ticket.

Of course, it is very normal that sources may have to be adapted for the
possibility of a Unicode context.  There should be some good style and
habits for writing re-encodable programs.  So this exchange of thoughts.

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard
From guido at python.org  Tue Aug  3 22:14:58 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug  3 22:15:05 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: Your message of "Tue, 03 Aug 2004 15:47:24 EDT."
	<20040803194724.GB9031@titan.progiciels-bpi.ca> 
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com> 
	<20040803194724.GB9031@titan.progiciels-bpi.ca> 
Message-ID: <200408032014.i73KEwp31876@guido.python.org>

> P.S. - Command line switch for command line switch :-), a switch for
> fully turning on the newer type system would be more productive than
> `-U', and put some pressure for refreshening the library in this area.
> Just curious, as I do not intend to volunteer in this area, is there
> something else than Exception in the Python internals that rely on
> old-style classes?

Probably not, but making Exception a new-style class won't be easy.

(It will also break code that creates a class used as an exception
that doesn't derive from Exception, but those should be shot. :-)

Until Exception is a new-style class, such a switch wouldn't really
work.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From pf_moore at yahoo.co.uk  Tue Aug  3 22:15:30 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Aug  3 22:15:33 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
References: <1091552865.410fc661b4abd@mcherm.com>
Message-ID: <r7qo6krh.fsf@yahoo.co.uk>

Michael Chermside <mcherm@mcherm.com> writes:

> Guido writes:
>> I believe ActiveState's Trent Mick has released something
>> along these lines.  Does your proposed API match theirs?
>
> Peter replies:
>    [...]
>> So, the chances of getting our modules API-compatible are very small.
>
> I'll note that my impression is that Peter's module is not aiming to
> be particularly innovative in terms of implementation, it's primary
> "feature" is having a decent API. And my impression is that it has
> been well received on c.l.py (for instance, I often see it recommended
> in response to newbie questions). He got a lot of input during the PEP
> process and tried to incorporate it.

Although Peter has been very receptive to feedback, he has some strong
design goals (noted in the PEP) which make the resulting module feel
too "low level" for me.

I'd love to see a function 

   process.filter(cmd, input=None)

Execute cmd (if it's a string, use the shell, if it's a list, treat as
an argv sequence) and pass it input as stdin (default, no stdin).
Return the command's stdout. If an error occurs (either in the exec,
or if the command returns a nonzero return code) raise an exception
(and capture the stderr in the exception data). This function should
deal with deadlock issues.

I don't know of *any* process module that makes this "expect it to
work" model any easier. And Peter's design constraint of avoiding the
shell actively hurts here (although his interact() function is
specifically there to avoid the deadlock issues that could otherwise
occur with a naive attempt to implement this).

It's possible that I could implement this function on top of Trent's
module. Must try...

Paul.
-- 
"Bother," said the Borg, "We've assimilated Pooh."

From eppstein at ics.uci.edu  Tue Aug  3 22:22:11 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Tue Aug  3 22:22:16 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408031440.i73Ee6V30875@guido.python.org>
	<89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net>
Message-ID: <eppstein-21570B.13221103082004@sea.gmane.org>

In article <89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net>,
 James Y Knight <foom@fuhm.net> wrote:

> As a postscript, I must say I am alarmed by some comments others have 
> used to justify the current restricted state. In particular 
> (paraphrased), "it's good because it keeps people from using lambda", 
> and "it's good because it keeps people from doing '@1+1' by mistake. 
> For the first: you hate lambdas so much?

I think I was the one who posted the lambda comment.
I also posted in a different message a use-case for decorators that had 
a line with two lambdas in it, so obviously I don't hate lambdas.

I'm actually pretty neutral on restricted decorators vs @expression.  But
   @lambda(func): body
   def func(...): ...
is (I think) much more likely to be an abuse of language than the 
Pythonic way of writing something, so I don't see a lot of harm in 
preventing it.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From pf_moore at yahoo.co.uk  Tue Aug  3 22:24:13 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Aug  3 22:24:14 2004
Subject: [Python-Dev] PEP 309 (partial function application)
Message-ID: <n01c6kcy.fsf@yahoo.co.uk>

Should this go into 2.4a2? It's marked as "accepted" in the PEP list,
and an implementation exists as SF patch 941881.

Paul
-- 
The major difference between a thing that might go wrong and a thing
that cannot possibly go wrong is that when a thing that cannot
possibly go wrong goes wrong it usually turns out to be impossible to
get at or repair. -- Douglas Adams

From jhylton at gmail.com  Tue Aug  3 23:08:07 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Tue Aug  3 23:08:09 2004
Subject: [Python-Dev] PEP 309 (partial function application)
In-Reply-To: <n01c6kcy.fsf@yahoo.co.uk>
References: <n01c6kcy.fsf@yahoo.co.uk>
Message-ID: <e8bf7a53040803140878c34b3d@mail.gmail.com>

On Tue, 03 Aug 2004 21:24:13 +0100, Paul Moore <pf_moore@yahoo.co.uk> wrote:
> Should this go into 2.4a2? It's marked as "accepted" in the PEP list,
> and an implementation exists as SF patch 941881.

Paul,

Without comment on the merits of the PEP, it seems out of date to me. 
The PEP presents three different implementations of curry (one in
Pyrex) without clarifying which one is the preferred one.  It presents
a summary at the end that would make more sense at the beginning.

I would revise the PEP to:

1. Clearly describe what is being implemented for 2.4.  Provide a
specification for the behavior along with examples of use.  Make sure
you cover all the corner cases, e.g. can you provide a keyword arg
without providing positional args.

2. Provide a rationale for why this belongs in the standard library.

3. Describe the @ syntax, but put it in a self-contained section and
explain why it was not implemented.

Jeremy
From djc at object-craft.com.au  Wed Aug  4 01:55:49 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Wed Aug  4 01:55:54 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <410F389D.6090609@iinet.net.au>
References: <410F3465.9000005@object-craft.com.au>
	<410F389D.6090609@iinet.net.au>
Message-ID: <41102605.2080807@object-craft.com.au>

Nick Coghlan wrote:
> Dave Cole wrote:
> 
>> Is there any reason why something like this would not be a good idea?
>>
>>  >>> a_list = [1, 2, 3, 4, 5]
>>  >>> a, b, *c = a_list
>>
>> You could then do things like this:
>>
>>  >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]]
>>  >>> for a, b *c in lol:
>> ...
>>
>> - Dave
>>
> 
> As opposed to:
> 
>  >>> for a, b, c in ((x[0], x[1], x[2:]) for x in lol): print a, b, c

Yes, as opposed to.

> With generator expressions around, I don't know that this case is common 
> enough for special casing. . .

This begs the question; do you prefer:

 >>> args = [4, 5, 6]
 >>> a_func(1, *args)

or this:

 >>> args = [4, 5, 6]
 >>> apply(a_func, [1] + args)

- Dave

-- 
http://www.object-craft.com.au
From tjreedy at udel.edu  Wed Aug  4 02:25:19 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed Aug  4 02:25:25 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
References: <Your message of "Mon, 02 Aug 2004 13:30:03
	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org>
	<410FC9FE.6060305@divmod.com>
Message-ID: <cepadg$dp2$1@sea.gmane.org>


"Jp Calderone" <exarkun@divmod.com> wrote in message
news:410FC9FE.6060305@divmod.com...
>    I like Python(*) because it is simple and generally unsurprising.

Me too.

>  I  think that '@deco' adds complexity, both for beginners and
experienced
> programmers alike,

Decorators will add complexity, regardless of syntax.  An advantage of the
@ syntax is than it makes the complexiity separable and ignorable for
someone learning/teaching basic Python, which includes list literals and
function defs.  To me, the [deco list] syntax is particularly bad in adding
an advanced, esoteric, context-dependent, magic meaning to an hour 1 basic
concept.

> so I do not see it as an improvement.

Apparently, the question of whether decos are an improvement has at least
temporarily been decided in the affirmative.

Terry J. Reedy



From nas at arctrix.com  Wed Aug  4 03:07:12 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed Aug  4 03:07:15 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <cepadg$dp2$1@sea.gmane.org>
References: <410FC9FE.6060305@divmod.com> <cepadg$dp2$1@sea.gmane.org>
Message-ID: <20040804010712.GA10819@mems-exchange.org>

On Tue, Aug 03, 2004 at 08:25:19PM -0400, Terry Reedy wrote:
> Apparently, the question of whether decos are an improvement has
> at least temporarily been decided in the affirmative.

That's not apparent to me.  It seems to me that most people just got
sick of the endless debate and tuned out.  The people who had urgent
need for the feature were probably a bit more persistent.
Personally, I don't see the cost vs. benefit of the feature to be
compelling.  It seems there is not a lot you can do with a decorator
without resorting to ugly hacks (e.g. messing with bytecode).

  returning-to-tuned-out-mode-ly yr's Neil
From greg at cosc.canterbury.ac.nz  Wed Aug  4 03:14:32 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug  4 03:14:38 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <1f7befae040802201470e3d4c@mail.gmail.com>
Message-ID: <200408040114.i741EWcl029740@cosc353.cosc.canterbury.ac.nz>

Tim Peters <tim.peters@gmail.com>:

> there's one routine that executes a module's initialization code,
> all imports go thru it eventually, and that's the routine that now
> removes the module's name from sys.modules if the initialization
> code barfs.

Couldn't that routine also save sys.modules before
starting to execute the module's code?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From andrewm at object-craft.com.au  Wed Aug  4 03:23:20 2004
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Wed Aug  4 03:23:19 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators 
In-Reply-To: <20040804010712.GA10819@mems-exchange.org> 
References: <410FC9FE.6060305@divmod.com> <cepadg$dp2$1@sea.gmane.org>
	<20040804010712.GA10819@mems-exchange.org>
Message-ID: <20040804012320.B38523C7A0@coffee.object-craft.com.au>

>That's not apparent to me.  It seems to me that most people just got
>sick of the endless debate and tuned out.

It's also entirely possible that "most people" were happy with the way
things were going and chose not to post...

My personal opinion is that the pie syntax is acceptable, and the
continuing debate is doing harm and providing no enlightenment.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/
From aahz at pythoncraft.com  Wed Aug  4 03:33:59 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed Aug  4 03:34:02 2004
Subject: [Python-Dev] 2.4a2 and relative import (PEP 328)
In-Reply-To: <200408021850.i72IoCg28650@guido.python.org>
References: <410DDFE3.7010006@interlink.com.au>
	<200408021850.i72IoCg28650@guido.python.org>
Message-ID: <20040804013359.GA14122@panix.com>

On Mon, Aug 02, 2004, Guido van Rossum wrote:
>
> Did the relative import syntax that was approved ever get checked in?
> I thought it was on the list of things to land in 2.4?

Sorry, I've been slacking on driving it.  What it really needs right now
is someone to implement it.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From tim.peters at gmail.com  Wed Aug  4 04:03:44 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Aug  4 04:03:46 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <200408040114.i741EWcl029740@cosc353.cosc.canterbury.ac.nz>
References: <200408040114.i741EWcl029740@cosc353.cosc.canterbury.ac.nz>
Message-ID: <1f7befae04080319031d9b1c94@mail.gmail.com>

[Tim Peters]
>> there's one routine that executes a module's initialization code,
>> all imports go thru it eventually,

Which isn't true.  I'm hacking imputils and ihooks now.

>> and that's the routine that now removes the module's name from
>> sys.modules if the initialization code barfs.

[Greg Ewing]
> Couldn't that routine also save sys.modules before
> starting to execute the module's code?

With enough bother, sure.  Go for it <wink -- but explaining
subtleties is time-consuming and futile; better to find them
yourself>.
From tim.peters at gmail.com  Wed Aug  4 04:14:04 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Aug  4 04:14:07 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <20040804012320.B38523C7A0@coffee.object-craft.com.au>
References: <410FC9FE.6060305@divmod.com> <cepadg$dp2$1@sea.gmane.org>
	<20040804010712.GA10819@mems-exchange.org>
	<20040804012320.B38523C7A0@coffee.object-craft.com.au>
Message-ID: <1f7befae040803191421447be6@mail.gmail.com>

[Andrew McNamara]
> It's also entirely possible that "most people" were happy with the way
> things were going and chose not to post...

I was indeed "happy enough".

> My personal opinion is that the pie syntax is acceptable,

Ditto.  It does simple things quite well, and it makes it obvious
they're being done.  Then again, I like "print >> fileobj" too <0.7
wink>.

> and the continuing debate is doing harm

We're in no danger of running out of pixels.

> and providing no enlightenment.

That's for sure.  In the old days, Guido would Pronounce, and we'd all
bite our tongues (although not necessarily each his own).  The less
time Guido can make for Python, the more important becomes graceful
capitulation.
From guido at python.org  Wed Aug  4 04:24:59 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug  4 04:25:09 2004
Subject: [Python-Dev] 2.4a2 and relative import (PEP 328)
In-Reply-To: Your message of "Tue, 03 Aug 2004 21:33:59 EDT."
	<20040804013359.GA14122@panix.com> 
References: <410DDFE3.7010006@interlink.com.au>
	<200408021850.i72IoCg28650@guido.python.org> 
	<20040804013359.GA14122@panix.com> 
Message-ID: <200408040224.i742Oxc32452@guido.python.org>

> > Did the relative import syntax that was approved ever get checked in?
> > I thought it was on the list of things to land in 2.4?
> 
> Sorry, I've been slacking on driving it.  What it really needs right now
> is someone to implement it.

It's not too late -- the decorator syntax quickly found someone who
implemented the whole thing too.  Hard to make a2, but a3 is still a
possibility.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From ncoghlan at iinet.net.au  Wed Aug  4 05:08:51 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Aug  4 05:09:18 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <eppstein-21570B.13221103082004@sea.gmane.org>
References: <1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>	<200408030105.i7315Fj29310@guido.python.org>	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>	<200408031440.i73Ee6V30875@guido.python.org>	<89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net>
	<eppstein-21570B.13221103082004@sea.gmane.org>
Message-ID: <41105343.7020904@iinet.net.au>

David Eppstein wrote:

> I'm actually pretty neutral on restricted decorators vs @expression.  But
>    @lambda(func): body
>    def func(...): ...
> is (I think) much more likely to be an abuse of language than the 
> Pythonic way of writing something, so I don't see a lot of harm in 
> preventing it.

And, as someone pointed out, *lifting* a restriction is a heck of a lot 
easier than imposing one later on (even after only 1 alpha). 
YouAintGonnaNeedIt and all that. . .

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From ncoghlan at iinet.net.au  Wed Aug  4 05:21:56 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Aug  4 05:22:18 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <41102605.2080807@object-craft.com.au>
References: <410F3465.9000005@object-craft.com.au>	<410F389D.6090609@iinet.net.au>
	<41102605.2080807@object-craft.com.au>
Message-ID: <41105654.6080305@iinet.net.au>

Dave Cole wrote:

> This begs the question; do you prefer:
> 
>  >>> args = [4, 5, 6]
>  >>> a_func(1, *args)
> 
> or this:
> 
>  >>> args = [4, 5, 6]
>  >>> apply(a_func, [1] + args)
> 
> - Dave

Oh, I certainly prefer the variable argument format, rather than having 
to use apply. The difference is that I've found that useful on several 
occasions, mainly because keyword arguments let you get around the 'only 
at the end' problem.

As far as I know, the new syntax was to able to *completely* replace the 
functionality of 'apply' (I've certainly never needed to use it, no 
matter how complex the variable argument list games got).

Whereas the list assignment proposal gives special synactic sugar to 
*one* form of slicing (x[0], ..., x[n], x[(n+1):]).

As soon as the form of your slice changes, you're going to have to 
switch to a generator expression anyway. Hence the question about 
whether or not this special case was special enough to be given its own 
syntax, given that the comparable generator expression really isn't that 
complicated.

*shrug* I'm not implacably opposed or anything - although I'd be 
interested in hearing from those around hear who teach Python as to 
whether they think it would be beneficial or not.

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268
From greg at cosc.canterbury.ac.nz  Wed Aug  4 05:50:27 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug  4 05:50:37 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <41105654.6080305@iinet.net.au>
Message-ID: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz>

Nick Coghlan <ncoghlan@iinet.net.au>:

> Whereas the list assignment proposal gives special synactic sugar to 
> *one* form of slicing (x[0], ..., x[n], x[(n+1):]).

It's a comparatively common one, though, I think.  When doing things
like parsing input, it's often natural to want to peel off the first
few items of a list and leave the rest for processing later, based on
what the initial items turn out to be.

Having to explicitly slice off the part you want to unpack not only
seems inefficient (constructing an intermediate list or tuple just to
immediately unpack it and throw it away) it also tends to obfuscate
what is going on.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From edloper at gradient.cis.upenn.edu  Wed Aug  4 06:11:07 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Wed Aug  4 06:10:21 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <20040804030933.210F61E4020@bag.python.org>
References: <20040804030933.210F61E4020@bag.python.org>
Message-ID: <411061DB.8010803@gradient.cis.upenn.edu>

> It's also entirely possible that "most people" were happy with the way
> things were going and chose not to post...
> 
> My personal opinion is that the pie syntax is acceptable, and the
> continuing debate is doing harm and providing no enlightenment.

+1.

(Thanks to the people who've devoted so much thought and energy to 
figuring out the best decorator syntax; I, for one, will be happy to 
have it available.)

-Edward

From trentm at ActiveState.com  Wed Aug  4 06:31:21 2004
From: trentm at ActiveState.com (Trent Mick)
Date: Wed Aug  4 06:31:28 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>;
	from astrand@lysator.liu.se on Tue, Aug 03, 2004 at 05:22:35PM
	+0200
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
Message-ID: <20040803213121.C9777@ActiveState.com>

[Peter Astrand wrote]
> 
> > I believe ActiveState's Trent Mick has released something along these
> > lines.  Does your proposed API match theirs?
> 
> No, not at all. There are some issues with this module, both regards the
> API and the implementation, which I'm not happy with. I tried to establish
> a discussion with Trent about this, but if I remember correctly, I never
> recieved a response.

Yes, I am an ass. I am sorry about that Peter. I have had my blinders on
for quite a while with work stuff.

I have just started looking at your module so I can't be sure that we
could find a happy common ground but I hope so. I realize that my
process.py has some warts (including some you have pointed out below)
and I am willing to work on those.

> Here's some things which I'm not happy with:
> 
> * There are three different "factory" classes: Process, ProcessOpen,
> ProcessProxy. I have only one, covering all cases. I think three different
> classes are confusing to users. Even I have trouble understanding the
> difference between them.

Part of the confusion, I think, is that my process.py's docstrings have
lagged a little bit behind development and there is one glaring problem
that ProcessOpen.__init__.__doc__ mistakenly has the description from
ProcessProxy.__init__. 

Process:
    Create a process without doing without doing anything with
    stdin/stderr/stdout. Kind of like os.system in that regard.
ProcessOpen:
    ...add stdin/stdout/stderr attributes (file-like objects). Kind of
    like os.popen3 in that regard. 

Separating the two allows Process' implementation to be simpler: no
dup'ing (and inheritability handling, etc) of the std handlers is
required. If general opinion is that this separation is not useful, then
I am fine with merging the two. (Note that my vague memory is that this
separation may be necessary to allow for new console handling with
subsystem:windows apps on Windows. I can't remember exactly though.)

ProcessProxy:
    ...a behemoth using a thread for each of stdin/stdout/stderr to
    allow the user to get an event-like IO interface.
    
This is something that was required for Komodo (for my job) to allow
interactive running (and stdin handling) of processes in a GUI terminal
(i.e., if you know Komodo, to run commands in the "Command Output" tab).
If this is deemed too controversial for the Python core then I think it
would be possible to move this out to a separate module -- though I know
that some people outside of Komodo have found this useful.

> 
> * The code size is very large. The code is complex.
> 
> * Trent's module always executes things through the shell, and deals with
> every ugly cornercase that comes from this.

Some of the large code size is _for_ execution through the shell. :)  I
think that execution via the shell should be a feature of a module like
this (so that users can use some shell features) and I even think that
it should be the default (so that, for example, Windows users don't have
to learn what cmd.exe or command.com are to run "dir"). However, I
absolutely agree that one should be able to run withOUT the shell (i.e.
have an option for this).

Other reasons for the size/complexity of my process.py over yours:
- My process objects have a .kill() method -- which is actually quite a
  pain on Windows.
- My module contains a work-around (113 lines) for a known bug in
  LinuxThreads where one cannot .wait() on a created process from a
  subthread of the thread that created the process.
  (1) This feature was a requirement for Komodo. Pulling it out to a
      separate module and just documenting the limitation (which really
      isn't a big deal for most common uses) is probably an option for me.
      In fact the workaround is probably not generally acceptible and
      should, at the very least, be made optional.
  (2) I haven't had a chance to check but I _think_ that recent Linuxs
      may have switched to the newer threading libraries out there that
      fix this. At least I remember reading about the new threading
      libraries (there was more that one competitor) a couple of years
      ago.
- The ProcessProxy "behemoth" is responsible for about half of my
  process.py module.
- My module includes some handling that for subsystem:windows vs.
  subsystem:console apps on Windows that I don't think yours does.
  (That isn't totally fair: I think yours has a few features that mine
  doesn't.)


> * The modules uses destructors, which I'm usually avoiding.

Is that evil? I have gotten mixed signals from the occassional lurking
that I have done on the Python lists. Currently refcounts will ensure
that my __del__'s get called. And I *do* have .close() methods for
proper usage. Though that seem anathema to some Python programmers.


> So, the chances of getting our modules API-compatible are very small.

Presuming that I don't go dark again I hope that maybe we can reach some
concensus. In the meantime, if you'd be willing to change your module's
name to something other than process.py I think that would help
discussions. (I don't know if that is a pain for you at this point. You
mentioned "subprocess". Alternatively, how about "posixprocess"? Though,
despite PEP 324's title, I don't know if that is completely accurate
anymore.)


Trent

-- 
Trent Mick
TrentM@ActiveState.com
From djc at object-craft.com.au  Wed Aug  4 06:36:56 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Wed Aug  4 06:37:01 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz>
References: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz>
Message-ID: <411067E8.3020002@object-craft.com.au>

Greg Ewing wrote:
> Nick Coghlan <ncoghlan@iinet.net.au>:
> 
> 
>>Whereas the list assignment proposal gives special synactic sugar to 
>>*one* form of slicing (x[0], ..., x[n], x[(n+1):]).
> 
> 
> It's a comparatively common one, though, I think.  When doing things
> like parsing input, it's often natural to want to peel off the first
> few items of a list and leave the rest for processing later, based on
> what the initial items turn out to be.
> 
> Having to explicitly slice off the part you want to unpack not only
> seems inefficient (constructing an intermediate list or tuple just to
> immediately unpack it and throw it away) it also tends to obfuscate
> what is going on.

So I suppose you could avoid the unnecessary tuple construction by doing 
something like this:

 >>> a_list = [1, 2, 3, 4, 5]
 >>> a, b, * = a_list

 >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]]
 >>> for a, b, * in lol:
...

But that looks a bit bogus - but does avoid having to create the extra 
tuple.

- Dave

-- 
http://www.object-craft.com.au
From martin at v.loewis.de  Wed Aug  4 06:51:28 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug  4 06:51:27 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz>
References: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz>
Message-ID: <41106B50.2000201@v.loewis.de>

Greg Ewing wrote:
> Having to explicitly slice off the part you want to unpack not only
> seems inefficient (constructing an intermediate list or tuple just to
> immediately unpack it and throw it away) it also tends to obfuscate
> what is going on.

Of course, the proposed syntax does not change this. Compare

a,b,c = foo()[:3]

to

a,b,c,*_ = foo()

In either case, the extra fields obfuscate things, and in either
case, a second tuple is created. In the latter case, the tuple
is even stored in a variable.

Regards,
Martin
From kbk at shore.net  Wed Aug  4 06:54:17 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Aug  4 06:54:22 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200408040454.i744sHIe016349@h006008a7bda6.ne.client2.attbi.com>


Patch / Bug Summary
___________________

Patches :  276 open ( -3) /  2493 closed (+10) /  2769 total ( +7)
Bugs    :  766 open ( +4) /  4317 closed (+14) /  5083 total (+18)
RFE     :  146 open ( +1) /   130 closed ( +0) /   276 total ( +1)

New / Reopened Patches
______________________

A BSD-style wait4 implementation  (2004-07-29)
       http://python.org/sf/1000267  opened by  chads

posixpath._resolve_link typo  (2004-07-31)
CLOSED http://python.org/sf/1001109  opened by  Ronald Oussoren

socketmodule on OpenBSD/sparc64 (64bit machine)  (2004-08-01)
       http://python.org/sf/1001610  opened by  Aleksander Piotrowski

Logger file objects  (2004-08-02)
       http://python.org/sf/1001864  opened by  Miki Tebeka

O(1) xrange membership testing  (2004-08-02)
       http://python.org/sf/1002085  opened by  Mac-arena the Bored Zo

Better extensibility for distutils commands  (2004-08-02)
CLOSED http://python.org/sf/1002241  opened by  Fred L. Drake, Jr.

Patches Closed
______________

posixpath._resolve_link typo  (2004-07-31)
       http://python.org/sf/1001109  closed by  ronaldoussoren

Implementation for PEP 318 using java-style syntax  (2004-06-25)
       http://python.org/sf/979728  closed by  anthonybaxter

Better extensibility for distutils commands  (2004-08-02)
       http://python.org/sf/1002241  closed by  fdrake

HTTP basic authentication problem  (2004-07-20)
       http://python.org/sf/994595  closed by  loewis

speed up md5: allow Encode and Decode to inline  (2004-06-22)
       http://python.org/sf/977074  closed by  loewis

add option to NOT use ~/.netrc in nntplib.NNTP()  (2003-10-14)
       http://python.org/sf/823072  closed by  loewis

askstring => grab fail  (2004-05-02)
       http://python.org/sf/946153  closed by  loewis

658254: accept None for time.ctime() and friends  (2003-01-06)
       http://python.org/sf/663482  closed by  fdrake

Add support for sync and use "wish" options  (2004-07-08)
       http://python.org/sf/986929  closed by  loewis

commands.py for Windows  (2002-12-17)
       http://python.org/sf/655421  closed by  zopezen

New / Reopened Bugs
___________________

optparse error method doesn't print option info  (2004-07-29)
       http://python.org/sf/1000439  opened by  Ivan Nestlerode

"make pdf" failure w/ 2.4 docs  (2004-07-30)
       http://python.org/sf/1000841  opened by  Mat Martineau

Carbon.CF.CFMutableArray hangs interpreter  (2004-07-30)
       http://python.org/sf/1000914  opened by  Ronald Oussoren

str.join([ str-subtype-instance ]) misbehaves  (2004-07-31)
       http://python.org/sf/1001011  opened by  Thomas Wouters

setdefaulttimeout causes unnecessary timeouts on connect err  (2004-07-31)
CLOSED http://python.org/sf/1001018  opened by  Mark Hammond

wave.open() with unicode filename fails  (2004-07-31)
CLOSED http://python.org/sf/1001053  opened by  John Popplewell

incorrect reference to macro named foo  (2004-07-31)
CLOSED http://python.org/sf/1001088  opened by  Clinton Roy

hotspot profiler does not work correctly on P4 CPUs with HT  (2004-07-31)
       http://python.org/sf/1001150  opened by  Viktor Ferenczi

glob  unicode  (2004-08-01)
       http://python.org/sf/1001604  opened by  leve

socketmodule does not build under cygwin  (2004-08-02)
       http://python.org/sf/1001857  opened by  Miki Tebeka

IDLE hangs when inactive more than 2 hours  (2004-08-02)
       http://python.org/sf/1001869  opened by  Miki Tebeka

os.path.realpath can't handle symlink loops  (2004-04-05)
       http://python.org/sf/930024  reopened by  loewis

os.path.sameopenfile documentation wrong.  (2004-08-02)
       http://python.org/sf/1002398  opened by  Jeremy Fincher

Wrong documentation of __radd__ etc.  (2004-08-03)
       http://python.org/sf/1002453  opened by  Hallvard B Furuseth

MemoryError on AIX52  (2004-08-03)
       http://python.org/sf/1002465  opened by  Datranet

email message parser doesn't handle \r\n correctly  (2004-08-03)
       http://python.org/sf/1002475  opened by  Sjoerd Mullender

test_decimal fails if repeated  (2004-08-03)
       http://python.org/sf/1002530  opened by  Michael Hudson

test_logging fails if run twice  (2004-08-03)
       http://python.org/sf/1002537  opened by  Michael Hudson

glob()  unicode bug  (2004-08-03)
CLOSED http://python.org/sf/1002632  opened by  leve

asynchat does not accept 'long' terminator  (2004-08-03)
       http://python.org/sf/1002763  opened by  Jey Kottalam

Bugs Closed
___________

"disjunct" should be "disjoint"  (2004-07-25)
       http://python.org/sf/997533  closed by  rhettinger

Mingw needs to use the same MSVC runtime as Python  (2004-01-04)
       http://python.org/sf/870382  closed by  loewis

Python 2.3.4, Solaris 7, socketmodule.c does not compile  (2004-06-14)
       http://python.org/sf/972724  closed by  brucedray

setdefaulttimeout causes unnecessary timeouts on connect err  (2004-07-30)
       http://python.org/sf/1001018  closed by  tim_one

wave.open() with unicode filename fails  (2004-07-30)
       http://python.org/sf/1001053  closed by  nnorwitz

incorrect reference to macro named foo  (2004-07-31)
       http://python.org/sf/1001088  closed by  nnorwitz

os.path.realpath can't handle symlink loops  (2004-04-05)
       http://python.org/sf/930024  closed by  akuchling

leaking snippet  (2003-09-18)
       http://python.org/sf/808596  closed by  mwh

refleaks in _hotshot.c  (2003-09-18)
       http://python.org/sf/808756  closed by  arigo

math and cmath docs don't specify radians  (2004-07-23)
       http://python.org/sf/996392  closed by  tim_one

bsddb has default flag of c, not r  (2004-07-26)
       http://python.org/sf/998001  closed by  akuchling

glob()  unicode bug  (2004-08-03)
       http://python.org/sf/1002632  closed by  lemburg

boolobject.h documentation missing from Python/C API  (2004-01-22)
       http://python.org/sf/882332  closed by  fdrake

Accept None for time.ctime() and friends  (2002-12-24)
       http://python.org/sf/658254  closed by  fdrake

idle.py doesn't accept ( in some cases  (2003-07-22)
       http://python.org/sf/775541  closed by  kbk

New / Reopened RFE
__________________

Adding missing ISO 8859 codecs, especially Thai  (2004-08-02)
       http://python.org/sf/1001895  opened by  Peter Jacobi

From djc at object-craft.com.au  Wed Aug  4 06:57:29 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Wed Aug  4 06:57:34 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <41106B50.2000201@v.loewis.de>
References: <200408040350.i743oRiB029962@cosc353.cosc.canterbury.ac.nz>
	<41106B50.2000201@v.loewis.de>
Message-ID: <41106CB9.8000302@object-craft.com.au>

Martin v. L?wis wrote:
> Greg Ewing wrote:
> 
>> Having to explicitly slice off the part you want to unpack not only
>> seems inefficient (constructing an intermediate list or tuple just to
>> immediately unpack it and throw it away) it also tends to obfuscate
>> what is going on.
> 
> 
> Of course, the proposed syntax does not change this. Compare
> 
> a,b,c = foo()[:3]

That is not really the use case for the feature.  What I would really 
like is a nice way to avoid doing this:

 >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]]
 >>> for tmp in lol:
...     a, b = tmp[:2]
...     c = tmp[2:]

I think that the original is a nicer looking syntax, and is fairly 
consistent with the existing "varargs" handling in function signatures.

 >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]]
 >>> for a, b, *c in lol:
...

> to
> 
> a,b,c,*_ = foo()
> 
> In either case, the extra fields obfuscate things, and in either
> case, a second tuple is created. In the latter case, the tuple
> is even stored in a variable.

-- 
http://www.object-craft.com.au
From greg at cosc.canterbury.ac.nz  Wed Aug  4 07:03:31 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug  4 07:03:36 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <41106CB9.8000302@object-craft.com.au>
Message-ID: <200408040503.i7453VQE030085@cosc353.cosc.canterbury.ac.nz>

> That is not really the use case for the feature.  What I would really 
> like is a nice way to avoid doing this:
> 
>  >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]]
>  >>> for tmp in lol:
> ...     a, b = tmp[:2]
> ...     c = tmp[2:]

Maybe the case of not caring about the rest could be
addressed explicitly, e.g.

  a, b, c, ... = stuff

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From tdelaney at avaya.com  Wed Aug  4 07:07:42 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Aug  4 07:07:48 2004
Subject: [Python-Dev] Tuple/list assignment question
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C000@au3010avexu1.global.avaya.com>

Dave Cole wrote:

> Is there any reason why something like this would not be a good idea?
> 
>  >>> a_list = [1, 2, 3, 4, 5]
>  >>> a, b, *c = a_list
> 
> You could then do things like this:
> 
>  >>> lol = [[1, 2], [3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13]]
>  >>> for a, b *c in lol:
> ...

This proposal has come up several times before (for the record, I like
it). I believe there was a particular reason it's never gone forward,
but I can't remember what it was. Unfortunately, it's not an easy thing
to search for in the c.l.py archives :(

I think it's about time this proposal got a PEP of its own ... something
like "Enhanced Tuple Unpacking" (to match "Enhanced Argument Tuples" ...

Tim Delaney
From florian.proff.schulze at gmx.net  Wed Aug  4 09:15:56 2004
From: florian.proff.schulze at gmx.net (Florian Schulze)
Date: Wed Aug  4 09:12:03 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
References: <200408030142.i731gufH027985@cosc353.cosc.canterbury.ac.nz>
	<200408030351.i733pcO29560@guido.python.org>
	<1091556102.7597.26.camel@localhost>
Message-ID: <opsb644u0v5c87rg@nwlab5.mni.fh-giessen.de>

On Tue, 03 Aug 2004 14:01:42 -0400, Barry Warsaw <barry@python.org> wrote:

> For longer functions, it's a clear win; the '@' stands out nicely,
> especially with the python-mode.el font-locking support I just added. :)
> It certainly exhibits all the benefits that motivated adding special
> syntax for decorators in the first place.

I think this is also a strong point for the '@' syntax. How should an  
editor know that a list before a method is a decoration and should be  
highlighted differently?

The backwards compatibility to python < 2.4 doesn't count in my opinion,  
because other features like generators and list-comprehensions also can't  
be used in older pythons, and the old syntax with the decoration after the  
method body is still available, so this is no problem.

+1 for the pie syntax from my little voice as a user of python

Regards,
Florian Schulze

From martin at v.loewis.de  Wed Aug  4 09:15:23 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug  4 09:15:21 2004
Subject: [Python-Dev] Cygwin and sockets
Message-ID: <41108D0B.4020003@v.loewis.de>

We have a report (#1001857) that the socket module does not build
on Cygwin. Can anybody confirm that?

Regards,
Martin
From t-meyer at ihug.co.nz  Wed Aug  4 09:42:30 2004
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Wed Aug  4 09:42:37 2004
Subject: [Python-Dev] Cygwin and sockets
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E874F135@its-xchg4.massey.ac.nz>
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E86C541C@its-xchg4.massey.ac.nz>

> We have a report (#1001857) that the socket module does not 
> build on Cygwin. Can anybody confirm that?

By "anybody", did you mean anybody that you know & trust?  <0.5 wink>.

I can duplicate the process outlined in the bug report and get the same
failure to compile (I've added a note to it).

However, I don't know much more than that!  (If I even use python in cygwin
I typically use the distributed binary).

=Tony Meyer

From martin at v.loewis.de  Wed Aug  4 11:22:02 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug  4 11:22:01 2004
Subject: [Python-Dev] Cygwin and sockets
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E86C541C@its-xchg4.massey.ac.nz>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E86C541C@its-xchg4.massey.ac.nz>
Message-ID: <4110AABA.80407@v.loewis.de>

Tony Meyer wrote:
> By "anybody", did you mean anybody that you know & trust?  <0.5 wink>.
> 
> I can duplicate the process outlined in the bug report and get the same
> failure to compile (I've added a note to it).

This is good enough for the moment, although I would have preferred an
answer like "I know, and I have a fix sitting on my harddisk" :-)

Regards,
Martin
From mwh at python.net  Wed Aug  4 12:29:48 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Aug  4 12:29:50 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <410FC9FE.6060305@divmod.com> (Jp Calderone's message of "Tue,
	03 Aug 2004 13:23:10 -0400")
References: <Your message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<410F2453.7000506@divmod.com>
	<200408031404.i73E4cS30629@guido.python.org>
	<410FB7F4.2070309@divmod.com>
	<200408031700.i73H0MY31391@guido.python.org>
	<410FC9FE.6060305@divmod.com>
Message-ID: <2macxb9owz.fsf@starship.python.net>

Jp Calderone <exarkun@divmod.com> writes:

>    How many features that everyone hates can Python support?

It's disingenuous to suggest that this or any other feature of Python
is one "everyone hates".  Hey, I am even quite happy with ''.join,
another source of controversy that seems to have failed to completely
halt the adoption of Python <wink>.

Cheers,
mwh

-- 
  If your telephone company installs a system in the woods with no
  one around to see them, do they still get it wrong?
                                 -- Robert Moir, alt.sysadmin.recovery
From mwh at python.net  Wed Aug  4 12:31:37 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Aug  4 12:31:39 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net> (James Y.
	Knight's message of "Tue, 3 Aug 2004 15:08:43 -0400")
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408031440.i73Ee6V30875@guido.python.org>
	<89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net>
Message-ID: <2m657z9oty.fsf@starship.python.net>

James Y Knight <foom@fuhm.net> writes:

> Now I'm damn sure I'd never write code that says
>    except FooError and f() or MyErrors[5]:
> because that would be insane,

I think

class C(random.choice([dict, list])):
    pass

is my favourite example of this :-)

Cheers,
mwh

-- 
  That being done, all you have to do next is call free() slightly
  less often than malloc(). You may want to examine the Solaris
  system libraries for a particularly ambitious implementation of
  this technique.          -- Eric O'Dell, comp.lang.dylan (& x-posts)
From mwh at python.net  Wed Aug  4 12:38:58 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Aug  4 12:39:00 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <200408032014.i73KEwp31876@guido.python.org> (Guido van
	Rossum's message of "Tue, 03 Aug 2004 13:14:58 -0700")
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com>
	<20040803194724.GB9031@titan.progiciels-bpi.ca>
	<200408032014.i73KEwp31876@guido.python.org>
Message-ID: <2m1xin9ohp.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> P.S. - Command line switch for command line switch :-), a switch for
>> fully turning on the newer type system would be more productive than
>> `-U', and put some pressure for refreshening the library in this area.
>> Just curious, as I do not intend to volunteer in this area, is there
>> something else than Exception in the Python internals that rely on
>> old-style classes?
>
> Probably not, but making Exception a new-style class won't be easy.

What makes you say that?  I've just been remarking on comp.lang.python
how having Exception be new-style in PyPy -- indeed, not having
old-style classes and all -- has caused essentially no problems at
all.

Perhaps I'll work up a patch sometime and see what breaks.

Cheers,
mwh

-- 
  > Emacs is a fashion statement.
  No, Gnus is a fashion statement.  Emacs is clothing.  Everyone 
  else is running around naked.
        -- Karl Kleinpaste & Jonadab the Unsightly One, gnu.emacs.gnus
From mwh at python.net  Wed Aug  4 12:53:57 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Aug  4 12:53:58 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.311, 2.312
In-Reply-To: <410FC78C.4010609@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Tue,
	03 Aug 2004 19:12:44 +0200")
References: <E1BrwSl-0005lx-7A@sc8-pr-cvs1.sourceforge.net>
	<200408031418.i73EIFR30699@guido.python.org>
	<410FA073.5070105@v.loewis.de>
	<200408031434.i73EYvv30846@guido.python.org>
	<410FB12C.4010105@v.loewis.de> <2misc09po1.fsf@starship.python.net>
	<410FC78C.4010609@v.loewis.de>
Message-ID: <2mu0vj898a.fsf@starship.python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

>> I'd be happy with any other (I mentioned not doing
>> refcounting on dictobject.c's dummy; maybe I should actually see if
>> that's possible :-).
>
> Feel free to change the marshalling to try sharing all strings;
> it might be easier to revert the change to compile.c, though.

I've reverted the change.  I just I hope I remember this next time I
run regrtest.py with -R...

Cheers,
mwh

-- 
  Clue: You've got the appropriate amount of hostility for the
  Monastery, however you are metaphorically getting out of the
  safari jeep and kicking the lions.                         -- coonec
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
From jim at zope.com  Wed Aug  4 15:46:55 2004
From: jim at zope.com (Jim Fulton)
Date: Wed Aug  4 15:46:59 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <cepadg$dp2$1@sea.gmane.org>
References: <Your message of "Mon, 02 Aug 2004
	13:30:03	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org>	<410FC9FE.6060305@divmod.com>
	<cepadg$dp2$1@sea.gmane.org>
Message-ID: <4110E8CF.6070900@zope.com>

Terry Reedy wrote:

...

> Decorators will add complexity, regardless of syntax. 

Especially in their current over-powered form.  If decorators didn't have to
take arguments, then you could use a syntax like:

   def classmethod foo(cls, ...):
       ...

This does add complexity, but I think it enhances understandability.

I understand why some people want to be able to pass arguments to decorators,
but frankly, I find many of the examples of doing so down right scary.


 > An advantage of the
> @ syntax is than it makes the complexiity separable and ignorable for
> someone learning/teaching basic Python,

I don't agree that they are ignorable.  People will see them in code
and will *have* to understand what they mean.  Given sme examples
I've seen here, this will sometimes be a significant chalenge.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From jim at zope.com  Wed Aug  4 15:53:37 2004
From: jim at zope.com (Jim Fulton)
Date: Wed Aug  4 15:53:40 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <5.1.1.6.0.20040802125556.05c5bc20@mail.telecommunity.com>
References: <5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org> <Your message of "Wed,
	28 Jul 2004 13:40:03
	EDT."	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org>
	<5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>
	<5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com>
	<5.1.1.6.0.20040802125556.05c5bc20@mail.telecommunity.com>
Message-ID: <4110EA61.4010003@zope.com>

Phillip J. Eby wrote:
> At 10:01 AM 8/2/04 -0400, Jim Fulton wrote:
> 

...

> The point of my 
> proposal is to make it possible *without* rewriting imports, in versions 
> of Python from 2.2 up.

Your proposal would, at least, require canges to the pickle framework.
I imagine it would affect other serialization and persistence frameworks
as well.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From andrewm at object-craft.com.au  Wed Aug  4 16:04:42 2004
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Wed Aug  4 16:04:37 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators 
In-Reply-To: <4110E8CF.6070900@zope.com> 
References: <Your message of "Mon, 02 Aug 2004 13:30:03
	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org>
	<410FC9FE.6060305@divmod.com> <cepadg$dp2$1@sea.gmane.org>
	<4110E8CF.6070900@zope.com>
Message-ID: <20040804140442.89F2B3C7A0@coffee.object-craft.com.au>

> > An advantage of the
>> @ syntax is than it makes the complexiity separable and ignorable for
>> someone learning/teaching basic Python,
>
>I don't agree that they are ignorable.  People will see them in code
>and will *have* to understand what they mean.  Given sme examples
>I've seen here, this will sometimes be a significant chalenge.

There are a number of "advanced" features in python that could be a
barrier to understanding for new users, but which are rarely seen in
practice - metaclasses being the classic example.

I imagine these features are rarely seen because most people don't
understand them, and those that do have the good taste not to use them
gratuitously.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/
From astrand at lysator.liu.se  Wed Aug  4 16:05:22 2004
From: astrand at lysator.liu.se (Peter Astrand)
Date: Wed Aug  4 16:10:43 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <20040803213121.C9777@ActiveState.com>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
Message-ID: <Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>

> ProcessProxy:
>     ...a behemoth using a thread for each of stdin/stdout/stderr to
>     allow the user to get an event-like IO interface.

Why not avoid threads on POSIX systems, and use select instead? My module
does, although it does not provide an event-like IO interface. If you get
rid of the threads, then you don't need the workaround code for Linux.


> Some of the large code size is _for_ execution through the shell. :)  I
> think that execution via the shell should be a feature of a module like
> this (so that users can use some shell features) and I even think that
> it should be the default (so that, for example, Windows users don't have
> to learn what cmd.exe or command.com are to run "dir"). However, I
> absolutely agree that one should be able to run withOUT the shell (i.e.
> have an option for this).

You're right. My module should probably have an option for invoking
through the shell, or at least document how to do it. I really don't want
it as default, though.


> Other reasons for the size/complexity of my process.py over yours:
> - My process objects have a .kill() method -- which is actually quite a
>   pain on Windows.

True. I guess my module would benefit from such a method as well.


> - My module includes some handling that for subsystem:windows vs.
>   subsystem:console apps on Windows that I don't think yours does.

Can you describe why this is needed/useful?


> > * The modules uses destructors, which I'm usually avoiding.
>
> Is that evil?

Destructors interferes with the GC. Someone else can probably fill in the
details.


> concensus. In the meantime, if you'd be willing to change your module's
> name to something other than process.py I think that would help
> discussions. (I don't know if that is a pain for you at this point. You
> mentioned "subprocess".

I can change, but I'd like more feedback before that. No-one has told me
their opinion on the name "subprocess", for example, not even you :-)


>Alternatively, how about "posixprocess"? Though,
> despite PEP 324's title, I don't know if that is completely accurate
> anymore.)

Oh, I've forgotten to change the title. Yes, this is wrong, because the
module certainly aims to work on non-POSIX systems as well (read:
Windows).


/Peter ?strand <astrand@lysator.liu.se>

From guido at python.org  Wed Aug  4 16:18:55 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug  4 16:19:01 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: Your message of "Wed, 04 Aug 2004 17:03:31 +1200."
	<200408040503.i7453VQE030085@cosc353.cosc.canterbury.ac.nz> 
References: <200408040503.i7453VQE030085@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408041418.i74EIt001130@guido.python.org>

For the record, let me just say that I'm -1 on adding this feature
now.  It's only a minor convenience while it's a fairly big change to
the parser and code generator, but most of all, Python's growth needs
to be reduced.  If we were to add every "nice" feature that was
invented (or present in other languages), Python would lose one of its
main charms, which is that it is really a rather simple language that
can be learned and put to production quickly.

So while I appreciate the idea (which BTW has been proposed before) I
think it's not worth adding now, and if you don't hear from me again
on this topic it's because I haven't changed my mind...

--Guido van Rossum (home page: http://www.python.org/~guido/)
From jim at zope.com  Wed Aug  4 16:31:57 2004
From: jim at zope.com (Jim Fulton)
Date: Wed Aug  4 16:32:01 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <20040804140442.89F2B3C7A0@coffee.object-craft.com.au>
References: <Your message of "Mon, 02 Aug 2004
	13:30:03	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org>	<410FC9FE.6060305@divmod.com>
	<cepadg$dp2$1@sea.gmane.org>	<4110E8CF.6070900@zope.com>
	<20040804140442.89F2B3C7A0@coffee.object-craft.com.au>
Message-ID: <4110F35D.8020904@zope.com>

Andrew McNamara wrote:
>>>An advantage of the
>>>@ syntax is than it makes the complexiity separable and ignorable for
>>>someone learning/teaching basic Python,
>>
>>I don't agree that they are ignorable.  People will see them in code
>>and will *have* to understand what they mean.  Given sme examples
>>I've seen here, this will sometimes be a significant chalenge.
> 
> 
> There are a number of "advanced" features in python that could be a
> barrier to understanding for new users, but which are rarely seen in
> practice - metaclasses being the classic example.
> 
> I imagine these features are rarely seen because most people don't
> understand them, and those that do have the good taste not to use them
> gratuitously.

This feature won't be rarely used.  Class and static methods
and properties are created fairly often, certainly far more often
than meta classes.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From guido at python.org  Wed Aug  4 16:34:38 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug  4 16:34:45 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: Your message of "Wed, 04 Aug 2004 11:38:58 BST."
	<2m1xin9ohp.fsf@starship.python.net> 
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com>
	<20040803194724.GB9031@titan.progiciels-bpi.ca>
	<200408032014.i73KEwp31876@guido.python.org> 
	<2m1xin9ohp.fsf@starship.python.net> 
Message-ID: <200408041434.i74EYdU01260@guido.python.org>

> > Probably not, but making Exception a new-style class won't be easy.
> 
> What makes you say that?  I've just been remarking on
> comp.lang.python how having Exception be new-style in PyPy --
> indeed, not having old-style classes and all -- has caused
> essentially no problems at all.

I believe that -- it's more that the existing infrastructure that
creates the standard exception hierarchy isn't easily adapted.

I also believe there's a conceptual problem with defining when
something is an acceptable argument to 'raise' -- unless we insist
that exceptions inherit from a designated base class, *every* object
is acceptable, because if it isn't a class, it's an instance of a
class, and raise allows either.  I don't really think that "raise 42"
ought to be acceptable, but I don't know how to prevent it without
requiring a specific base class (excluding a whole slew of specific
base classes seems wrong).

Maybe we can accept old-style classes and instances, strings, and
instances of Exception and its subclasses.  But then we better be sure
that we really want to insist on subclassing from Exception, because
that's rather unpythonic.

> Perhaps I'll work up a patch sometime and see what breaks.

Please do!

--Guido van Rossum (home page: http://www.python.org/~guido/)
From bob at redivi.com  Wed Aug  4 16:50:24 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed Aug  4 16:50:30 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <4110E8CF.6070900@zope.com>
References: <Your message of "Mon, 02 Aug 2004
	13:30:03	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org>	<410FC9FE.6060305@divmod.com>
	<cepadg$dp2$1@sea.gmane.org> <4110E8CF.6070900@zope.com>
Message-ID: <9E2079F2-E625-11D8-B64E-000A95686CD8@redivi.com>


On Aug 4, 2004, at 9:46 AM, Jim Fulton wrote:

> Terry Reedy wrote:
>
> ...
>
>> Decorators will add complexity, regardless of syntax.
>
> Especially in their current over-powered form.  If decorators didn't 
> have to
> take arguments, then you could use a syntax like:
>
>   def classmethod foo(cls, ...):
>       ...
>
> This does add complexity, but I think it enhances understandability.
>
> I understand why some people want to be able to pass arguments to 
> decorators,
> but frankly, I find many of the examples of doing so down right scary.

All of the important use cases I've found and seen for decorators have 
arguments.  PyObjC and ctypes (possibly Jython, IronPython) will use 
them to specify the input and output argument types for a function 
(integers, pointers, etc.).  PJE has cooked up some interesting stuff 
using decorators to do generic functions.  I use these sorts of things 
far more often than classmethod and staticmethod (or the like).  I can 
also imagine using them for something like decorating functions with 
four character suite and event codes for responding to AppleEvents in 
Mac apps.

I wouldn't really care for decorators at all if I couldn't use them to 
these sorts of things.  Your proposal would make decorators nearly as 
bad as the current pre-2.4 situation.  This is REALLY ugly:

foo = decorator(....)
@foo
def fun(....):
	...

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3589 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040804/ec2570d1/smime.bin
From jim at zope.com  Wed Aug  4 16:52:25 2004
From: jim at zope.com (Jim Fulton)
Date: Wed Aug  4 16:52:29 2004
Subject: [Python-Dev] Plea for simpler decorator syntax,
	in addition to pie-shaped syntax
Message-ID: <4110F829.2000609@zope.com>


IMO, the most common uses of decorators will be to define properties,
and class and static methods.  IMO, these uses would be better served
by a simpler syntax:

   def classmethod foo(cls, ...):
       ...

This simplified syntax only allows names to specify decorators.  It
could allow multiple names, although I'm not sure it should,

I find this *far* more readable and obvious than any of the other syntaxs
I've seen propsed.

Those applications that *need* decorator arguments could use the more complex
pie-shaped notation.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From chrism at plope.com  Wed Aug  4 16:57:33 2004
From: chrism at plope.com (Chris McDonough)
Date: Wed Aug  4 16:57:37 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
Message-ID: <1091631453.26267.390.camel@athlon>

On Wed, 2004-08-04 at 10:05, Peter Astrand wrote:
> > ProcessProxy:
> >     ...a behemoth using a thread for each of stdin/stdout/stderr to
> >     allow the user to get an event-like IO interface.
> 
> Why not avoid threads on POSIX systems, and use select instead? My module
> does, although it does not provide an event-like IO interface. If you get
> rid of the threads, then you don't need the workaround code for Linux.

Doesn't select() effectively busy-wait on "real" files (pipes and file
descriptors obtained via open(), as opposed to network sockets) on most
(all?) UNIXen?  At least this has been my finding under Linux.

(See
http://www.plope.com/Members/chrism/News_Item.2004-07-29.4755190458/talkback/1091579983/discussionitem_view for more info).

- C


From jim at zope.com  Wed Aug  4 17:00:37 2004
From: jim at zope.com (Jim Fulton)
Date: Wed Aug  4 17:00:40 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <9E2079F2-E625-11D8-B64E-000A95686CD8@redivi.com>
References: <Your message of "Mon, 02 Aug 2004
	13:30:03	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org>	<410FC9FE.6060305@divmod.com>
	<cepadg$dp2$1@sea.gmane.org> <4110E8CF.6070900@zope.com>
	<9E2079F2-E625-11D8-B64E-000A95686CD8@redivi.com>
Message-ID: <4110FA15.2040808@zope.com>

Bob Ippolito wrote:
> 
...

 > Your proposal would make decorators nearly as
> bad as the current pre-2.4 situation.  This is REALLY ugly:
> 
> foo = decorator(....)
> @foo
> def fun(....):
>     ...

I agree, that's really ugly.  I wasn't proposing that.

I proposed not allowing decorator arguments. Without decorator arguments,
a much simpler syntax can be used, as in:

   def classmethod foo(...):
       ...

Or, alternatively, I'm proposing allowing the above simpler syntax
when arguments are not needed and allowing the pie syntax to handle
more complex cases.

The original motivation for decirators was to deal with things like
properties and specialized methods.  I'd like to see these cases handled
well.  I think these are going to be the most common cases and the cases
that new Python programmers are most likely to come accross.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From fdrake at acm.org  Wed Aug  4 17:00:34 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed Aug  4 17:00:45 2004
Subject: [Python-Dev] Plea for simpler decorator syntax,
	in addition to pie-shaped syntax
In-Reply-To: <4110F829.2000609@zope.com>
References: <4110F829.2000609@zope.com>
Message-ID: <200408041100.34439.fdrake@acm.org>

On Wednesday 04 August 2004 10:52 am, Jim Fulton wrote:
 > IMO, the most common uses of decorators will be to define properties,
 > and class and static methods.  IMO, these uses would be better served
 > by a simpler syntax:
 >
 >    def classmethod foo(cls, ...):
 >        ...

This was rejected a long time ago because it complicated life for editor 
colorizing support and many similar tools.  It especially complicates the 
creation of ad-hoc tools, and breaks ones that are already working.

While pie-notation isn't my favorite, it's reasonable enough.  The example

    @classmethod
    def makeAnother(cls):
        return cls("magic", 42)

seems readable enough to me.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From barry at python.org  Wed Aug  4 17:07:52 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug  4 17:07:46 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <20040804010712.GA10819@mems-exchange.org>
References: <410FC9FE.6060305@divmod.com> <cepadg$dp2$1@sea.gmane.org>
	<20040804010712.GA10819@mems-exchange.org>
Message-ID: <1091632072.8324.72.camel@localhost>

On Tue, 2004-08-03 at 21:07, Neil Schemenauer wrote:

> That's not apparent to me.  It seems to me that most people just got
> sick of the endless debate and tuned out.  The people who had urgent
> need for the feature were probably a bit more persistent.
> Personally, I don't see the cost vs. benefit of the feature to be
> compelling.  It seems there is not a lot you can do with a decorator
> without resorting to ugly hacks (e.g. messing with bytecode).

I disagree.  I've written a lot of code that uses pre-pie decorators,
but it's all fairly localized.  It isn't spread out in every module, but
it's contained, e.g., within a database package (wrap mutators in
transaction handling decorators).  Those use my own custom decorator,
but in almost all other examples (in my own code -- I know others have
different use cases), the decorators are built-ins like property,
staticmethod, etc.

For me, the benefit is significant because while I would still continue
to use this particular feature, I think the pie decorator syntax makes
the code /more/ readable, or more readable in the right way and in the
right place.

In general, I predict most Python code will continue to be blissfully
unadorned with decorators.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040804/2436bbc3/attachment-0001.pgp
From barry at python.org  Wed Aug  4 17:12:02 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug  4 17:11:55 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <4110E8CF.6070900@zope.com>
References: <Your message of "Mon, 02 Aug 2004
	13:30:03	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com>
	<200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com>
	<200408031700.i73H0MY31391@guido.python.org>	<410FC9FE.6060305@divmod.com>
	<cepadg$dp2$1@sea.gmane.org>  <4110E8CF.6070900@zope.com>
Message-ID: <1091632322.8323.81.camel@localhost>

On Wed, 2004-08-04 at 09:46, Jim Fulton wrote:

> I don't agree that they are ignorable.  People will see them in code
> and will *have* to understand what they mean.  Given sme examples
> I've seen here, this will sometimes be a significant chalenge.

I think most of the cognitive challenge for decorators will be in
understanding the side-effects of the decorator, not in understanding
the syntax for wrapping a function in a decorator.  That won't change by
using some other (or no new) syntax -- the Python 2.3. spelling would be
just as hairy.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040804/9776219e/attachment.pgp
From guido at python.org  Wed Aug  4 17:15:58 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug  4 17:16:03 2004
Subject: [Python-Dev] Plea for simpler decorator syntax,
	in addition to pie-shaped syntax
In-Reply-To: Your message of "Wed, 04 Aug 2004 10:52:25 EDT."
	<4110F829.2000609@zope.com> 
References: <4110F829.2000609@zope.com> 
Message-ID: <200408041515.i74FFxl01426@guido.python.org>

Jim, you should've argued this months ago.  Everything you bring up
has already been debated endlessly, and I really don't feel like
regurgitating the same answer again.  Please stop.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Wed Aug  4 17:20:48 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug  4 17:20:53 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: Your message of "Wed, 04 Aug 2004 10:57:33 EDT."
	<1091631453.26267.390.camel@athlon> 
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se> 
	<1091631453.26267.390.camel@athlon> 
Message-ID: <200408041520.i74FKm001469@guido.python.org>

> Doesn't select() effectively busy-wait on "real" files (pipes and file
> descriptors obtained via open(), as opposed to network sockets) on most
> (all?) UNIXen?  At least this has been my finding under Linux.

select() doesn't make sense for regular files, so tail -f can't use
it.  For Unix pipes, it works as advertised.  On Windows, it *only*
applies to sockets.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From FBatista at uniFON.com.ar  Wed Aug  4 17:17:38 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Wed Aug  4 17:21:27 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
Message-ID: <A128D751272CD411BC9200508BC2194D053C7749@escpl.tcp.com.ar>

Didn't followed closely this discussion, but my general feeling is that the
development community position is not clear regarding this feature.

Python is what it is now because of well thinked changes. I don't want,
because of the hurry of 2.4, to include this particular feature without a
hard approval of the community.

Will be easiest to include it in 2.5 than to extract it.

So, all that said, I'm +1 to take this out from 2.4.

.	Facundo


#- -----Mensaje original-----
#- De: Jim Fulton [mailto:jim@zope.com]
#- Enviado el: Miercoles, 04 de Agosto de 2004 11:32
#- Para: Andrew McNamara
#- CC: python-dev@python.org
#- Asunto: Re: [Python-Dev] Re: 2.4a2, and @decorators
#- 
#- 
#- Andrew McNamara wrote:
#- >>>An advantage of the
#- >>>@ syntax is than it makes the complexiity separable and 
#- ignorable for
#- >>>someone learning/teaching basic Python,
#- >>
#- >>I don't agree that they are ignorable.  People will see 
#- them in code
#- >>and will *have* to understand what they mean.  Given sme examples
#- >>I've seen here, this will sometimes be a significant chalenge.
#- > 
#- > 
#- > There are a number of "advanced" features in python that could be a
#- > barrier to understanding for new users, but which are 
#- rarely seen in
#- > practice - metaclasses being the classic example.
#- > 
#- > I imagine these features are rarely seen because most people don't
#- > understand them, and those that do have the good taste not 
#- to use them
#- > gratuitously.
#- 
#- This feature won't be rarely used.  Class and static methods
#- and properties are created fairly often, certainly far more often
#- than meta classes.
#- 
#- Jim
#- 
#- -- 
#- Jim Fulton           mailto:jim@zope.com       Python Powered!
#- CTO                  (540) 361-1714            http://www.python.org
#- Zope Corporation     http://www.zope.com       http://www.zope.org
#- _______________________________________________
#- Python-Dev mailing list
#- Python-Dev@python.org
#- http://mail.python.org/mailman/listinfo/python-dev
#- Unsubscribe: 
#- http://mail.python.org/mailman/options/python-dev/fbatista%40
unifon.com.ar
From guido at python.org  Wed Aug  4 17:22:20 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug  4 17:22:24 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: Your message of "Wed, 04 Aug 2004 11:07:52 EDT."
	<1091632072.8324.72.camel@localhost> 
References: <410FC9FE.6060305@divmod.com> <cepadg$dp2$1@sea.gmane.org>
	<20040804010712.GA10819@mems-exchange.org> 
	<1091632072.8324.72.camel@localhost> 
Message-ID: <200408041522.i74FMKW01491@guido.python.org>

> In general, I predict most Python code will continue to be
> blissfully unadorned with decorators.

Right.  Outside the test suite, there are very few uses of
classmethod, staticmethod and property in the standard library.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From bob at redivi.com  Wed Aug  4 17:45:17 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed Aug  4 17:45:26 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <4110FA15.2040808@zope.com>
References: <Your message of "Mon, 02 Aug 2004
	13:30:03	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org>	<410FC9FE.6060305@divmod.com>
	<cepadg$dp2$1@sea.gmane.org> <4110E8CF.6070900@zope.com>
	<9E2079F2-E625-11D8-B64E-000A95686CD8@redivi.com>
	<4110FA15.2040808@zope.com>
Message-ID: <48F450D5-E62D-11D8-B64E-000A95686CD8@redivi.com>


On Aug 4, 2004, at 11:00 AM, Jim Fulton wrote:

> Bob Ippolito wrote:
> ...
>
> > Your proposal would make decorators nearly as
>> bad as the current pre-2.4 situation.  This is REALLY ugly:
>> foo = decorator(....)
>> @foo
>> def fun(....):
>>     ...
>
> I agree, that's really ugly.  I wasn't proposing that.
>
> I proposed not allowing decorator arguments. Without decorator 
> arguments,
> a much simpler syntax can be used, as in:
>
>   def classmethod foo(...):
>       ...
>
> Or, alternatively, I'm proposing allowing the above simpler syntax
> when arguments are not needed and allowing the pie syntax to handle
> more complex cases.
>
> The original motivation for decirators was to deal with things like
> properties and specialized methods.  I'd like to see these cases 
> handled
> well.  I think these are going to be the most common cases and the 
> cases
> that new Python programmers are most likely to come accross.

Adding even more syntax sounds like a horrible idea (along with the 
fact that this particular syntax is hard to syntax highlight, will 
really confuse lots of existing tools, etc.).

I also don't agree that those are the most common cases.  I certainly 
haven't seen a whole lot of staticmethod/classmethod usage, and 
property is only useful for writing getters (with decorator syntax), so 
I don't think that will be used very often either.  I think that 
decorators will be used about 98% for domain specific purposes with 
non-standard decorators and 2% will be uses of decorators that come in 
the standard library such as staticmethod, classmethod, and property 
(for writing getters).

Anyway, it's probably too late to change anyone's mind.  Decorators are 
in @as-is, and I don't think anything you can say now is going to 
change that.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3589 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040804/a95fb78f/smime.bin
From janssen at parc.com  Wed Aug  4 17:47:32 2004
From: janssen at parc.com (Bill Janssen)
Date: Wed Aug  4 17:48:03 2004
Subject: [Python-Dev] Plea for simpler decorator syntax,
	in addition to pie-shaped syntax 
In-Reply-To: Your message of "Wed, 04 Aug 2004 08:00:34 PDT."
	<200408041100.34439.fdrake@acm.org> 
Message-ID: <04Aug4.084736pdt."58612"@synergy1.parc.xerox.com>

>  >    def classmethod foo(cls, ...):
>  >        ...
> 
> This was rejected a long time ago because it complicated life for editor 
> colorizing support

Oh, Fred, please!  "Complicated life"?  (and, "Editor colorizing
support"?!)  Much as I love python-mode, it hardly seems a good reason
to add a whole new syntactic branch to Python, particularly when
editors are really good at identifying a small set of distinguished
words in distinguished positions.

By the way, if we're going down the road of compile-time markup, how
about a real macro language :-?

Bill
From heikowu at ceosg.de  Wed Aug  4 18:09:54 2004
From: heikowu at ceosg.de (Heiko Wundram)
Date: Wed Aug  4 18:09:29 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <A128D751272CD411BC9200508BC2194D053C7749@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D053C7749@escpl.tcp.com.ar>
Message-ID: <200408041809.54164.heikowu@ceosg.de>

Am Mittwoch, 4. August 2004 17:17 schrieb Batista, Facundo:
> So, all that said, I'm +1 to take this out from 2.4.

-1000 to take it out from 2.4... And +1 on Guido's intuition for choosing the 
@ syntax (it goes easily for me). I'd love to see something of the following 
form:

class x:
	synchronized = threading.Synchronizer()

	@synchronized
	def y(self):
		<do something>

When's threading.Synchronizer coming (just a threading.(R)Lock with an extra 
__call__ which prepares a method for synchronization with this lock)? I 
already have some patches which implement module/class/instance locking using 
just a simple RLock and the decorator syntax, and I'd gladly sign over the 
patches to the PSF. ;)

Heiko.
From pje at telecommunity.com  Wed Aug  4 18:16:37 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Aug  4 18:12:33 2004
Subject: [Python-Dev] Fix import errors to have data
In-Reply-To: <4110EA61.4010003@zope.com>
References: <5.1.1.6.0.20040802125556.05c5bc20@mail.telecommunity.com>
	<5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org> <Your message of "Wed,
	28 Jul 2004 13:40:03
	EDT."	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<Your message of "Wed,
	28 Jul 2004 06:59:24 EDT." <4107870C.1070404@zope.com>
	<41065006.6020607@zope.com>
	<200407271445.i6REjEK24454@guido.python.org>
	<41067660.2070109@zope.com>
	<1f7befae040727085715ce2536@mail.gmail.com>
	<4106853A.8080109@zope.com>
	<20040727164913.GS7708@epoch.metaslash.com>
	<41068C2F.40102@zope.com>
	<200407272050.i6RKoEY25164@guido.python.org>
	<4107870C.1070404@zope.com>
	<5.1.1.6.0.20040728132215.031cf110@mail.telecommunity.com>
	<5.1.1.6.0.20040728165228.03123570@mail.telecommunity.com>
	<200407282213.i6SMDWD06878@guido.python.org>
	<5.1.1.6.0.20040729172447.032487a0@mail.telecommunity.com>
	<5.1.1.6.0.20040730105512.02145c00@mail.telecommunity.com>
	<5.1.1.6.0.20040802125556.05c5bc20@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040804115739.02c88050@mail.telecommunity.com>

At 09:53 AM 8/4/04 -0400, Jim Fulton wrote:
>Phillip J. Eby wrote:
>>At 10:01 AM 8/2/04 -0400, Jim Fulton wrote:
>
>..
>
>>The point of my proposal is to make it possible *without* rewriting 
>>imports, in versions of Python from 2.2 up.
>
>Your proposal would, at least, require canges to the pickle framework.
>I imagine it would affect other serialization and persistence frameworks
>as well.

Actually, in the updated interface I've been discussing off-list with a few 
people, all that would happen is that you'd need to be explicit about what 
interpreter you use to do the unpickling, if you don't want it to be the 
current one.  E.g.:

    result = someInterpreter.call(somePickler.loads, data)

In a framework like ZODB, however, I would assume that if supporting 
multiple interpreters were desirable, one would change the persistent 
object ID to include a reference to the {bundle|parcel|product} that 
originated the object.

In any case, the implementation we're discussing (based on the C multiple 
interpreter API) should be completely transparent to end user code, to the 
extent that it should be possible to run two different versions of Zope 
side by side in the same process -- and that will probably make a nice 
"stress test" for the implementation when we're done with it.  :)

I hope that the end-product of our work will be an implementation of a 
Python-accessible API for interpreter management, and a PEP covering both 
that API and how to fix the most problematic aspects of the Python C API 
for multiple interpreters.  (Specifically, additions to the API to allow 
extensions to manage selected static data as interpreter-specific rather 
than global, and for ensuring that imports stay interpreter-local when 
execution crosses interpreter boundaries.)  I expect the fixes will be able 
to be used with versions of Python back to 2.2, which was the version that 
introduced the necessary base APIs for walking the interpreter and thread 
state chains.

From pje at telecommunity.com  Wed Aug  4 18:22:51 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Aug  4 18:18:48 2004
Subject: [Python-Dev] Exception and new-style classes
In-Reply-To: <200408041434.i74EYdU01260@guido.python.org>
References: <Your message of "Wed,
	04 Aug 2004 11:38:58 BST." <2m1xin9ohp.fsf@starship.python.net>
	<20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com>
	<20040803194724.GB9031@titan.progiciels-bpi.ca>
	<200408032014.i73KEwp31876@guido.python.org>
	<2m1xin9ohp.fsf@starship.python.net>
Message-ID: <5.1.1.6.0.20040804121826.02d0c110@mail.telecommunity.com>

At 07:34 AM 8/4/04 -0700, Guido van Rossum wrote:
> > > Probably not, but making Exception a new-style class won't be easy.
> >
> > What makes you say that?  I've just been remarking on
> > comp.lang.python how having Exception be new-style in PyPy --
> > indeed, not having old-style classes and all -- has caused
> > essentially no problems at all.
>
>I believe that -- it's more that the existing infrastructure that
>creates the standard exception hierarchy isn't easily adapted.
>
>I also believe there's a conceptual problem with defining when
>something is an acceptable argument to 'raise' -- unless we insist
>that exceptions inherit from a designated base class, *every* object
>is acceptable, because if it isn't a class, it's an instance of a
>class, and raise allows either.  I don't really think that "raise 42"
>ought to be acceptable, but I don't know how to prevent it without
>requiring a specific base class (excluding a whole slew of specific
>base classes seems wrong).
>
>Maybe we can accept old-style classes and instances, strings, and
>instances of Exception and its subclasses.  But then we better be sure
>that we really want to insist on subclassing from Exception, because
>that's rather unpythonic.

I thought that was already the plan; I seem to recall dire warnings that it 
was going to be required someday.

In any case, I had my eye on "fixing" this for next bug day (there's a SF 
bug # for it, that I don't recall at the moment).

My plan was to allow any object that was an instance of Exception, even if 
it was new-style.  In other words, new-style exceptions would have to 
include Exception in their base classes.  Old-style exceptions wouldn't 
have to meet that requirement, for backward compatibility purposes.

I assumed that the old-style (and string) compatibility would need to 
remain until 3.0.

From astrand at lysator.liu.se  Wed Aug  4 18:25:36 2004
From: astrand at lysator.liu.se (Peter Astrand)
Date: Wed Aug  4 18:25:46 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <1091631453.26267.390.camel@athlon>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net> 
	<200408031427.i73ERn330790@guido.python.org> 
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se> 
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<1091631453.26267.390.camel@athlon>
Message-ID: <Pine.GSO.4.51L2.0408041822280.17161@koeberg.lysator.liu.se>

> > Why not avoid threads on POSIX systems, and use select instead? My module
> > does, although it does not provide an event-like IO interface. If you get
> > rid of the threads, then you don't need the workaround code for Linux.
>
> Doesn't select() effectively busy-wait on "real" files (pipes and file
> descriptors obtained via open(), as opposed to network sockets) on most
> (all?) UNIXen?  At least this has been my finding under Linux.

Yes, but a solution with threads will have problems as well, since read()
and friends will return EOF rather than block when you reach the end.


/Peter ?strand <astrand@lysator.liu.se>
From chrism at plope.com  Wed Aug  4 18:27:18 2004
From: chrism at plope.com (Chris McDonough)
Date: Wed Aug  4 18:27:21 2004
Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module))
In-Reply-To: <200408041520.i74FKm001469@guido.python.org>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<1091631453.26267.390.camel@athlon>
	<200408041520.i74FKm001469@guido.python.org>
Message-ID: <1091636838.26267.406.camel@athlon>

On Wed, 2004-08-04 at 11:20, Guido van Rossum wrote:
> > Doesn't select() effectively busy-wait on "real" files (pipes and file
> > descriptors obtained via open(), as opposed to network sockets) on most
> > (all?) UNIXen?  At least this has been my finding under Linux.
> 
> select() doesn't make sense for regular files, so tail -f can't use
> it.  For Unix pipes, it works as advertised.  On Windows, it *only*
> applies to sockets.

Aha.  I think I understand now.

Sorry for going off-topic, but this program (UNIX-only):

import select
import errno
import fcntl
import os

def go(r, timeout=1):
    while 1:
        try:
            r, w, x = select.select(r, [], [], timeout)
            print r
        except select.error, err:
            if err[0] != errno.EINTR:
                raise
            r = w = x = []

p_in, p_out = os.pipe()

for fd in p_in, p_out:
    flags = fcntl.fcntl(fd, fcntl.F_GETFL)
    fcntl.fcntl(fd, fcntl.F_SETFL,  flags | os.O_NDELAY)

os.close(p_in)

go([p_out])

... will busywait in select (p_out is always in the ready state; the
select timeout is never reached).

But if you comment out the "os.close(p_in)" line, it no longer busywaits
(the select timeout is reached on every iteration).  At least this is
the behavior under Linux.

This is a little unfortunate because the normal dance when communicating
between parent and child in order to capture the output of a child
process seems to be:

1) In the parent process, create a set of pipes that will represent
stdin/stdout/stderr of the child.

2)  fork

3) In the parent process, close the "unused" fds representing the ends
of the pipes that the child will use as fds for stdin/stderr/stdout.

4) In the child process, dup these same fds to stdin, stdout, and
stderr.

... so select seems to be unuseful here, unless step 3 isn't actually
necessary.

Still a bit confused and researching...

- C


From pje at telecommunity.com  Wed Aug  4 18:36:09 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Aug  4 18:32:05 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <200408041809.54164.heikowu@ceosg.de>
References: <A128D751272CD411BC9200508BC2194D053C7749@escpl.tcp.com.ar>
	<A128D751272CD411BC9200508BC2194D053C7749@escpl.tcp.com.ar>
Message-ID: <5.1.1.6.0.20040804123418.032619b0@mail.telecommunity.com>

At 06:09 PM 8/4/04 +0200, Heiko Wundram wrote:
>Am Mittwoch, 4. August 2004 17:17 schrieb Batista, Facundo:
> > So, all that said, I'm +1 to take this out from 2.4.
>
>-1000 to take it out from 2.4... And +1 on Guido's intuition for choosing the
>@ syntax (it goes easily for me). I'd love to see something of the following
>form:
>
>class x:
>         synchronized = threading.Synchronizer()
>
>         @synchronized
>         def y(self):
>                 <do something>
>
>When's threading.Synchronizer coming (just a threading.(R)Lock with an extra
>__call__ which prepares a method for synchronization with this lock)? I
>already have some patches which implement module/class/instance locking using
>just a simple RLock and the decorator syntax, and I'd gladly sign over the
>patches to the PSF. ;)

Note that your example, if I understand it correctly, creates a single lock 
for all instances of class 'x', rather than for individual instances of 
'x'.  This is not what I'd normally expect from a 'synchronized' method.

From amk at amk.ca  Wed Aug  4 18:35:45 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Wed Aug  4 18:35:55 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <200408041809.54164.heikowu@ceosg.de>
References: <A128D751272CD411BC9200508BC2194D053C7749@escpl.tcp.com.ar>
	<200408041809.54164.heikowu@ceosg.de>
Message-ID: <20040804163545.GB21551@rogue.amk.ca>

On Wed, Aug 04, 2004 at 06:09:54PM +0200, Heiko Wundram wrote:
> When's threading.Synchronizer coming (just a threading.(R)Lock with an extra 
> __call__ which prepares a method for synchronization with this lock)? I 
> already have some patches which implement module/class/instance locking using 
There was a python-dev thread about useful decorators, starting here:
http://mail.python.org/pipermail/python-dev/2004-April/043957.html
I expect 2.4alpha3 will see the addition of a 'decorators' module.

--amk
From jacobs at theopalgroup.com  Wed Aug  4 18:35:56 2004
From: jacobs at theopalgroup.com (Kevin Jacobs)
Date: Wed Aug  4 18:35:59 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <5.1.1.6.0.20040804123418.032619b0@mail.telecommunity.com>
References: <A128D751272CD411BC9200508BC2194D053C7749@escpl.tcp.com.ar>	<A128D751272CD411BC9200508BC2194D053C7749@escpl.tcp.com.ar>
	<5.1.1.6.0.20040804123418.032619b0@mail.telecommunity.com>
Message-ID: <4111106C.4020300@theopalgroup.com>

Phillip J. Eby wrote:

> At 06:09 PM 8/4/04 +0200, Heiko Wundram wrote:
>
>> class x:
>>         synchronized = threading.Synchronizer()
>>
>>         @synchronized
>>         def y(self):
>>                 <do something>
>>
>> When's threading.Synchronizer coming (just a threading.(R)Lock with 
>> an extra
>> __call__ which prepares a method for synchronization with this lock)? I
>> already have some patches which implement module/class/instance 
>> locking using
>> just a simple RLock and the decorator syntax, and I'd gladly sign 
>> over the
>> patches to the PSF. ;)
>
>
> Note that your example, if I understand it correctly, creates a single 
> lock for all instances of class 'x', rather than for individual 
> instances of 'x'.  This is not what I'd normally expect from a 
> 'synchronized' method.


Indeed it would, if implemented as described.  However, one
could use the same syntax and implement a Synchronizer
that implemented locks at the instance level.  Dirty tricks would
be involved, but it is possible.  (If I _ever_ get some free time,
I will take a stab at doing it, too.  Sounds like a good cookbook
recipe)

-Kevin
From foom at fuhm.net  Wed Aug  4 18:39:53 2004
From: foom at fuhm.net (James Y Knight)
Date: Wed Aug  4 18:39:57 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <eppstein-21570B.13221103082004@sea.gmane.org>
References: <1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<C90D4FDA-E4C8-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408030105.i7315Fj29310@guido.python.org>
	<110DAF75-E504-11D8-8D12-000A95A50FB2@fuhm.net>
	<200408031440.i73Ee6V30875@guido.python.org>
	<89EC7888-E580-11D8-8D12-000A95A50FB2@fuhm.net>
	<eppstein-21570B.13221103082004@sea.gmane.org>
Message-ID: <E97329A7-E634-11D8-8D12-000A95A50FB2@fuhm.net>


On Aug 3, 2004, at 4:22 PM, David Eppstein wrote:

> I think I was the one who posted the lambda comment.
> I also posted in a different message a use-case for decorators that had
> a line with two lambdas in it, so obviously I don't hate lambdas.

Ah, I'm sorry then. I have noticed many people seem to be quite biased 
against lambdas here, so I just assumed.

> I'm actually pretty neutral on restricted decorators vs @expression.  
> But
>    @lambda(func): body
>    def func(...): ...
> is (I think) much more likely to be an abuse of language than the
> Pythonic way of writing something, so I don't see a lot of harm in
> preventing it.

Oh, I agree, it's quite silly to write a piece of code like that, and 
if anyone wrote it in code I have to maintain I'd probably be quite 
irritated.

So, if the conclusion to this is "well, we can always unrestrict it 
later, if someone comes up with a convincing use-case", then it'll 
likely never be changed, because I doubt that's gonna happen. The best 
I can do is foo().bar(), which I suspect at least one person will run 
into at some point and wonder about, then work around instead of 
presenting it to python-dev.

My main argument is and has been simply: that's how everything else in 
python works, so this should be the same.

Last message on this subject, I think everyone got the point by now and 
is either convinced or not. :)

James

From heikowu at ceosg.de  Wed Aug  4 18:43:11 2004
From: heikowu at ceosg.de (Heiko Wundram)
Date: Wed Aug  4 18:42:46 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <4111106C.4020300@theopalgroup.com>
References: <A128D751272CD411BC9200508BC2194D053C7749@escpl.tcp.com.ar>
	<5.1.1.6.0.20040804123418.032619b0@mail.telecommunity.com>
	<4111106C.4020300@theopalgroup.com>
Message-ID: <200408041843.11994.heikowu@ceosg.de>

Am Mittwoch, 4. August 2004 18:35 schrieb Kevin Jacobs:
> > Note that your example, if I understand it correctly, creates a single
> > lock for all instances of class 'x', rather than for individual
> > instances of 'x'.  This is not what I'd normally expect from a
> > 'synchronized' method.
>
> Indeed it would, if implemented as described.  However, one
> could use the same syntax and implement a Synchronizer
> that implemented locks at the instance level.  Dirty tricks would
> be involved, but it is possible.  (If I _ever_ get some free time,
> I will take a stab at doing it, too.  Sounds like a good cookbook
> recipe)

It's pretty easy to implement a Synchronizer on instance level using the same 
semantics, as all method calls will receive the self parameter as the first 
parameter (except on builtin types, but you could also deal with this). I'll 
upload the patches I did to the sourceforge patch tracker once I get time to 
do so...

Anyway, I think this should be part of the threading module (as it's about 
threads, not about decorators), and should also be part of the stdlib, as 
there's lots of things that can go wrong if you try to implement it yourself, 
and this is useful enough to warrant inclusion in the stdlib...

Heiko.
From chrism at plope.com  Wed Aug  4 18:46:56 2004
From: chrism at plope.com (Chris McDonough)
Date: Wed Aug  4 18:47:00 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <Pine.GSO.4.51L2.0408041822280.17161@koeberg.lysator.liu.se>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<1091631453.26267.390.camel@athlon>
	<Pine.GSO.4.51L2.0408041822280.17161@koeberg.lysator.liu.se>
Message-ID: <1091638016.26267.418.camel@athlon>

On Wed, 2004-08-04 at 12:25, Peter Astrand wrote:
> > > Why not avoid threads on POSIX systems, and use select instead? My module
> > > does, although it does not provide an event-like IO interface. If you get
> > > rid of the threads, then you don't need the workaround code for Linux.
> >
> > Doesn't select() effectively busy-wait on "real" files (pipes and file
> > descriptors obtained via open(), as opposed to network sockets) on most
> > (all?) UNIXen?  At least this has been my finding under Linux.
> 
> Yes, but a solution with threads will have problems as well, since read()
> and friends will return EOF rather than block when you reach the end.

As I've found, if the end of the pipes in that represent the child's
stderr/stdout fds are closed in the parent, a select() reading the
"other" ends of these pipes will busywaut (at least on Linux).  Other
than than, I think this  select() would be the absolute right thing on
POSIX rather than using threads or polling.  Is there a way around this
problem or is it just a fact of life?

- C


From skip at pobox.com  Wed Aug  4 19:26:27 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Aug  4 19:26:53 2004
Subject: [Python-Dev] Plea for simpler decorator syntax,
	in addition to pie-shaped syntax 
In-Reply-To: <04Aug4.084736pdt."58612"@synergy1.parc.xerox.com>
References: <200408041100.34439.fdrake@acm.org>
	<04Aug4.084736pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <16657.7235.345770.620493@montanaro.dyndns.org>


    Fred> This was rejected a long time ago because it complicated life for
    Fred> editor colorizing support

    Bill> Much as I love python-mode, it hardly seems a good reason to add a
    Bill> whole new syntactic branch to Python, ..

Though X/Emacs is popular within the Unix community, it's hardly the only
editor that attempts to colorize Python code:

    http://www.python.org/cgi-bin/moinmoin/PythonEditors

In fact, given the increased use of Python on Windows these days I'm
skeptical it's the most widely used editor for Python code.

(Not breaking etags/ctags is also a nice-to-have.)

Skip
From anthony at interlink.com.au  Wed Aug  4 20:21:23 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Aug  4 20:21:58 2004
Subject: [Python-Dev] trunk frozen for 2.4a2
Message-ID: <41112923.8060505@interlink.com.au>

Please hold off on any checkins to the trunk for the next 24
hours or so (I'll post a followup when we're done) for 2.4a2.
(The usual exception for the release team of me, Fred and
Martin applies).

Once the second alpha is out the door, I plan to put together
a list of items that are still pending for 2.4 - any that can't
find someone to do the work will have to seriously be considered
at risk. At the moment, the major piece of orphaned work is the
relative imports and multi-line imports. The only other piece of
major unfinished work that springs to mind is the $string
interpolation, but there's a bunch of other smaller pieces. I
updated the release PEP after the first alpha - it's probably
useful for a few folks to scan through this and see if I missed
anything.

Ta,
Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From janssen at parc.com  Wed Aug  4 20:26:36 2004
From: janssen at parc.com (Bill Janssen)
Date: Wed Aug  4 20:28:17 2004
Subject: [Python-Dev] Plea for simpler decorator syntax,
	in addition to pie-shaped syntax 
In-Reply-To: Your message of "Wed, 04 Aug 2004 10:26:27 PDT."
	<16657.7235.345770.620493@montanaro.dyndns.org> 
Message-ID: <04Aug4.112642pdt."58612"@synergy1.parc.xerox.com>

> In fact, given the increased use of Python on Windows these days I'm
> skeptical it's the most widely used editor for Python code.

Didn't mean to imply that it was, it's just what I use.

By the way, I had firmly resolved to keep quiet about this decorators
issue, but I lost my willpower there for a minute :0).

Bill
From trentm at ActiveState.com  Wed Aug  4 20:30:56 2004
From: trentm at ActiveState.com (Trent Mick)
Date: Wed Aug  4 20:31:06 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>;
	from astrand@lysator.liu.se on Wed, Aug 04, 2004 at 04:05:22PM
	+0200
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
Message-ID: <20040804113056.H3207@ActiveState.com>

[Peter Astrand wrote]
> > ProcessProxy:
> >     ...a behemoth using a thread for each of stdin/stdout/stderr to
> >     allow the user to get an event-like IO interface.
> 
> Why not avoid threads on POSIX systems, and use select instead? My module
> does, although it does not provide an event-like IO interface. 

Because that wouldn't be cross-platform... perhaps it would be possible
though. I am not that experienced with select() -- I have generally
eschewed it because I can't use it on Windows as well.

> If you get rid of the threads, then you don't need the workaround code
> for Linux.

Slight misunderstanding there: the separate thread from which you cannot
kill a subprocess on Linux is not one of these ProcessProxy threads.
I.e. ignoring ProcessProxy the LinuxThreads-bug workaround is still
necessary for Process and ProcessOpen for the user that starts a
subprocess on their thread-A and wants to kill it on their thread-B.


> You're right. My module should probably have an option for invoking
> through the shell, or at least document how to do it. I really don't want
> it as default, though.

I have code to find the shell adequately so I don't see why we couldn't
use it. As to making it the default or not: perhaps clear documentation
could help avoid the expected "I can't run 'dir'!" newbie complaint, but
I don't think it would get all of the them. The current popen*() execute
via the shell. The lower-level guys, exec*() and spawn*(), do not. Still
up for debate, I guess. :)


> > - My module includes some handling that for subsystem:windows vs.
> >   subsystem:console apps on Windows that I don't think yours does.
> 
> Can you describe why this is needed/useful?

A subsystem:windows app (like Komodo, or Idle, or pythonw.exe, or any
GUI app) doesn't have a console and hence doesn't have std handles. By
default, executing any subsytem:console app (like 'ls' or 'echo' or
'python' or 'perl) from a subsystem:windows app will result in an
AllocConsole call
(http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/allocconsole.asp)
that will result in a console dialog popping up.  You can see this on
some Windows apps that haven't dealt with this issue. In particular,
some installers out there will pop up an unprofessional console dialog
when running some sub-processes.  To avoid this one has to muck around
with CreateProcess options.  Or, kind of the reverse, if a seeming GUI
app *does* have std handles (often redirected to log files, Komodo does
this) and it wants to offer the ability to pop up a console (e.g.
Komodo's feature to debug in a separate console) then one needs the
ability to specify CREATE_NEW_CONSOLE.


> I can change, but I'd like more feedback before that. No-one has told me
> their opinion on the name "subprocess", for example, not even you :-)

Sure, I don't mind "subprocess". I am hoping, though, that we can
eventually merge our two modules and just call that one "process".


Trent

-- 
Trent Mick
TrentM@ActiveState.com
From tjreedy at udel.edu  Wed Aug  4 20:40:08 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed Aug  4 20:40:17 2004
Subject: [Python-Dev] Re: Re: 2.4a2, and @decorators
References: <Your message of "Mon, 02 Aug
	200413:30:03	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org>	<410FC9FE.6060305@divmod.com><cepadg$dp2$1@sea.gmane.org>
	<4110E8CF.6070900@zope.com>
Message-ID: <ceraia$pbi$1@sea.gmane.org>


"Jim Fulton" <jim@zope.com> wrote in message
news:4110E8CF.6070900@zope.com...
> Terry Reedy wrote:
>  > An advantage of the
> > @ syntax is than it makes the complexiity separable and ignorable for
> > someone learning/teaching basic Python,
>
> I don't agree that they are ignorable.

Any advanced feature is ignorable is the context I specified -- 
learning/teaching basic Python.

>  People will see them in code  and will *have* to understand what they
mean.

Any ignored feature becomes unignorable when one stumbles across it in
'wild' Python and want to really understand.  And even with an
understanding of the general feature, specific knowledge may also be
required -- of the particular module or objects imported or in this case,
the particular decorator.

But there is currently *lots* of publicly available code without decorators
to read, and more will be written.  In addition, one will be able to use at
least some modules that use them without knowing about them, just as with
metaclasses, for instance.

My particular point was separability.  With @, one will still be able to
learn everything about [] and def without hearing the word decorator, and
without having to revise one's understanding upon encountering decorators.
This was not true of the [deco] and deco within def syntax proposals.  It
is a minor point, but one that is a plus for me.

Terry J. Reedy



From jhylton at gmail.com  Wed Aug  4 20:45:54 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Wed Aug  4 20:45:56 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <4110E8CF.6070900@zope.com>
References: <Your message of "Mon, 02 Aug 2004
	13:30:03	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org>	<410FC9FE.6060305@divmod.com>
	<cepadg$dp2$1@sea.gmane.org> <4110E8CF.6070900@zope.com>
Message-ID: <e8bf7a53040804114536017369@mail.gmail.com>

On Wed, 04 Aug 2004 09:46:55 -0400, Jim Fulton <jim@zope.com> wrote:
> Terry Reedy wrote:
> 
> ....
> 
> > Decorators will add complexity, regardless of syntax.
> 
> Especially in their current over-powered form.  If decorators didn't have to
> take arguments, then you could use a syntax like:
> 
>    def classmethod foo(cls, ...):
>        ...
> 
> This does add complexity, but I think it enhances understandability.
> 
> I understand why some people want to be able to pass arguments to decorators,
> but frankly, I find many of the examples of doing so down right scary.

I wonder if we should have add a note to the Python style guide that
forbids such things from the standard library <0.2 wink>.
 
>  > An advantage of the
> > @ syntax is than it makes the complexiity separable and ignorable for
> > someone learning/teaching basic Python,
> 
> I don't agree that they are ignorable.  People will see them in code
> and will *have* to understand what they mean.  Given sme examples
> I've seen here, this will sometimes be a significant chalenge.

It certainly depends on the use.  I don't think @classmethod will be
any more or less confusing than the current practice of calling foo =
staticmethod(foo) after the def.  I agree that more elaborate uses may
be confusing; then again, I suspect that more elaborate code will be
confusing regardless of how it uses decorators.

Jeremy
From astrand at lysator.liu.se  Wed Aug  4 21:01:22 2004
From: astrand at lysator.liu.se (Peter Astrand)
Date: Wed Aug  4 21:01:30 2004
Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module))
In-Reply-To: <1091636838.26267.406.camel@athlon>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net> 
	<200408031427.i73ERn330790@guido.python.org> 
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se> 
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<1091631453.26267.390.camel@athlon>
	<200408041520.i74FKm001469@guido.python.org>
	<1091636838.26267.406.camel@athlon>
Message-ID: <Pine.GSO.4.51L2.0408042054130.18090@koeberg.lysator.liu.se>

> Sorry for going off-topic, but this program (UNIX-only):

> ... will busywait in select (p_out is always in the ready state; the
> select timeout is never reached).
>
> But if you comment out the "os.close(p_in)" line, it no longer busywaits
> (the select timeout is reached on every iteration).  At least this is
> the behavior under Linux.

This isn't strange. You are closing the (only) read-end of the pipe. When
you do this, the pipe is broken. Consider this:

>>> import os
>>> r, w = os.pipe()
>>> os.close(r)
>>> os.write(w, "a")
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
OSError: [Errno 32] Broken pipe

select() only indicates that something has happened on this
filedescriptor.

> This is a little unfortunate because the normal dance when communicating
> between parent and child in order to capture the output of a child
> process seems to be:
>
> 1) In the parent process, create a set of pipes that will represent
> stdin/stdout/stderr of the child.
>
> 2)  fork

The problem with your example was that it didn't fork...

So, there is no problem with using select() on pipes when communicating
with a subprocess. It works great. Take a look at (my) process.py's
communicate() method for some inspiration.

/Peter ?strand <astrand@lysator.liu.se>

From tim.peters at gmail.com  Wed Aug  4 21:03:26 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Aug  4 21:03:28 2004
Subject: [Python-Dev] No handlers could be found for logger "cookielib"
Message-ID: <1f7befae0408041203475de3d9@mail.gmail.com>

I first saw this last night, when running the -all tests on WinXP, CVS HEAD:

...
test_univnewlines
test_unpack
test_urllib
test_urllib2
No handlers could be found for logger "cookielib"   HERE
test_urllibnet
...

It didn't act the same way in isolation.  Anyone have a clue?  The
test still passes, it just spits out that unhelpful logger output --
sometimes.
From astrand at lysator.liu.se  Wed Aug  4 21:10:58 2004
From: astrand at lysator.liu.se (Peter Astrand)
Date: Wed Aug  4 21:16:14 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <20040804113056.H3207@ActiveState.com>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<20040804113056.H3207@ActiveState.com>
Message-ID: <Pine.GSO.4.51L2.0408042101510.18090@koeberg.lysator.liu.se>

> > Why not avoid threads on POSIX systems, and use select instead? My module
> > does, although it does not provide an event-like IO interface.
>
> Because that wouldn't be cross-platform... perhaps it would be possible
> though. I am not that experienced with select() -- I have generally
> eschewed it because I can't use it on Windows as well.

The best way, IMHO, is to use select() on POSIX systems and threads on
Windows. This is what my module does.


> > Can you describe why this is needed/useful?
>
> A subsystem:windows app (like Komodo, or Idle, or pythonw.exe, or any
> GUI app) doesn't have a console and hence doesn't have std handles. By
> default, executing any subsytem:console app (like 'ls' or 'echo' or
> 'python' or 'perl) from a subsystem:windows app will result in an
> AllocConsole call
> (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/allocconsole.asp)
> that will result in a console dialog popping up.  You can see this on
> some Windows apps that haven't dealt with this issue. In particular,
> some installers out there will pop up an unprofessional console dialog
> when running some sub-processes.  To avoid this one has to muck around
> with CreateProcess options.  Or, kind of the reverse, if a seeming GUI
> app *does* have std handles (often redirected to log files, Komodo does
> this) and it wants to offer the ability to pop up a console (e.g.
> Komodo's feature to debug in a separate console) then one needs the
> ability to specify CREATE_NEW_CONSOLE.

I see what you mean. My module actually supports passing all types of
flags to CreateProcess, including CREATE_NEW_CONSOLE, so I would say that
my module deals with these issues. Well, I don't have any special boolean
flag or something like that for this specific case, but I see no reason
to.


> > I can change, but I'd like more feedback before that. No-one has told me
> > their opinion on the name "subprocess", for example, not even you :-)
>
> Sure, I don't mind "subprocess". I am hoping, though, that we can
> eventually merge our two modules and just call that one "process".

I don't really see what we could gain from merging our modules. What we
have now is two different modules with two different APIs, and
applications which uses these. If we were to merge our modules, then
the API of either your, mine or both modules would have to change, which
means that the applications using these would not work with the merged
module.

I'm positive to cooperating with you, and perhaps borrow code and ideas
from your module though (and you can of course borrow from mine).


/Peter ?strand <astrand@lysator.liu.se>

From python at rcn.com  Wed Aug  4 09:16:54 2004
From: python at rcn.com (Raymond Hettinger)
Date: Wed Aug  4 21:18:37 2004
Subject: [Python-Dev] trunk frozen for 2.4a2
In-Reply-To: <41112923.8060505@interlink.com.au>
Message-ID: <004f01c479f3$06327380$e841fea9@oemcomputer>

[Anthony Baxter]
> I plan to put together
> a list of items that are still pending for 2.4 - any that can't
> find someone to do the work will have to seriously be considered
> at risk. . . . The only other piece of
> major unfinished work that springs to mind is the $string
> interpolation, 

If Barry doesn't have time for this one, I'll work on it.


Raymond

From trentm at ActiveState.com  Wed Aug  4 21:39:18 2004
From: trentm at ActiveState.com (Trent Mick)
Date: Wed Aug  4 21:39:24 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: <Pine.GSO.4.51L2.0408042101510.18090@koeberg.lysator.liu.se>;
	from astrand@lysator.liu.se on Wed, Aug 04, 2004 at 09:10:58PM
	+0200
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<20040804113056.H3207@ActiveState.com>
	<Pine.GSO.4.51L2.0408042101510.18090@koeberg.lysator.liu.se>
Message-ID: <20040804123918.C7039@ActiveState.com>

[Peter Astrand wrote]
> > > Why not avoid threads on POSIX systems, and use select instead? My module
> > > does, although it does not provide an event-like IO interface.
> >
> > Because that wouldn't be cross-platform... perhaps it would be possible
> > though. I am not that experienced with select() -- I have generally
> > eschewed it because I can't use it on Windows as well.
> 
> The best way, IMHO, is to use select() on POSIX systems and threads on
> Windows. This is what my module does.

I'm willing to check that out for ProcessProxy. However, as I said, I
would be willing to remove ProcessProxy to a separate module (perhaps
just used by Komodo -- which was the original motivation for me) if it
is deemed not necessary/worthy for the core.


> I don't really see what we could gain from merging our modules.

One process control module to rule them all. Muwahahaha!
Seriously, I'd like to see "one obvious way to do it" for process
control in the core at some point (perhaps for Python 2.5) and my guess
is that a something between our two can provide it.

> What we have now is two different modules with two different APIs, and
> applications which uses these. If we were to merge our modules, then
> the API of either your, mine or both modules would have to change, which
> means that the applications using these would not work with the merged
> module.

I can change Komodo. And I don't use Twisted so I can break them.
(Muwahaha!)  ... more seriously I think a good final API (even if it
break backwards compat a little bit) is fine if the end result is a
better module for the core -- and as long as there is a __version__
attribute or something that users can key on.

Trent

-- 
Trent Mick
TrentM@ActiveState.com
From chrism at plope.com  Wed Aug  4 22:03:51 2004
From: chrism at plope.com (Chris McDonough)
Date: Wed Aug  4 22:03:57 2004
Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module))
In-Reply-To: <Pine.GSO.4.51L2.0408042054130.18090@koeberg.lysator.liu.se>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<1091631453.26267.390.camel@athlon>
	<200408041520.i74FKm001469@guido.python.org>
	<1091636838.26267.406.camel@athlon>
	<Pine.GSO.4.51L2.0408042054130.18090@koeberg.lysator.liu.se>
Message-ID: <1091649831.26267.527.camel@athlon>

On Wed, 2004-08-04 at 15:01, Peter Astrand wrote:
> >
> > But if you comment out the "os.close(p_in)" line, it no longer busywaits
> > (the select timeout is reached on every iteration).  At least this is
> > the behavior under Linux.
> 
> This isn't strange. You are closing the (only) read-end of the pipe. When
> you do this, the pipe is broken. Consider this:
> 
> >>> import os
> >>> r, w = os.pipe()
> >>> os.close(r)
> >>> os.write(w, "a")
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> OSError: [Errno 32] Broken pipe
> 
> select() only indicates that something has happened on this
> filedescriptor.

Right.  I understand that returning EOF on the reader side of the pipe
is the inverse of the "broken pipe" behavior you demonstrate above.

> > This is a little unfortunate because the normal dance when communicating
> > between parent and child in order to capture the output of a child
> > process seems to be:
> >
> > 1) In the parent process, create a set of pipes that will represent
> > stdin/stdout/stderr of the child.
> >
> > 2)  fork
> 
> The problem with your example was that it didn't fork...

I was all set to try to refute this, but after writing a minimal test
program to do what I want to do, I find that you're right.  That's good
news!  I'll need to revisit my workaroudns in the program that caused me
to need to do this.  Thanks for the schooling.

> So, there is no problem with using select() on pipes when communicating
> with a subprocess. It works great. Take a look at (my) process.py's
> communicate() method for some inspiration.

I've actually looked at it and it's quite nice, but it doesn't do one
thing that I'd like to see as part of a process stdlib library.  The use
case I'm thinking of is one where a long-running program needs to
monitor the output of many other potentially long-running processes,
doing other things in the meantime.  This kind of program tends to use
select as a part of a mainloop where there might be other things going
on (like handling network communications to/from sockets, updating a
GUI, etc).  Also, the output from child stderr and stdout potentially
never end because the child process(es) may never end.

In popen5, "communicate" is terminal.  It calls select until there's no
more data to get back and then unconditionally waits for the subprocess
to finish, blocking the entire time.  This isn't useful for the type of
program I describe above.  Of course, it wasn't meant to be, but having
an API that could help with this sort of thing would be nice as well,
although probably out of scope for PEP 234.

- C



From chrism at plope.com  Wed Aug  4 22:15:50 2004
From: chrism at plope.com (Chris McDonough)
Date: Wed Aug  4 22:15:54 2004
Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module))
In-Reply-To: <1091649831.26267.527.camel@athlon>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<1091631453.26267.390.camel@athlon>
	<200408041520.i74FKm001469@guido.python.org>
	<1091636838.26267.406.camel@athlon>
	<Pine.GSO.4.51L2.0408042054130.18090@koeberg.lysator.liu.se>
	<1091649831.26267.527.camel@athlon>
Message-ID: <1091650550.26267.538.camel@athlon>

On Wed, 2004-08-04 at 16:03, Chris McDonough wrote:
> I was all set to try to refute this, but after writing a minimal test
> program to do what I want to do, I find that you're right.  That's good
> news!  I'll need to revisit my workaroudns in the program that caused me
> to need to do this.  Thanks for the schooling.

Ugh.  I spoke a bit too soon..

The following program demonstrates that a particular usage of select
(under Linux at least) always returns the output side of a pipe
connected to a child process' stdout as "ready" after it gets any output
from that child process, even if the child process has no further data
to provide after it has provided a bit of data to the parent.  This is
what causes the "busywait" behavior I've experienced in the past (note
that as this program runs, your CPU utilization will likely be near
100%).

Or am I doing something silly?

import select
import errno
import fcntl
import os
import stat
import sys
from fcntl import F_SETFL, F_GETFL

def get_path():
    """Return a list corresponding to $PATH, or a default."""
    path = ["/bin", "/usr/bin", "/usr/local/bin"]
    if os.environ.has_key("PATH"):
        p = os.environ["PATH"]
        if p:
            path = p.split(os.pathsep)
    return path

def get_execv_args(command):
    """Internal: turn a program name into a file name, using 
    $PATH."""
    commandargs = command.split()
    program = commandargs[0]
    if "/" in program:
        filename = program
        try:
            st = os.stat(filename)
        except os.error:
            return None, None
    else:
        path = get_path()
        for dir in path:
            filename = os.path.join(dir, program)
            try:
                st = os.stat(filename)
            except os.error:
                continue
            mode = st[stat.ST_MODE]
            if mode & 0111:
                break
        else:
            return None, None
    if not os.access(filename, os.X_OK):
        return None, None
    return filename, commandargs

def spawn(command):
    """Start the subprocess."""

    filename, argv = get_execv_args(command)
    if filename is None:
        raise RuntimeError, '%s is an invalid command' % command

    child_stdin, stdin = os.pipe()
    stdout, child_stdout = os.pipe()
    stderr, child_stderr = os.pipe()

    # open stderr, stdout in nonblocking mode so we can tail them
    # in the mainloop without blocking
    for fd in stdout, stderr:
        flags = fcntl.fcntl(fd, F_GETFL)
        fcntl.fcntl(fd, F_SETFL, flags | os.O_NDELAY)

    pid = os.fork()

    if pid != 0:
        # Parent
        os.close(child_stdin)
        os.close(child_stdout)
        os.close(child_stderr)
        return stdin, stdout, stderr

    else:
        # Child
        try:
            os.dup2(child_stdin, 0)
            os.dup2(child_stdout, 1)
            os.dup2(child_stderr, 2)
            for i in range(3, 256):
                try:
                    os.close(i)
                except:
                    pass
            os.execv(filename, argv)
        finally:
            os._exit(127)

def go(out_fds):
    while 1:
        try:
            r, w, x = select.select(out_fds, [], [], 1)
            if not r:
                print "timed out"
        except select.error, err:
            if err[0] != errno.EINTR:
                raise
        for fd in r:
            sys.stdout.write(os.read(fd, 1024))

stdin, stderr, stdout = spawn('echo "foo"')

go([stderr, stdout])



From bob at redivi.com  Wed Aug  4 22:17:23 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed Aug  4 22:17:34 2004
Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module))
In-Reply-To: <1091649831.26267.527.camel@athlon>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<1091631453.26267.390.camel@athlon>
	<200408041520.i74FKm001469@guido.python.org>
	<1091636838.26267.406.camel@athlon>
	<Pine.GSO.4.51L2.0408042054130.18090@koeberg.lysator.liu.se>
	<1091649831.26267.527.camel@athlon>
Message-ID: <4B77CAF8-E653-11D8-B9E0-000A95686CD8@redivi.com>


On Aug 4, 2004, at 4:03 PM, Chris McDonough wrote:

> On Wed, 2004-08-04 at 15:01, Peter Astrand wrote:
>> So, there is no problem with using select() on pipes when 
>> communicating
>> with a subprocess. It works great. Take a look at (my) process.py's
>> communicate() method for some inspiration.
>
> I've actually looked at it and it's quite nice, but it doesn't do one
> thing that I'd like to see as part of a process stdlib library.  The 
> use
> case I'm thinking of is one where a long-running program needs to
> monitor the output of many other potentially long-running processes,
> doing other things in the meantime.  This kind of program tends to use
> select as a part of a mainloop where there might be other things going
> on (like handling network communications to/from sockets, updating a
> GUI, etc).  Also, the output from child stderr and stdout potentially
> never end because the child process(es) may never end.

Twisted handles this quite well already.  The stdlib doesn't really do 
much to support this style of programming.

-bob
From barry at python.org  Wed Aug  4 22:21:44 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug  4 22:21:37 2004
Subject: [Python-Dev] trunk frozen for 2.4a2
In-Reply-To: <004f01c479f3$06327380$e841fea9@oemcomputer>
References: <004f01c479f3$06327380$e841fea9@oemcomputer>
Message-ID: <1091650904.8324.299.camel@localhost>

On Wed, 2004-08-04 at 03:16, Raymond Hettinger wrote:
> [Anthony Baxter]
> > I plan to put together
> > a list of items that are still pending for 2.4 - any that can't
> > find someone to do the work will have to seriously be considered
> > at risk. . . . The only other piece of
> > major unfinished work that springs to mind is the $string
> > interpolation, 
> 
> If Barry doesn't have time for this one, I'll work on it.

I'm still planning to work on it and get it in for 2.4a3, and I know
that Raymond has offered to help.  Raymond, email me directly and we'll
arrange a time to chat about it on irc.  I have a couple of open issues
that are waiting to be resolved before this PEP lands.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040804/e60e9354/attachment.pgp
From nhodgson at bigpond.net.au  Thu Aug  5 00:04:52 2004
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Thu Aug  5 00:04:57 2004
Subject: [Python-Dev] Plea for simpler decorator syntax,
	in addition to pie-shaped syntax
References: <200408041100.34439.fdrake@acm.org>
	<04Aug4.084736pdt.58612@synergy1.parc.xerox.com>
	<16657.7235.345770.620493@montanaro.dyndns.org>
Message-ID: <016101c47a6f$11959230$034b8890@neil>

Skip Montanaro:

> Though X/Emacs is popular within the Unix community, it's hardly the only
> editor that attempts to colorize Python code:
>
>     http://www.python.org/cgi-bin/moinmoin/PythonEditors
>
> In fact, given the increased use of Python on Windows these days I'm
> skeptical it's the most widely used editor for Python code.

   Many of the other editors use the Scintilla editing component and rely on
its built-in Python lexer. If no one else gets to it first, I'll update the
lexer to recognize pie shaped decorators. Or whichever other syntax is
accepted.

   Neil

From nidoizo at yahoo.com  Thu Aug  5 03:11:28 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Thu Aug  5 03:10:26 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: <200408031450.i73Eo0e30956@guido.python.org>
References: <Your message of "Tue,
	03 Aug 2004 01:36:19 EDT." <410F2453.7000506@divmod.com>
	<Your	message of "Mon, 02 Aug 2004 13:30:03
	EDT."	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>	<1091458891.9134.29.camel@localhost>	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>	<200408030058.i730wSQ29267@guido.python.org>	<410F2453.7000506@divmod.com>
	<5.1.1.6.0.20040803101242.02a0b750@mail.telecommunity.com>
	<200408031450.i73Eo0e30956@guido.python.org>
Message-ID: <ces1dq$aig$1@sea.gmane.org>

Just a question, when "as" become a keyword in Python 3, can we expect a 
new syntax using that keyword instead (or additionally)?  There's 
multiple possible syntaxes using "as", even replacing simply @ by "as" 
would be an improvement IMHO, since that would be english-readable (as 
static method define foo), like most of Python.

Guido van Rossum wrote:
> Not IMO, that would just add more mystery (and would require yet
> another hack in classmethod to allow it to be called without args).

Regards,
Nicolas

From greg at cosc.canterbury.ac.nz  Thu Aug  5 03:16:14 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug  5 03:16:20 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C000@au3010avexu1.global.avaya.com>
Message-ID: <200408050116.i751GEW7031460@cosc353.cosc.canterbury.ac.nz>

> This proposal has come up several times before (for the record, I like
> it).

The fact that it keeps coming up suggests that it's a natural
thing for people to want to do once they've seen the use of
* in argument lists.

I can't remember seeing any particular objection -- just
a matter of inertia, I think (i.e. nice to have but not
compelling enough to spur the BDFL to accept it).

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Thu Aug  5 03:33:24 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug  5 03:33:35 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <200408041434.i74EYdU01260@guido.python.org>
Message-ID: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>

> Maybe we can accept old-style classes and instances, strings, and
> instances of Exception and its subclasses.

Seems to me the point at which we start allowing new-style classes as
exceptions should also be the point at which we drop the idea of
string exceptions. Would that help?

> I don't really think that "raise 42"
> ought to be acceptable, but I don't know how to prevent it

Maybe we need to think more deeply about *why* it shouldn't be
acceptable. If we can figure out exactly what the criterion should be,
maybe we can think of a reasonable way of testing for it.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Thu Aug  5 03:35:48 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug  5 03:35:53 2004
Subject: [Python-Dev] Plea for simpler decorator syntax,
	in addition to pie-shaped syntax
In-Reply-To: <4110F829.2000609@zope.com>
Message-ID: <200408050135.i751ZmiH031501@cosc353.cosc.canterbury.ac.nz>

> IMO, the most common uses of decorators will be to define properties,
> and class and static methods.

I thought we had decided that properties weren't one of
the things addressed by the decorator mechanism?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From python at rcn.com  Wed Aug  4 16:15:38 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Aug  5 04:17:19 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <200408041418.i74EIt001130@guido.python.org>
Message-ID: <000e01c47a2d$84f3be60$e841fea9@oemcomputer>

> For the record, let me just say that I'm -1 on adding this feature
> now.  It's only a minor convenience while it's a fairly big change to
> the parser and code generator ...

That is a strong reason to deny the feature request.



> ... but most of all, Python's growth needs
> to be reduced.  If we were to add every "nice" feature that was
> invented (or present in other languages), Python would lose one of its
> main charms, which is that it is really a rather simple language that
> can be learned and put to production quickly.

However, IMO this reason does not apply to the *arg feature request.
Whenever a concept is already in a language, applying the same idea in
other places does not make the language fatter or harder to learn.

For instance, since generator expressions grow naturally from list
comprehensions and regular generators, they are not likely to present a
learning challenge.



> ... I appreciate the idea (which BTW has been proposed before)

IIRC, the conversation for the original proposal fizzled out without
reaching a conclusion.  So, resuming the discussion was probably a good
idea.  But now it has been officially zapped.



Raymond

From djc at object-craft.com.au  Thu Aug  5 04:18:42 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Thu Aug  5 04:18:52 2004
Subject: [Python-Dev] Submitted socket.socketpair() patch to SF
Message-ID: <41119902.8060305@object-craft.com.au>

For the interested, I have just submitted patch #1003700 which 
implements the socketpair function in the socket module.

The patch is relative to 2.4a1.

There is a question in the patch summary:

Outstanding question is whether the family, type, proto arguments should 
be kwargs as per socket() function, or positional as per fromfd() 
function. If kwargs, then it is arguable that the fromfd() arguments 
should be changed (in another patch).

I noticed a couple of things in the documentation while implementing the 
patch:

1) The documentation for socket() does not make mention of the fact that 
all arguments are kwargs.

2) The documentation for fromfd() does not mention that it does a dup() 
on the supplied fd to create a socket object.

- Dave

-- 
http://www.object-craft.com.au
From bac at OCF.Berkeley.EDU  Thu Aug  5 04:19:17 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug  5 04:19:15 2004
Subject: [Python-Dev] No handlers could be found for logger "cookielib"
In-Reply-To: <1f7befae0408041203475de3d9@mail.gmail.com>
References: <1f7befae0408041203475de3d9@mail.gmail.com>
Message-ID: <41119925.1060001@ocf.berkeley.edu>

Tim Peters wrote:

> I first saw this last night, when running the -all tests on WinXP, CVS HEAD:
> 
> ...
> test_univnewlines
> test_unpack
> test_urllib
> test_urllib2
> No handlers could be found for logger "cookielib"   HERE
> test_urllibnet
> ...
> 
> It didn't act the same way in isolation.  Anyone have a clue?  The
> test still passes, it just spits out that unhelpful logger output --
> sometimes.

See the thread at 
http://mail.python.org/pipermail/python-dev/2004-July/046396.html for an 
explanation of the message.  Basically means no default handler is 
specified.

-Brett
From paul at prescod.net  Thu Aug  5 04:32:30 2004
From: paul at prescod.net (Paul Prescod)
Date: Thu Aug  5 04:37:46 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
Message-ID: <41119C3E.5060509@prescod.net>

Greg Ewing wrote:

>...
>>I don't really think that "raise 42"
>>ought to be acceptable, but I don't know how to prevent it
> 
> 
> Maybe we need to think more deeply about *why* it shouldn't be
> acceptable. If we can figure out exactly what the criterion should be,
> maybe we can think of a reasonable way of testing for it.

Exceptions naturally form a hierarchy. At the same time, inheritance of 
behaviour among exceptions is seldom necessary. Therefore, exceptions 
inherit from each other in order to build a classification system, not 
to share code. This is the opposite of the traditional reasons for 
classes inheriting from other classes in Python. This is why it seems 
"unpythonic" to require exceptions to be single-rooted.

But having a proper classification system is exactly what is required to 
allow robust, modular code that catches the right exceptions under the 
right circumstances and responds in the right way. So it is pythonic 
after all.

In a few senses the _current model_ is unpythonic. There is no catch-all 
root so you have to use a "bare" except to catch every exception type. 
This makes it hard to introspect on the caught object. But 
introspection is the MOST IMPORTANT THING when you are catching all 
exceptions (because you should be logging the exception or something).

  Paul Prescod

From tim.peters at gmail.com  Thu Aug  5 04:40:24 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug  5 04:40:30 2004
Subject: [Python-Dev] No handlers could be found for logger "cookielib"
In-Reply-To: <41119925.1060001@ocf.berkeley.edu>
References: <1f7befae0408041203475de3d9@mail.gmail.com>
	<41119925.1060001@ocf.berkeley.edu>
Message-ID: <1f7befae040804194050d68f20@mail.gmail.com>

[Tim]
>> I first saw this last night, when running the -all tests on WinXP, CVS HEAD:
>>
>> ...
>> test_univnewlines
>> test_unpack
>> test_urllib
>> test_urllib2
>> No handlers could be found for logger "cookielib"   HERE
>> test_urllibnet
>> ...
>>
>> It didn't act the same way in isolation.  Anyone have a clue?  The
>> test still passes, it just spits out that unhelpful logger output --
>> sometimes.

[Brett]
> See the thread at
> http://mail.python.org/pipermail/python-dev/2004-July/046396.html for an
> explanation of the message.  Basically means no default handler is
> specified.

Ya, I know about that.  The real questions are two others:

1. Why does it only appear *sometimes*, depending on how the test suite
    is run?  It always appears when running -uall now.  It never appears
    when running the surrounding tests in isolation.

2. Since it started happening very recently (I've run the full test suite
    often the last week), does it ring a bell with anyone who checked something
    in very recently?  Somebody changed something relevant very recently.
From ncoghlan at iinet.net.au  Thu Aug  5 05:33:02 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Aug  5 05:33:23 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <200408050116.i751GEW7031460@cosc353.cosc.canterbury.ac.nz>
References: <200408050116.i751GEW7031460@cosc353.cosc.canterbury.ac.nz>
Message-ID: <4111AA6E.3090600@iinet.net.au>

Greg Ewing wrote:
> I can't remember seeing any particular objection -- just
> a matter of inertia, I think (i.e. nice to have but not
> compelling enough to spur the BDFL to accept it).

This certainly summarises my reason for criticising it. It's application 
to a basic 'command parsing' scenario was a useful case I hadn't thought 
of, though:

 >>> commands = line.split() for line in commandFile.readlines()
 >>> for command, *args in commands do: func_table[command](*args)

However, the slicing based alternative is fairly readable, too and if 
the BDFL says the interpreter changes needed to support the feature are 
too drastic, I'm willing to believe him :)

Regards,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From andrewm at object-craft.com.au  Thu Aug  5 05:36:33 2004
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Thu Aug  5 05:36:23 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators 
In-Reply-To: <4110F35D.8020904@zope.com> 
References: <Your message of "Mon, 02 Aug 2004 13:30:03
	EDT."<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com><1091458891.9134.29.camel@localhost><5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com><5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com><5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com><200408030058.i730wSQ29267@guido.python.org><410F2453.7000506@divmod.com><200408031404.i73E4cS30629@guido.python.org><410FB7F4.2070309@divmod.com><200408031700.i73H0MY31391@guido.python.org>
	<410FC9FE.6060305@divmod.com> <cepadg$dp2$1@sea.gmane.org>
	<4110E8CF.6070900@zope.com>
	<20040804140442.89F2B3C7A0@coffee.object-craft.com.au>
	<4110F35D.8020904@zope.com>
Message-ID: <20040805033633.2D1BA3C221@coffee.object-craft.com.au>

>> There are a number of "advanced" features in python that could be a
>> barrier to understanding for new users, but which are rarely seen in
>> practice - metaclasses being the classic example.
>> 
>> I imagine these features are rarely seen because most people don't
>> understand them, and those that do have the good taste not to use them
>> gratuitously.
>
>This feature won't be rarely used.  Class and static methods
>and properties are created fairly often, certainly far more often
>than meta classes.

Sorry, what I meant was advanced uses of decorators will be rare.
Certainly "staticmethod" and "classmethod" will be common. It's a personal
opinion, but I find @staticmethod easier to parse visually than the
current situation.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/
From guido at python.org  Thu Aug  5 06:17:53 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug  5 06:18:33 2004
Subject: [Python-Dev] Re: PEP 324 (process module)
In-Reply-To: Your message of "Wed, 04 Aug 2004 12:46:56 EDT."
	<1091638016.26267.418.camel@athlon> 
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net>
	<200408031427.i73ERn330790@guido.python.org>
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se>
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<1091631453.26267.390.camel@athlon>
	<Pine.GSO.4.51L2.0408041822280.17161@koeberg.lysator.liu.se> 
	<1091638016.26267.418.camel@athlon> 
Message-ID: <200408050417.i754Hr602375@guido.python.org>

> As I've found, if the end of the pipes in that represent the child's
> stderr/stdout fds are closed in the parent, a select() reading the
> "other" ends of these pipes will busywaut (at least on Linux).  Other
> than than, I think this  select() would be the absolute right thing on
> POSIX rather than using threads or polling.  Is there a way around this
> problem or is it just a fact of life?

The reading code needs to recognize the EOF and then conclude that
it's not going to get anything else from that pipe.  No different than
sockets.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug  5 06:36:04 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug  5 06:36:24 2004
Subject: [Python-Dev] Re: 2.4a2, and @decorators
In-Reply-To: Your message of "Wed, 04 Aug 2004 21:11:28 EDT."
	<ces1dq$aig$1@sea.gmane.org> 
References: <Your message of "Tue,
	03 Aug 2004 01:36:19 EDT." <410F2453.7000506@divmod.com> <Your
	message of "Mon, 02 Aug 2004 13:30:03 EDT."
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802104000.035e48a0@mail.telecommunity.com>
	<1091458891.9134.29.camel@localhost>
	<5.1.1.6.0.20040802125225.05c5b1d0@mail.telecommunity.com>
	<5.1.1.6.0.20040802132417.053f5b70@mail.telecommunity.com>
	<5.1.1.6.0.20040802154405.03676b70@mail.telecommunity.com>
	<200408030058.i730wSQ29267@guido.python.org>
	<410F2453.7000506@divmod.com>
	<5.1.1.6.0.20040803101242.02a0b750@mail.telecommunity.com>
	<200408031450.i73Eo0e30956@guido.python.org> 
	<ces1dq$aig$1@sea.gmane.org> 
Message-ID: <200408050436.i754a4k02477@guido.python.org>

> Just a question, when "as" become a keyword in Python 3, can we expect a 
> new syntax using that keyword instead (or additionally)?  There's 
> multiple possible syntaxes using "as", even replacing simply @ by "as" 
> would be an improvement IMHO, since that would be english-readable (as 
> static method define foo), like most of Python.

When 'as' becomes a keyword, it is open for use in other syntax, but
it won't be used for decorators (as long as I'm BDFL).  'as' suggests
a renaming (like it does in import and in SQL) or perhaps a type (I've
seen "as duck" in a new vaguely Pythonic language named "boo"),
but that's not what decorators doo.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at cosc.canterbury.ac.nz  Thu Aug  5 07:50:37 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug  5 07:50:43 2004
Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module))
In-Reply-To: <1091650550.26267.538.camel@athlon>
Message-ID: <200408050550.i755ob1s031939@cosc353.cosc.canterbury.ac.nz>

Chris McDonough <chrism@plope.com>:

> The following program demonstrates that a particular usage of select
> (under Linux at least) always returns the output side of a pipe
> connected to a child process' stdout as "ready" after it gets any output
> from that child process, even if the child process has no further data
> to provide after it has provided a bit of data to the parent.
>
> Or am I doing something silly?
> 
>         for fd in r:
>             sys.stdout.write(os.read(fd, 1024))

You're not taking account of the possibility of EOF. When
the child process finishes and closes its end of the pipe,
the select will always report the pipe as ready for reading,
because it is -- subsequent reads will return immediately
with 0 bytes.

You need to check whether the read returned 0 bytes, and
take that as meaning that the child process has finished.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From bac at OCF.Berkeley.EDU  Thu Aug  5 09:05:16 2004
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Thu Aug  5 09:05:11 2004
Subject: [Python-Dev] python-dev Summary for 2004-07-16 through 2004-07-31
	[draft]
Message-ID: <4111DC2C.8070302@ocf.berkeley.edu>

OK, so I procrastinated from doing more work on my thesis and did the 
next summary.  Plan to send this out some time over the weekend.

-------------------------------


=====================
Summary Announcements
=====================
Python 2.4a2 should be coming out any day now, so keep an eye out for it 
and test the heck out of it.


========
Summary
========
--------------------------------------------------------------------------------------------------
Makin' bdist_wininst purty and nicer for them non-English speaking folks
--------------------------------------------------------------------------------------------------
Thomas Heller asked if some better images for Distutil's bdist_wininst 
could be used.  Some existing images were discussed, but no specific 
resolution was mentioned in the thread.

Walter D?rwald brought up the point that if this was all being worked on 
he wouldn't mind having Unicode support for things such as author name 
and such.  Thomas and Walter worked on it and that has gone in.

Contributing threads:
   - `bdist_wininst 
<http://mail.python.org/pipermail/python-dev/2004-July/046268.html>`__


-------------------------------------------------------------------------
Assigning to None a no-no, but still okay for booleans
-------------------------------------------------------------------------
Assigning to None is now a SyntaxError instead of a SyntaxWarning. 
Assignment to True and False, though, is still allowed and probably will 
not be restricted until Python 3.0 (which will probably have to wait 
until, if Guido's OSCON slides are to be believed, until he retires). 
The idea of restricting assignment, though, was brought up by Raymond 
Hettinger.  Michael Hudson, though, pointed out it would be difficult 
and possibly require a special opcode to handle it.

Contributing threads:
   - `Re: [Python-checkins] python/dist/src/Python compile.c, 
<http://mail.python.org/pipermail/python-dev/2004-July/046271.html>`__
   - `None as constant. Still SyntaxWarning 
<http://mail.python.org/pipermail/python-dev/2004-July/046323.html>`__


-----------------------------------------------------------------
'as' does not get to be treated with kiddie gloves
-----------------------------------------------------------------
Philip Eby asked if 'as' was going to become a keyword.  Guido said not 
now, but in Python 3.0 it will be.

Contributing threads:
   - `"as" to be a keyword? 
<http://mail.python.org/pipermail/python-dev/2004-July/046304.html>`__


-------------------------------------------------------------
Hopefully you didn't love FCNTL like a child...
--------------------------------------------------------------
... since if you did that kid was a lazy punk that did nothing but cause 
trouble for Windows users.  That's why it got kicked out of the house to 
never come back.

Contributing threads:
   - `Kill FCNTL.py 
<http://mail.python.org/pipermail/python-dev/2004-July/046322.html>`__


------------------------------------------------------------------------------------------------------------------------------------------
How to get Python to compile with Microsoft's free compiler that should 
just come with the OS standard
------------------------------------------------------------------------------------------------------------------------------------------
1. Download the free .NET compiler
2. Download the Windows SDK (at 
http://www.microsoft.com/msdownload/platformsdk/sdkupdate/ with only IE)
3. Download Garth's tools to generate a Makefile from the .sln files (at 
http://mail.python.org/pipermail/python-dev/2004-February/042595.html )
4. Compile
5. Realize you should be using an OS that doesn't make you go through 
this many hoops just to have a compiler for your platform

Contributing threads:
   - `Non-Visual Studio builds on Windows XP? 
<http://mail.python.org/pipermail/python-dev/2004-July/046324.html>`__


---------------------------------------------------------------------------------------------------------------------------------
Tim Peters, whitespace Czar, will break your legs if you mess up the 
pristine whitespace in CVS
---------------------------------------------------------------------------------------------------------------------------------
Tim Peters ran Tools/Scripts/reindent.py over the entire CVS tree and 
fixed it up.  This means you had better not mess up and check in using 
anything but 4-space indents with no tabs!

Contributing threads:
   - `Fun with whitespace 
<http://mail.python.org/pipermail/python-dev/2004-July/046331.html>`__


-----------------------------------------------------------
Thread safety == won't blow up in your face
-----------------------------------------------------------
Paul Moore pointed out that the documentation for deques in the 
collections module stated they were thread-safe.  It was clarified that 
the statement meant that internal state would not be corrupted if 
multiple threads accessed the same object; no guarantee that it isn't 
accessed in some order or anything.  Basically the idea of thread-safety 
for C code is that it won't lead to the interpreter exploding, nothing 
more.  And you don't even get that guarantee with Python code.

Contributing threads:
   - `Thread safety of deques 
<http://mail.python.org/pipermail/python-dev/2004-July/046343.html>`__


------------------------------------------------------------------------
LC_NUMERIC PEP gets a pristine, new PEP number
------------------------------------------------------------------------
Even though the code had already been put into the core, the PEP about 
LC_NUMERIC and being locale-independent never got a PEP number.  Well, 
now it has one: `PEP 331`_.

.. _PEP 331: http://www.python.org/peps/pep-0331.html

Contributing threads:
   - `PEP 331: Locale-Independent Float/String Conversions 
<http://mail.python.org/pipermail/python-dev/2004-July/046358.html>`__


-----------------------------------------------------------------------
Edward Loper becomes one of us rowdy developers
-----------------------------------------------------------------------
And he has already made his initial contribution by helping to rework 
doctest.

Contributing threads:
   - `Would like to add Edward Loper as a Python developer 
<http://mail.python.org/pipermail/python-dev/2004-July/046395.html>`__


------------------------------------------------------------------------------------------------------
Any misbehaving modules during initialization from import now get the boot
------------------------------------------------------------------------------------------------------
Jim Fulton wanted a better way to detect when an import failed thanks to 
another module being directly imported (e.g., module A imports B which 
raised TypeError from some initialization code from importing; importing 
would leave A and B in a shoddy state in sys.modules along with raising 
TypeError in the import from A instead of raising ImportError).  While 
the latter still occurs, modules are not left in sys.modules in a broken 
state from exceptions being raised during initialization thanks to Guido 
and Tim Peters.

There was a discussion on circular imports and how to handle those for 
proper rollback.  Some suggestions were taking a snapshot of sys.modules 
and then restoring with that if something bad happens to putting in 
placeholder modules in sys.modules so as to not try to extra imports on 
modules in the process and to help track state.  But this all gets 
sticky from side-effects that modules can do outside of themselves 
before they finish importing everyone else.  If, for instance, module A, 
as a side-effect of importation, injected a custom version of len into 
module B.  Now, if module A did this before doing all of its imports it 
could pull of the len injection but still fail from a bad import.  That 
is not good.  Basically the best solution is to not do that; there is a 
reason you should do all of your global imports as the first thing in a 
module.

Contributing threads:
   - `Fix import errors to have data 
<http://mail.python.org/pipermail/python-dev/2004-July/046424.html>`__


----------------------------------------------------------------------------------------------------------------------------
Stuff about Unicode, state, and how to handle when a stream terminated 
early; all CJK to me
----------------------------------------------------------------------------------------------------------------------------
Walter D?rwald noticed that codecs.StreamReader.read() would read a few 
more bytes when it discovered an error.  That's bad since there might 
not be more bytes and continuing once an error has been found is just 
not right.  So he wanted to fix that problem.

Unfortunately he and MA Lemburg started to talk and I just couldn't 
follow everything about stateful and stateless decoders/encoders and the 
issues; I'm American so Unicode just doesn't fit in my brain well.  So 
if you want to know what conclusions they reached you are going to read 
the thread on your own.

Contributing threads:
   - `Decoding incomplete unicode 
<http://mail.python.org/pipermail/python-dev/2004-July/046450.html>`__


----------------------------------------------------------------------
Use the docs to know what the public API is, people
----------------------------------------------------------------------
Fernando Perez got bit by having something removed from the 'keyword' 
module that was not part of the documented API.  Turns out that running 
'help' on the module listed the function in question and so he just went 
ahead an used it.  That's not the right way to go about finding the 
public API of stdlib code.

Always follow the documentation.  If it is not listed there don't expect 
it to be there in the next version.  While we try not to yank stuff out 
needlessly and will tweak things to be nice on occasion, we make no 
guarantees on stuff not listed in the API.

The next best thing is what is listed in __all__ for a module.  Since 
that is explicitly listed that usually can be considered part of the API.

And lastly, anything starting with an underscore is off limits in terms 
of promises of keeping it around.

Luckily Skip Montanaro applied a patch for pydoc to have it only list 
stuff in modules as specified by __all__, so it's a little safer out 
there.  But the actual documentation is still king.

Contributing threads:
   - `Rationale behind removing kwdict from keyword.py? 
<http://mail.python.org/pipermail/python-dev/2004-July/046465.html>`__


-------------------------------------------------------
Linking against static builds on Windows
-------------------------------------------------------
Thomas Heller pointed out that one couldn't dynamically link against a 
statically built Python interpreter DLL.  He and Martin v. L?wis 
discussed various ways to try to get this to work, but it doesn't appear 
they came up with one.

Contributing threads:
   - `Static builds on Windows 
<http://mail.python.org/pipermail/python-dev/2004-July/046499.html>`__
From mal at egenix.com  Thu Aug  5 09:45:28 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug  5 09:45:32 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <41119C3E.5060509@prescod.net>
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
	<41119C3E.5060509@prescod.net>
Message-ID: <4111E598.5020302@egenix.com>

Paul Prescod wrote:
> Greg Ewing wrote:
> 
>> ...
>>
>>> I don't really think that "raise 42"
>>> ought to be acceptable, but I don't know how to prevent it
>>
>>
>>
>> Maybe we need to think more deeply about *why* it shouldn't be
>> acceptable. If we can figure out exactly what the criterion should be,
>> maybe we can think of a reasonable way of testing for it.
> 
> 
> Exceptions naturally form a hierarchy. At the same time, inheritance of 
> behaviour among exceptions is seldom necessary. Therefore, exceptions 
> inherit from each other in order to build a classification system, not 
> to share code. 

I wouldn't say that: exceptions can have error handlers, callbacks,
inherited attributes, etc. etc. and you can put these to good
use in your application.

> This is the opposite of the traditional reasons for 
> classes inheriting from other classes in Python. This is why it seems 
> "unpythonic" to require exceptions to be single-rooted.

I don't know what should be "unpythonic" about having a single
root for exceptions. Would someone care to explain ?

To me ...

try:
     ...
except Exception, errobj:
     # catches all exceptions
     pass

... is the most natural way of using that single root (and it already
works great today).

> But having a proper classification system is exactly what is required to 
> allow robust, modular code that catches the right exceptions under the 
> right circumstances and responds in the right way. So it is pythonic 
> after all.
 >
> In a few senses the _current model_ is unpythonic. There is no catch-all 
> root so you have to use a "bare" except to catch every exception type. 
> This makes it hard to introspect on the caught object. But introspection 
> is the MOST IMPORTANT THING when you are catching all exceptions 
> (because you should be logging the exception or something).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 05 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From martin at v.loewis.de  Thu Aug  5 09:21:37 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 10:36:52 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <20040803193516.GA9031@titan.progiciels-bpi.ca>
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de>
	<20040803193516.GA9031@titan.progiciels-bpi.ca>
Message-ID: <4111E001.40601@v.loewis.de>

Fran?ois Pinard wrote:
> However, and I shall have the honesty to
> state it, this is *not* respectful of the general Unicode spirit: the
> Python implementation allows for independently addressable surrogate
> halves

This is only a problem if you have data which require surrogates (which
I claim are rather uncommon at the moment), and you don't have a UCS-4
build of Python (in which surrogates don't exist). As more users demand
convenient support for non-BMP characters, you'll find that more builds
of Python become UCS-4. In fact, you might find that the build you are
using already has sys.maxunicode > 65535.

> combining zero-width diacritics

Indeed. However, it is not clear to me how this problem could be
addressed, and I'm not aware of any API (any language) that addresses
it.

Typically, people need things like this:
- in a fixed-width terminal, what characters occupy what column.
   Notice that this involves East-Asian wide characters, where a single
   Unicode character (a "wide" character) occupies two columns. OTOH,
   with combining characters, a sequence of characters might be
   associated with a single column. Furthermore, some code points might
   not be associated with a column at all.
- for a given font, how many points does a string occupy, horizontally
   and vertically.
- where is the next word break

I don't know what your application is, but I somewhat doubt it is as
simple as "give me a thing describing the nth character, including
combining diacritics".

However, it is certainly possible to implement libraries on top of the
existing code, and if there is a real need for that, somebody will
contribute it.

 > normal _and_ decomposed forms,

Terminology alert: the are multiple normal forms in Unicode, and some
of them are decomposed (e.g. NFD, NFKD).

I fail to see a problem with that. There are applications for all
normal forms, and many applications don't need the overhead of
normalization. It might be that the code for your languages becomes
simpler when always assuming NFC, but this hardly holds for all
languages, or all applications.

> directional marks, linguistic marks and various other such complexities.

Same comment as above: if this becomes a real problem, people will
contribute code to deal with it.

> But in our case, where applications already work in Latin-1, abusing our
> Unicode luck, UTF-8 may _not_ be used as is, we ought to use Unicode or
> wide strings as well, for preserving S[N] addressability.  So changing
> source encodings may be intimately tied to going Unicode whenever UTF-8
> (or any other variable-length encoding) gets into the picture.

Yes. There is not much Python can do about this. UTF-8 is very nice for
transfer of character data, but it does have most of the problems of
a multi-byte encoding. I still prefer it over UTF-16 or UTF-32 for
transfer, though.

> I hope that my explanation above helps at seeing that source encoding
> and choice of string literals are not as independent as one may think.

It really depends on your processing needs. But yes, my advise still
stands: convert to Unicode objects as early as possible in the
processing. For source code involving non-ASCII characters, this means
you really should use Unicode literals.

Of course, my other advise also applies: if you have a program that
deals with multiple languages, use only ASCII in the source, and use
gettext for the messages.

> There ought to be a way to maintain a single Python source that would
> work dependably through re-encoding of the source, but not uselessly
> relying on wide strings when there is no need for them.  That is,
> without marking all literal strings as being Unicode.  Changing encoding
> from ISO 8859-1 to UTF-8 should not be a one-way, no-return ticket.

But it is not: as you say, you have to add u prefixes when going to 
UTF-8, yes. But then you can go back to Latin-1, with *no* change other
than recoding, and changing the encoding declaration. The string 
literals can all stay as Unicode literals - the conversion to Latin-1
then really has *no* effect on the runtime semantics.

> Of course, it is very normal that sources may have to be adapted for the
> possibility of a Unicode context.  There should be some good style and
> habits for writing re-encodable programs.  So this exchange of thoughts.

If that is the goal, you really need Unicode literals - everything else
*will* break under re-encoding.

Regards,
Martin
From mal at egenix.com  Thu Aug  5 11:00:22 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug  5 11:00:26 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <20040803193516.GA9031@titan.progiciels-bpi.ca>
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>	<410FCA3B.1020002@v.loewis.de>
	<20040803193516.GA9031@titan.progiciels-bpi.ca>
Message-ID: <4111F726.8050905@egenix.com>

[Source code encoding and string literals]

> I hope that my explanation above helps at seeing that source encoding
> and choice of string literals are not as independent as one may think.
> A choice that I surely do _not_ have is to see bugs appear in programs
> merely because I changed the source encoding.  Going from ISO 8859-1 to
> ISO 8859-15 for a Python source is probably fairly safe, because there
> is no need for switching the narrowness of strings.  Going from ISO
> 8859-1 to UTF-8 is very unsafe, and editing all literal strings from
> narrow to wide, using `u' prefixes, becomes almost unavoidable.

Indeed. As always: explicit is better than implicit :-) The small
"u" in front of the literal will tell all readers: this is Unicode
text.

We might introduce more ways to switch string literal interpretation
depending on module or interpreter process scope. However, the small
u is here to stay and it's available now, so why not use it ?

Your concerns about programs breaking because of changes to the
source encoding are valid, but not something that Python can address.
You have the same problem with normal text documents: a spell
checker might find wrong spellings of a word as a result of using
a wrong encoding, but it is not fool proof and things get worse
if you have multiple languages embedded in your program code.

As general advice for writing i18n compliant programs, I can
only suggest to keep programs written using a single source
encoding and language that appeals to the programmer and place
*all* string literals under gettext or similar tool control.

I usually write programs in ASCII or Latin-1 and use English
to write the string literals which then get mapped to user
languages as necessary by means of gettext or custom translation
logic.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 05 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From Paul.Moore at atosorigin.com  Thu Aug  5 11:18:25 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Thu Aug  5 11:18:25 2004
Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module))
Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060F47@UKDCX001.uk.int.atosorigin.com>

From: Chris McDonough
> I've actually looked at it and it's quite nice, but it doesn't
> do one thing that I'd like to see as part of a process stdlib
> library.  The use case I'm thinking of is one where a long-running
> program needs to monitor the output of many other potentially
> long-running processes

I have a couple of other use cases, as well.

1. Manage an interactive subprocess, sending it input and reading
   output in "chunks". This is an expect-like activity in some
   ways: I use it to wrap an alternative UI round an interactive
   command line process. I suspect Trent's ProcessProxy would
   help with this - must check.
2. Run a command with no input, and get its output back, like the
   shell backtick operator. Errors in the command should appear
   as exceptions.
3. Filter a string through an external command - the interact()
   method handles this, but again I'd like errors to appear as
   exceptions.

To some extent, (2) and (3) are simple "helper" functions round the
existing methods, but I believe such helpers would be useful. And
the "convert subcommand error to exception" code is non-trivial,
and hence useful to encapsulate in the library.

This is getting pretty off-topic for python-dev. Is there any value
in setting up a list/SIG for discussing the design of a stdlib
process module?

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________
From vinay_sajip at yahoo.co.uk  Thu Aug  5 11:44:37 2004
From: vinay_sajip at yahoo.co.uk (=?iso-8859-1?q?Vinay=20Sajip?=)
Date: Thu Aug  5 11:44:39 2004
Subject: [Python-Dev] No handlers could be found for logger "cookielib"
Message-ID: <20040805094437.32296.qmail@web25403.mail.ukl.yahoo.com>

I just made a change to test_logging to clean up after
itself, so that it removes all handlers that it
creates. test_urllib2 seems to have handler creation
commented out  in NetworkTests.setUp(). A combination
of the two may explain why the message shows up now.

Vinay Sajip


	
	
		
___________________________________________________________ALL-NEW Yahoo! Messenger - all new features - even more fun!  http://uk.messenger.yahoo.com
From mwh at python.net  Thu Aug  5 13:15:18 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug  5 13:15:19 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <200408032014.i73KEwp31876@guido.python.org> (Guido van
	Rossum's message of "Tue, 03 Aug 2004 13:14:58 -0700")
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com>
	<20040803194724.GB9031@titan.progiciels-bpi.ca>
	<200408032014.i73KEwp31876@guido.python.org>
Message-ID: <2mllgt96pl.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> (It will also break code that creates a class used as an exception
> that doesn't derive from Exception, but those should be shot. :-)

Would you like to guess how often that happens in the Python test
suite? :-)

Cheers,
mwh

-- 
  ARTHUR:  Yes.  It was on display in the bottom of a locked filing
           cabinet stuck in a disused lavatory with a sign on the door
           saying "Beware of the Leopard".
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 1
From mwh at python.net  Thu Aug  5 13:16:38 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug  5 13:16:40 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz> (Greg
	Ewing's message of "Thu, 05 Aug 2004 13:33:24 +1200")
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
Message-ID: <2mhdrh96nd.fsf@starship.python.net>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

>> Maybe we can accept old-style classes and instances, strings, and
>> instances of Exception and its subclasses.
>
> Seems to me the point at which we start allowing new-style classes as
> exceptions should also be the point at which we drop the idea of
> string exceptions. Would that help?

It would probably make things a little simpler, but probably not in a
major way.

Cheers,
mwh

-- 
  Or if you happen to be resigned to the size of your trouser
  snake and would rather not be reminded of it, training a shared
  classifier to reject penis-enlargement spam stops Barry from
  getting the help he so desperately needs.  -- Tim Peters, c.l.python
From mwh at python.net  Thu Aug  5 13:18:18 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug  5 13:18:19 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <4111E598.5020302@egenix.com> (M.'s message of "Thu, 05 Aug
	2004 09:45:28 +0200")
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
	<41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com>
Message-ID: <2md62596kl.fsf@starship.python.net>

"M.-A. Lemburg" <mal@egenix.com> writes:

> To me ...
>
> try:
>      ...
> except Exception, errobj:
>      # catches all exceptions
>      pass
>
> ... is the most natural way of using that single root (and it already
> works great today).

Well, uh, it's not totally bullet proof:

>>> class C: pass
... 
[24618 refs]
>>> try: raise C
... except Exception, err: print err
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
__main__.C: <__main__.C instance at 0x403a7814>
[24654 refs]
>>> 

but I this really doesn't seem to happen in the wild.

(I have a hacky patch which makes exceptions new-style which I'll post
in a moment).

Cheers,
mwh

-- 
  Lisp nearing the age of 50 is the most modern language out
  there. GC, dynamic, reflective, the best OO model extant including
  GFs, procedural macros, and the only thing old-fashioned about it 
  is that it is compiled and fast.   -- Kenny Tilton, comp.lang.python
From pyth at devel.trillke.net  Thu Aug  5 13:29:28 2004
From: pyth at devel.trillke.net (Holger Krekel)
Date: Thu Aug  5 13:29:31 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <200408032014.i73KEwp31876@guido.python.org>
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com>
	<20040803194724.GB9031@titan.progiciels-bpi.ca>
	<200408032014.i73KEwp31876@guido.python.org>
Message-ID: <20040805112928.GA5208@solar.trillke>

Guido van Rossum wrote:
> (It will also break code that creates a class used as an exception
> that doesn't derive from Exception, but those should be shot. :-)

Then i guess that searching down into a recursive structure and just 
raising an "i found it" result object up doesn't count as a use case in
your book, right?   It can avoid boilerplate code like return-if-not-None 
checks and I have used it for e.g. finding patterns in an AST-Tree. 

cheers, 

    Holger
From mal at egenix.com  Thu Aug  5 13:48:23 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug  5 13:48:26 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <2md62596kl.fsf@starship.python.net>
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>	<41119C3E.5060509@prescod.net>
	<4111E598.5020302@egenix.com> <2md62596kl.fsf@starship.python.net>
Message-ID: <41121E87.9010805@egenix.com>

Michael Hudson wrote:
> "M.-A. Lemburg" <mal@egenix.com> writes:
> 
> 
>>To me ...
>>
>>try:
>>     ...
>>except Exception, errobj:
>>     # catches all exceptions
>>     pass
>>
>>... is the most natural way of using that single root (and it already
>>works great today).
> 
> 
> Well, uh, it's not totally bullet proof:

I meant that it works for the vast majority of all cases
you see in practice. I haven't seen a non-Exception based
exception in years.

>>>>class C: pass
> 
> ... 
> [24618 refs]
> 
>>>>try: raise C
> 
> ... except Exception, err: print err
> ... 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> __main__.C: <__main__.C instance at 0x403a7814>
> [24654 refs]
> 
> 
> but I this really doesn't seem to happen in the wild.
> 
> (I have a hacky patch which makes exceptions new-style which I'll post
> in a moment).
> 
> Cheers,
> mwh
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 05 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mwh at python.net  Thu Aug  5 14:05:44 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug  5 14:05:46 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <41121E87.9010805@egenix.com> (M.'s message of "Thu, 05 Aug
	2004 13:48:23 +0200")
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
	<41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com>
	<2md62596kl.fsf@starship.python.net> <41121E87.9010805@egenix.com>
Message-ID: <2m8yct94dj.fsf@starship.python.net>

"M.-A. Lemburg" <mal@egenix.com> writes:

D> Michael Hudson wrote:
>> "M.-A. Lemburg" <mal@egenix.com> writes:
>> 
>>>To me ...
>>>
>>>try:
>>>     ...
>>>except Exception, errobj:
>>>     # catches all exceptions
>>>     pass
>>>
>>>... is the most natural way of using that single root (and it already
>>>works great today).
>> Well, uh, it's not totally bullet proof:
>
> I meant that it works for the vast majority of all cases
> you see in practice.

OK, then we're on the same page.

> I haven't seen a non-Exception based exception in years.

I hadn't until I looked into test_opcodes last night!

Cheers,
mwh

-- 
  In that case I suggest that to get the correct image you look at
  the screen from inside the monitor whilst standing on your head.  
               -- James Bonfield, http://www.ioccc.org/2000/rince.hint
From mcherm at mcherm.com  Thu Aug  5 15:04:09 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Aug  5 15:03:13 2004
Subject: [Python-Dev] 2.4a2, and @decorators
Message-ID: <1091711049.41123049900cb@mcherm.com>

Michael Hudson writes:
> > that would be insane,
>
> class C(random.choice([dict, list])):
>     pass
>
> is my favourite example of this :-)

Where is THAT monstrocity used!?

-- Michael Chermside

From anthony at interlink.com.au  Thu Aug  5 15:10:32 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Aug  5 15:10:58 2004
Subject: [Python-Dev] RELEASED Python 2.4, alpha 2
Message-ID: <411231C8.3020308@interlink.com.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team and the Python community, I'm
happy to announce the second alpha of Python 2.4.

Python 2.4a2 is an alpha release. We'd greatly appreciate it if you
could download it, kick the tires and let us know of any problems you
find, but it is not suitable for production usage.

~    http://www.python.org/2.4

In this release we have new syntax for function decorators, a fix for
failing imports so that they don't leave a broken module in sys.modules,
a host of updated modules in the standard library (including optparse
and doctest) and a large number of other bug fixes and improvements.
See either the highlights, the What's New in Python 2.4, or the detailed
NEWS file -- all available from the Python 2.4 webpage.

There will probably be one more alpha in a few weeks before the first
beta a few weeks after that.

Please log any problems you have with this release in the SourceForge
bug tracker (noting that you're using 2.4a2):

~    http://sourceforge.net/bugs/?group_id=5470

Enjoy the new release,
Anthony

Anthony Baxter
anthony@python.org
Python Release Manager
(on behalf of the entire python-dev team)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBEjHFDt3F8mpFyBYRAhgXAKCful63a7kAUcHnxFzCZSzq0bmZ7QCfZIqy
t/PqaaLcdRL6IVUKPAWiytA=
=pQNU
-----END PGP SIGNATURE-----
From mwh at python.net  Thu Aug  5 15:27:57 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug  5 15:27:58 2004
Subject: [Python-Dev] New-style exceptions
In-Reply-To: <2md62596kl.fsf@starship.python.net> (Michael Hudson's message
	of "Thu, 05 Aug 2004 12:18:18 +0100")
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
	<41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com>
	<2md62596kl.fsf@starship.python.net>
Message-ID: <2mzn597m02.fsf_-_@starship.python.net>

Michael Hudson <mwh@python.net> writes:

> (I have a hacky patch which makes exceptions new-style which I'll
> post in a moment).

Well, it turns out to be a bit big for attaching, so it's here:

http://starship.python.net/crew/mwh/hacks/new-style-exceptions-hacking.diff

This is very much a first cut; no attempt at subtlety.  The procedure
went roughly "Hack until it compiles, hack until it doesn't dump core
immediately, hack until most tests pass."

The good news: all tests but test_pickletools pass (and that's doomed;
just look at it).

The bad news: I've entirely broken raising old-style classes (well,
not quite:

>>> try: raise C
... except types.ClassType, c: print c
... 
__main__.C

) so I've had to make sure various classes used in the test suite
inherit from exception.

There was a bunch of shallow breakage -- f'ex str(old-style-class) is
quite different from str(new-style-class), which broke various output
comparison tests (try not to gag when you see how I worked around
this) -- but not much that's deep.

You can get a *different* kind of shallow breakage by essentially
removing old-style classes (changing the default metatype to type),
but then 'types.ClassType is type' and this from copy_reg.py:

def pickle(ob_type, pickle_function, constructor_ob=None):
    if type(ob_type) is _ClassType:
        raise TypeError("copy_reg is not intended for use with classes")

rather fails to do the right thing.  I didn't pursue this one very
far.

Obviously, a better attempt would be to allow raising any old-style
class or instance or any subtype of Exception or any instance of a
subtype of Exception -- but that becomes tedious to spell.

I suspect that it would be quite hard -- or at least prohibitively
tedious -- to write code that worked with both old- and new-style
exceptions, so I'm not sure a -N switch to turn them on would work.
At least not without a small battery of helper functions that noone
would bother to use.

I guess this means making exceptions new-style might have to wait for
a Python 3.0-ish flag day of some kind.

Cheers,
mwh

-- 
  Roll on a game of competetive offence-taking.
                                            -- Dan Sheppard, ucam.chat
From mwh at python.net  Thu Aug  5 15:30:20 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug  5 15:30:22 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <1091711049.41123049900cb@mcherm.com> (Michael Chermside's
	message of "Thu,  5 Aug 2004 06:04:09 -0700")
References: <1091711049.41123049900cb@mcherm.com>
Message-ID: <2mvffx7lw3.fsf@starship.python.net>

Michael Chermside <mcherm@mcherm.com> writes:

> Michael Hudson writes:
>> > that would be insane,
>>
>> class C(random.choice([dict, list])):
>>     pass
>>
>> is my favourite example of this :-)
>
> Where is THAT monstrocity used!?

Well, it's not, I hope :-)

It might occasionally be useful for proving to people that Python
really doesn't believe in declarations.

Cheers,
mwh

-- 
  Need to Know is usually an interesting UK digest of things that
  happened last week or might happen next week. [...] This week,
  nothing happened, and we don't care.
                           -- NTK Now, 2000-12-29, http://www.ntk.net/
From tameyer at ihug.co.nz  Thu Aug  5 03:56:49 2004
From: tameyer at ihug.co.nz (Tony Meyer)
Date: Thu Aug  5 16:00:11 2004
Subject: [Python-Dev] RE: [spambayes-dev] bug in imap filter or in email
	package
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E874F398@its-xchg4.massey.ac.nz>
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E86C36A1@its-xchg4.massey.ac.nz>

There has been a little bit of discussion on spambayes-dev about a bug with
the 2.4a1 email package, where header lines that end with \r\n are not
treated correctly (the value ends up with a \r at the end).

A SF bug was opened for this:

[ 1002475 ] email message parser doesn't handle \r\n correctly
<http://sf.net/tracker/?group_id=5470&atid=105470&func=detail&aid=1002475>

I've created a patch to fix this, and a couple of tests to add to
test_email.py:
[ 1003693 ] Fix for 1002475 (Feedparser not handling \r\n correctly)
<http://sf.net/tracker/index.php?func=detail&aid=1003693&group_id=5470&atid=
305470>

If someone would like to review this and check it in after 2.4a2 is all done
that would be great.  Maybe someone at the bug day?  (I might come along to
that, but it's the middle of the night, so probably not).

Thanks!

=Tony Meyer

From chrism at plope.com  Thu Aug  5 16:04:56 2004
From: chrism at plope.com (Chris McDonough)
Date: Thu Aug  5 16:05:01 2004
Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module))
In-Reply-To: <200408050550.i755ob1s031939@cosc353.cosc.canterbury.ac.nz>
References: <200408050550.i755ob1s031939@cosc353.cosc.canterbury.ac.nz>
Message-ID: <1091714696.26267.623.camel@athlon>

On Thu, 2004-08-05 at 01:50, Greg Ewing wrote:
> Chris McDonough <chrism@plope.com>:
> 
> > The following program demonstrates that a particular usage of select
> > (under Linux at least) always returns the output side of a pipe
> > connected to a child process' stdout as "ready" after it gets any output
> > from that child process, even if the child process has no further data
> > to provide after it has provided a bit of data to the parent.
> >
> > Or am I doing something silly?
> > 
> >         for fd in r:
> >             sys.stdout.write(os.read(fd, 1024))
> 
> You're not taking account of the possibility of EOF. When
> the child process finishes and closes its end of the pipe,
> the select will always report the pipe as ready for reading,
> because it is -- subsequent reads will return immediately
> with 0 bytes.
> 
> You need to check whether the read returned 0 bytes, and
> take that as meaning that the child process has finished.

Of course!  Thanks.  Let us speak no more of this. ;-)

- C


From martin at v.loewis.de  Thu Aug  5 16:19:33 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 16:19:30 2004
Subject: [Python-Dev] RELEASED Python 2.4, alpha 2
In-Reply-To: <411231C8.3020308@interlink.com.au>
References: <411231C8.3020308@interlink.com.au>
Message-ID: <411241F5.3080602@v.loewis.de>

Anthony Baxter wrote:
> Python 2.4a2 is an alpha release. We'd greatly appreciate it if you
> could download it, kick the tires and let us know of any problems you
> find, but it is not suitable for production usage.

The Windows installer should support upgrading from a previous Python 
2.4 installation. If you have previously installed 2.4a1, you may try
this out; please report any problems you find.

Regards,
Martin
From aahz at pythoncraft.com  Thu Aug  5 17:22:03 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Aug  5 17:22:07 2004
Subject: Except that! (was Re: [Python-Dev] Python in Unicode context)
In-Reply-To: <41121E87.9010805@egenix.com>
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
	<41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com>
	<2md62596kl.fsf@starship.python.net> <41121E87.9010805@egenix.com>
Message-ID: <20040805152203.GB23495@panix.com>

On Thu, Aug 05, 2004, M.-A. Lemburg wrote:
>
> I meant that it works for the vast majority of all cases you see in
> practice. I haven't seen a non-Exception based exception in years.

My current company still has lots of string exceptions.  :-(  I'm
working on changing that.  (Yes, we started with Python 1.4.)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From aahz at pythoncraft.com  Thu Aug  5 17:26:45 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Aug  5 17:26:48 2004
Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python in
	Unicode context)
In-Reply-To: <20040805112928.GA5208@solar.trillke>
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com>
	<20040803194724.GB9031@titan.progiciels-bpi.ca>
	<200408032014.i73KEwp31876@guido.python.org>
	<20040805112928.GA5208@solar.trillke>
Message-ID: <20040805152645.GC23495@panix.com>

On Thu, Aug 05, 2004, Holger Krekel wrote:
> Guido van Rossum wrote:
>>
>> (It will also break code that creates a class used as an exception
>> that doesn't derive from Exception, but those should be shot. :-)
> 
> Then i guess that searching down into a recursive structure and just 
> raising an "i found it" result object up doesn't count as a use case in
> your book, right?   It can avoid boilerplate code like return-if-not-None 
> checks and I have used it for e.g. finding patterns in an AST-Tree. 

In cases where I've done this, I've always inherited from Exception or a
subclass.  Is there any reason not to?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From tim.peters at gmail.com  Thu Aug  5 17:28:33 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug  5 17:28:42 2004
Subject: [Python-Dev] New-style exceptions
In-Reply-To: <2mzn597m02.fsf_-_@starship.python.net>
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
	<41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com>
	<2md62596kl.fsf@starship.python.net>
	<2mzn597m02.fsf_-_@starship.python.net>
Message-ID: <1f7befae04080508286f6bcae9@mail.gmail.com>

[Michael Hudson]
...
> Well, it turns out to be a bit big for attaching, so it's here:
> 
> http://starship.python.net/crew/mwh/hacks/new-style-exceptions-hacking.diff
> 
> This is very much a first cut; no attempt at subtlety.  The procedure
> went roughly "Hack until it compiles, hack until it doesn't dump core
> immediately, hack until most tests pass."
> 
> The good news: all tests but test_pickletools pass (and that's doomed;
> just look at it).

Eh?  test_pickletools is a three-line test, which just runs the
doctests in pickletools.py.  The only exceptions mentioned in the
latter are the builtin ValueError and OverflowError.  What's the
problem?
From mwh at python.net  Thu Aug  5 17:32:00 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug  5 17:32:01 2004
Subject: [Python-Dev] New-style exceptions
In-Reply-To: <1f7befae04080508286f6bcae9@mail.gmail.com> (Tim Peters's
	message of "Thu, 5 Aug 2004 11:28:33 -0400")
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
	<41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com>
	<2md62596kl.fsf@starship.python.net>
	<2mzn597m02.fsf_-_@starship.python.net>
	<1f7befae04080508286f6bcae9@mail.gmail.com>
Message-ID: <2mr7ql7g9b.fsf@starship.python.net>

Tim Peters <tim.peters@gmail.com> writes:

> [Michael Hudson]
> ...
>> Well, it turns out to be a bit big for attaching, so it's here:
>> 
>> http://starship.python.net/crew/mwh/hacks/new-style-exceptions-hacking.diff
>> 
>> This is very much a first cut; no attempt at subtlety.  The procedure
>> went roughly "Hack until it compiles, hack until it doesn't dump core
>> immediately, hack until most tests pass."
>> 
>> The good news: all tests but test_pickletools pass (and that's doomed;
>> just look at it).
>
> Eh?  test_pickletools is a three-line test, which just runs the
> doctests in pickletools.py.  The only exceptions mentioned in the
> latter are the builtin ValueError and OverflowError.  What's the
> problem?

Sorry, was too obscure.  One of the doctests pickles a couple
instances of PicklingError and disassembles the pickle.  That's not
going to stay the same past an old-style/new-style transition.

Cheers,
mwh

-- 
  If a train station is a place where a train stops, what's a
  workstation?                            -- unknown (to me, at least)
From pyth at devel.trillke.net  Thu Aug  5 17:34:43 2004
From: pyth at devel.trillke.net (Holger Krekel)
Date: Thu Aug  5 17:34:47 2004
Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python in
	Unicode context)
In-Reply-To: <20040805152645.GC23495@panix.com>
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com>
	<20040803194724.GB9031@titan.progiciels-bpi.ca>
	<200408032014.i73KEwp31876@guido.python.org>
	<20040805112928.GA5208@solar.trillke>
	<20040805152645.GC23495@panix.com>
Message-ID: <20040805153443.GB5208@solar.trillke>

Aahz wrote:
> On Thu, Aug 05, 2004, Holger Krekel wrote:
> > Guido van Rossum wrote:
> >>
> >> (It will also break code that creates a class used as an exception
> >> that doesn't derive from Exception, but those should be shot. :-)
> > 
> > Then i guess that searching down into a recursive structure and just 
> > raising an "i found it" result object up doesn't count as a use case in
> > your book, right?   It can avoid boilerplate code like return-if-not-None 
> > checks and I have used it for e.g. finding patterns in an AST-Tree. 
> 
> In cases where I've done this, I've always inherited from Exception or a
> subclass.  Is there any reason not to?

Sure, i can probably wrap the result object into some class
which inherits from Exception.  My point is that I like to
regard try/except as a mechanism for "out-of-band" objects.
Guidos "should be shot" seems to indicate he sees try/except
only useful/applicable to exception-handling. 

    Holger

P.S.: thanks for changing the subject line, should have done that earlier. 
From fumanchu at amor.org  Thu Aug  5 17:34:43 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Thu Aug  5 17:39:41 2004
Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python
	inUnicode context)
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DDF@exchange.hqamor.amorhq.net>

Holger Krekel wrote:
> Sure, i can probably wrap the result object into some class
> which inherits from Exception.  My point is that I like to
> regard try/except as a mechanism for "out-of-band" objects.
> Guidos "should be shot" seems to indicate he sees try/except
> only useful/applicable to exception-handling. 

Putting on my "language lawyer" hat:

"Exceptions are a means of breaking out of the normal flow of control of
a code block in order to handle errors or other exceptional conditions.
An exception is raised at the point where the error is detected; it may
be handled by the surrounding code block or by any code block that
directly or indirectly invoked the code block where the error occurred."

If "exception" is only meant for errors, then don't use it for
non-errors. But if "exception" is meant for "other exceptional
conditions", then why not use it for them? Sounds to me like you just
need to expand your use of the term "exception", and therefore, your use
of Exception(). ;)


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From aahz at pythoncraft.com  Thu Aug  5 18:04:33 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Aug  5 18:04:42 2004
Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python in
	Unicode context)
In-Reply-To: <20040805153443.GB5208@solar.trillke>
References: <20040803145345.GA6250@titan.progiciels-bpi.ca>
	<410FCA3B.1020002@v.loewis.de> <410FCCFF.4050806@egenix.com>
	<20040803194724.GB9031@titan.progiciels-bpi.ca>
	<200408032014.i73KEwp31876@guido.python.org>
	<20040805112928.GA5208@solar.trillke>
	<20040805152645.GC23495@panix.com>
	<20040805153443.GB5208@solar.trillke>
Message-ID: <20040805160433.GA9809@panix.com>

On Thu, Aug 05, 2004, Holger Krekel wrote:
> Aahz wrote:
>> On Thu, Aug 05, 2004, Holger Krekel wrote:
>>> Guido van Rossum wrote:
>>>>
>>>> (It will also break code that creates a class used as an exception
>>>> that doesn't derive from Exception, but those should be shot. :-)
>>> 
>>> Then i guess that searching down into a recursive structure and just 
>>> raising an "i found it" result object up doesn't count as a use case in
>>> your book, right?   It can avoid boilerplate code like return-if-not-None 
>>> checks and I have used it for e.g. finding patterns in an AST-Tree. 
>> 
>> In cases where I've done this, I've always inherited from Exception or a
>> subclass.  Is there any reason not to?
> 
> Sure, i can probably wrap the result object into some class which
> inherits from Exception.  My point is that I like to regard try/except
> as a mechanism for "out-of-band" objects.  Guidos "should be shot"
> seems to indicate he sees try/except only useful/applicable to
> exception-handling.

Nope, he meant exactly what he said: an exception that doesn't derive
from Exception.  After all, the iterator protocol relies on similar
out-of-band exceptions, and the ``for`` loop has done the same with
IndexError for years.

Further discussion about Pythonic exception handling should probably get
moved to comp.lang.python
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From foom at fuhm.net  Thu Aug  5 18:10:34 2004
From: foom at fuhm.net (James Y Knight)
Date: Thu Aug  5 18:10:39 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <410F3465.9000005@object-craft.com.au>
References: <410F3465.9000005@object-craft.com.au>
Message-ID: <FB04F1AA-E6F9-11D8-8D12-000A95A50FB2@fuhm.net>

On Aug 3, 2004, at 2:44 AM, Dave Cole wrote:
> >>> a_list = [1, 2, 3, 4, 5]
> >>> a, b, *c = a_list

While you're proposing expanding the domain of the * construct of 
function calling to other parts of python, I'd also like to suggest the 
following features, to make the ** construct also applicable to the 
rest of the language.

To be read with an 70% wink. (I wouldn't be *un*happy if these were in 
python, but I am not seriously proposing these features for inclusion):

Dict unpacking, parallels current sequence unpacking:
 >>> d = {'foo': 1, 'bar': 2}
 >>> {'foo': x, 'bar': y} = d
 >>> x, y
(1, 2)

Dict interpolation, similar to sequence interpolation and keyword 
interpolation in function calls:
 >>> d = {'foo': 1, 'bar': 2}
 >>> d2 = {'baz': 3, **d}
 >>> d2
{'baz': 3, 'foo': 1, 'bar': 2}

Combining the two:
 >>> d = {'foo': 1, 'bar': 2, 'baz': 3}
 >>> {'foo': x, 'bar': y, **d2} = d
 >>> x, y, d2
(1, 2, {'baz': 3})

James

From guido at python.org  Thu Aug  5 18:36:36 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug  5 18:36:41 2004
Subject: [Python-Dev] Call for defense of @decorators
Message-ID: <200408051636.i75Gaak03654@guido.python.org>

It seems a public outcry against @decorators has started.  When I let
Anthony check it in for 2.4a2, the plan was to see how it fares in a2
and a3, and possibly take it out in 2.4b1.  If you don't want that to
happen, you might want to say something in public in defense of
@decorators.

We also need help updating the PEP; it doesn't mention @decorators,
and it doesn't even begin to mention the pro and con of many other
suggestions (does it say why "def decorator func(...)" was rejected?).
I don't have time for any of this; I can barely make time for a few
emails per day.  Perhaps the @advocates can elect or volunteer a PEP
editor.

I really don't care if everybody thinks it's ugly.  I do care to find
out about usability issues.  For example, it may cause problems for
Leo, but I don't know how bad that is.  I also want to find out about
superior syntax proposals (from __future__ import decorators might be
acceptable).

An argument about timing: IMO there's no point in putting it off until
2.5 -- the syntax options don't get prettier, backwards compatibility
issues don't get easier by waiting, and as it is it will probably take
a year from 2.4's release in December before it is the mainstream
Python version.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From pje at telecommunity.com  Thu Aug  5 18:52:28 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug  5 18:48:21 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051636.i75Gaak03654@guido.python.org>
Message-ID: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>

At 09:36 AM 8/5/04 -0700, Guido van Rossum wrote:
>I also want to find out about
>superior syntax proposals (from __future__ import decorators might be
>acceptable).

Does this mean that the C#-style syntax has a chance if it's got a 
__future__?  :)

Also, you might want to define "superior" in order to avoid re-opening the 
floodgates of syntax argument.

With regard to the PEP, I thought there were two volunteers who mentioned 
an intent to work on it in the past week; if they are not still doing so, 
I'd be happy to at least add the issues with "def decorator functionname()" 
that I remember (visual confusion for decorators w/arguments, tool 
confusion for existing tools).

From anthony at interlink.com.au  Thu Aug  5 18:49:46 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Aug  5 18:50:12 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051636.i75Gaak03654@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
Message-ID: <4112652A.90004@interlink.com.au>

Guido van Rossum wrote:
> We also need help updating the PEP; it doesn't mention @decorators,
> and it doesn't even begin to mention the pro and con of many other
> suggestions (does it say why "def decorator func(...)" was rejected?).
> I don't have time for any of this; I can barely make time for a few
> emails per day.  Perhaps the @advocates can elect or volunteer a PEP
> editor.

I'm totally out of cycles to work on the PEP any more. I've sent
the changes I made to Skip, who'd previously volunteered to help.

Anthony


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From anthony at interlink.com.au  Thu Aug  5 18:59:37 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Aug  5 19:00:05 2004
Subject: [Python-Dev] trunk frozen for 2.4a2
In-Reply-To: <41112923.8060505@interlink.com.au>
References: <41112923.8060505@interlink.com.au>
Message-ID: <41126779.1080308@interlink.com.au>

Anthony Baxter wrote:
> Please hold off on any checkins to the trunk for the next 24
> hours or so (I'll post a followup when we're done) for 2.4a2.
> (The usual exception for the release team of me, Fred and
> Martin applies).

The release is done. Please keep the trunk frozen for another
8 hours or so, in case I need to cut an emergency fix-up release.

I'll send another email in the morning to open it up again.

Thanks,
Anthony
From mwh at python.net  Thu Aug  5 19:00:23 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug  5 19:00:25 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051636.i75Gaak03654@guido.python.org> (Guido van
	Rossum's message of "Thu, 05 Aug 2004 09:36:36 -0700")
References: <200408051636.i75Gaak03654@guido.python.org>
Message-ID: <2mn0197c60.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> It seems a public outcry against @decorators has started.  When I let
> Anthony check it in for 2.4a2, the plan was to see how it fares in a2
> and a3, and possibly take it out in 2.4b1.  If you don't want that to
> happen, you might want to say something in public in defense of
> @decorators.

Like what?  I don't want to see that happen.

Do you want justifications, too?  :-)

I would beg of you to not give the idea that you or anyone else is
going to be counting votes on this at some point.

> An argument about timing: IMO there's no point in putting it off until
> 2.5 -- the syntax options don't get prettier, backwards compatibility
> issues don't get easier by waiting, and as it is it will probably take
> a year from 2.4's release in December before it is the mainstream
> Python version.

+1

Cheers,
mwh

-- 
  ARTHUR:  The ravenours bugblatter beast of Traal ... is it safe?
    FORD:  Oh yes, it's perfectly safe ... it's just us who are in 
           trouble.
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 6
From guido at python.org  Thu Aug  5 19:14:48 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug  5 19:14:53 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: Your message of "Thu, 05 Aug 2004 12:52:28 EDT."
	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> 
References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> 
Message-ID: <200408051714.i75HEmE03796@guido.python.org>

[Phillip]
> Does this mean that the C#-style syntax has a chance if it's got a 
> __future__?  :)

I don't see how that would change the arguments against it.

> Also, you might want to define "superior" in order to avoid
> re-opening the floodgates of syntax argument.

No, but I suggest that the proponents of syntax alternatives will have
to agree amongst themselves on a single alternative that they can
present to me.

> With regard to the PEP, I thought there were two volunteers who
> mentioned an intent to work on it in the past week; if they are not
> still doing so, I'd be happy to at least add the issues with "def
> decorator functionname()" that I remember (visual confusion for
> decorators w/arguments, tool confusion for existing tools).

Please do (or coordinate with Skip who seems to have attracted this
volunteer position).

[Michael]
> Do you want justifications, too?  :-)

That's up to you. :-)

> I would beg of you to not give the idea that you or anyone else is
> going to be counting votes on this at some point.

Python is not a democracy.  I can't be swayed by votes, only by good
arguments.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Thu Aug  5 19:39:48 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug  5 19:39:52 2004
Subject: [Python-Dev] RELEASED Python 2.4, alpha 2
In-Reply-To: <411241F5.3080602@v.loewis.de>
References: <411231C8.3020308@interlink.com.au> <411241F5.3080602@v.loewis.de>
Message-ID: <1f7befae040805103911610582@mail.gmail.com>

[Martin v. L?wis]
> The Windows installer should support upgrading from a previous Python
> 2.4 installation. If you have previously installed 2.4a1, you may try
> this out; please report any problems you find.

I tried that, and it seemed to work well, although it took a while.

One glitch:  I'm pretty sure I selected "install only for me" in the
first screen.  I wanted to go back to check, but it doesn't seem
possible to go back to the first screen.  Anyway, the Python DLL ended
up in a system directory, and that's "the glitch".  I expected/wanted
it to end up in my Python directory.
From niemeyer at conectiva.com  Thu Aug  5 19:42:52 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Aug  5 19:40:38 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051636.i75Gaak03654@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
Message-ID: <20040805174252.GA27820@burma.localdomain>

[...]
> We also need help updating the PEP; it doesn't mention @decorators,
> and it doesn't even begin to mention the pro and con of many other
> suggestions (does it say why "def decorator func(...)" was rejected?).

It doesn't seem to even mention it. I also belive this is a much better
syntax, which doesn't introduce a new symbol, and reads more naturally.
I'd really like to see the current implemented syntax replaced by this
one, but I'm not sure what's the correct way to proceed from here.

> I don't have time for any of this; I can barely make time for a few
> emails per day.  Perhaps the @advocates can elect or volunteer a PEP
> editor.
> 
> I really don't care if everybody thinks it's ugly.  I do care to find

You don't care if everybody thinks this is ugly!? That's bad.

> out about usability issues.  For example, it may cause problems for
> Leo, but I don't know how bad that is.  I also want to find out about
> superior syntax proposals (from __future__ import decorators might be
> acceptable).

How can I help defining something superior? You said you don't want
voting, and that this is not a democracy, and that you don't care
if the current syntax is ugly. I'm worried about this issue.

-- 
Gustavo Niemeyer
http://niemeyer.net
From niemeyer at conectiva.com  Thu Aug  5 19:46:19 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Aug  5 19:44:01 2004
Subject: [Python-Dev] Plea for simpler decorator syntax,
	in addition to pie-shaped syntax
In-Reply-To: <4110F829.2000609@zope.com>
References: <4110F829.2000609@zope.com>
Message-ID: <20040805174619.GB27820@burma.localdomain>

> IMO, the most common uses of decorators will be to define properties,
> and class and static methods.  IMO, these uses would be better served
> by a simpler syntax:
> 
>   def classmethod foo(cls, ...):
>       ...
> 
> This simplified syntax only allows names to specify decorators.  It
> could allow multiple names, although I'm not sure it should,
> 
> I find this *far* more readable and obvious than any of the other syntaxs
> I've seen propsed.

Agreed.

> Those applications that *need* decorator arguments could use the more
> complex pie-shaped notation.

I wouldn't care to define a decorator function to introduce arguments,
and force every decorator function to take a single argument.

-- 
Gustavo Niemeyer
http://niemeyer.net
From martin at v.loewis.de  Thu Aug  5 19:56:16 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 19:56:13 2004
Subject: [Python-Dev] RELEASED Python 2.4, alpha 2
In-Reply-To: <1f7befae040805103911610582@mail.gmail.com>
References: <411231C8.3020308@interlink.com.au> <411241F5.3080602@v.loewis.de>
	<1f7befae040805103911610582@mail.gmail.com>
Message-ID: <411274C0.4070300@v.loewis.de>

Tim Peters wrote:

> I tried that, and it seemed to work well, although it took a while.

Yes. I should probably speed it up by changing the UI sequence number
of RemoveExistingApplications from 1510 to 1450. In case you wonder
what this means: Action 1500 is InstallInitialize; anything from then
on is under transaction control. This means that installer will roll
back the uninstall in case the subsequent install fails. This is
expensive, as it first moves all old files out of the place, then puts
the new ones in, then deletes the old ones. At 1450, the files would
be deleted immediately (I believe), at the cost of not being able to
undo the deinstallation if the installation later fails (e.g. by
user Cancel).

> One glitch:  I'm pretty sure I selected "install only for me" in the
> first screen.  I wanted to go back to check, but it doesn't seem
> possible to go back to the first screen.

Yes. I don't know what to do about this: the screen sets ALLUSERS=1,
then performs FindRelatedProducts to find the old installation. If the
old installation was per-machine, we only find it if we commit to making
the new installation also per-machine. That decision cannot be taken
back; the user would have to cancel and start over.

> Anyway, the Python DLL ended
> up in a system directory, and that's "the glitch".  I expected/wanted
> it to end up in my Python directory.

Ok. I will see whether I can reproduce that.

Regards,
Martin
From foom at fuhm.net  Thu Aug  5 20:04:56 2004
From: foom at fuhm.net (James Y Knight)
Date: Thu Aug  5 20:05:00 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051636.i75Gaak03654@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
Message-ID: <F5909402-E709-11D8-8D12-000A95A50FB2@fuhm.net>

One thing I thought of recently that would be IMO a nice feature to 
have is something like this:

 >>>
@public
def f(...):
  pass

@public
y = 5
<<<

which would essentially set __all__ = ['f', 'y'] for you. However, that 
isn't possible to do with the current function-modifying-only 
decorators. For this to be possible, the decorator would need to get as 
an argument the binding (perhaps in the form of the container and the 
name), as well as the object being assigned to that binding. (also the 
grammar would have to be updated but that's relatively easy I suspect)

I don't have an argument or proposal here, just putting this out to 
think about.

James

PS: I like decorators, and the pie-shape is really growing on me. :)

From tim.peters at gmail.com  Thu Aug  5 20:23:08 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug  5 20:23:11 2004
Subject: [Python-Dev] New-style exceptions
In-Reply-To: <2mr7ql7g9b.fsf@starship.python.net>
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
	<41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com>
	<2md62596kl.fsf@starship.python.net>
	<2mzn597m02.fsf_-_@starship.python.net>
	<1f7befae04080508286f6bcae9@mail.gmail.com>
	<2mr7ql7g9b.fsf@starship.python.net>
Message-ID: <1f7befae0408051123581f5f43@mail.gmail.com>

[Michael Hudson]
> Sorry, was too obscure.  One of the doctests pickles a couple
> instances of PicklingError and disassembles the pickle.  That's not
> going to stay the same past an old-style/new-style transition.

That's no problem.  The real point of that part of the test is to
exercise the different pickle protocols on an instance of a "foreign"
(not defined in the same module) class.  Any foreign class would do as
well  Creating an instance of pickle.PicklingError was just
convenient; there was no intent to pick on an exception class.  IOW,
if this goes forward, it's no problem to change the test.
From trentm at ActiveState.com  Thu Aug  5 20:23:18 2004
From: trentm at ActiveState.com (Trent Mick)
Date: Thu Aug  5 20:23:35 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805174252.GA27820@burma.localdomain>;
	from niemeyer@conectiva.com on Thu, Aug 05, 2004 at 02:42:52PM
	-0300
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
Message-ID: <20040805112318.B13062@ActiveState.com>

[Gustavo Niemeyer wrote]
> How can I help defining something superior? You said you don't want
> voting, and that this is not a democracy, and that you don't care
> if the current syntax is ugly. I'm worried about this issue.

You are being disingenuous, Gustavo. Guido said he doesn't want voting,
**he wants reasoned arguments**. He cares more about **usability
issues** than about the prettiness of the syntax.

Trent

-- 
Trent Mick
TrentM@ActiveState.com
From ronaldoussoren at mac.com  Thu Aug  5 20:26:42 2004
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu Aug  5 20:26:56 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051636.i75Gaak03654@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
Message-ID: <FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>


On 5-aug-04, at 18:36, Guido van Rossum wrote:

> It seems a public outcry against @decorators has started.  When I let
> Anthony check it in for 2.4a2, the plan was to see how it fares in a2
> and a3, and possibly take it out in 2.4b1.  If you don't want that to
> happen, you might want to say something in public in defense of
> @decorators.

The outcry seems to be the same discussion as ever :-(

I'm in favor of @decorators. It's easy to notice them when the are 
present and they are clearly special. The '[decorator] def ...' and 
'decorate(decorator) def ...' are very magic, and are IMHO unfriendly 
to newbies (you must metion them in any introduction to python, because 
otherwise users would complety mis the signicance of the decorations).

My particular use-case is PyObjC, which sometimes suffers from haveing 
veryLong_andSometimesAlsoVeryUgly_methodNames_. Having to repeat those 
three times when using a decorator is no fun. The only usage for 
decorators in PyObjC is for specifying Objective-C method signatures, 
which is sometimes required for the correct functioning of methods (I 
do try to deduce the correct method signature from methods in base 
classes and Objective-C protocols, but's that not always necessary).

Some examples:

class ModelObject (NSObject):

	@objc.accessor
	def messageLength(self):
		return "XIV"

	@objc.signature("v@:@i")
	def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, 
returnCode, contextInfo):
		pass

The argument to objc.signature is fairly magic and I do try to abstract 
it away (such as with the introduction of objc.accessor), but that's 
not always possible. If decorators were not allowed to have arguments 
I'd have to introduce temporary functions that wouldn't help in 
readability.

The generic example PJE is introducing in PEAK also seem a good usecase 
for decorators with arguments.

Ronald

From astrand at lysator.liu.se  Thu Aug  5 20:30:03 2004
From: astrand at lysator.liu.se (Peter Astrand)
Date: Thu Aug  5 20:30:13 2004
Subject: select() vs. pipes (was [Python-Dev] Re: PEP 324 (process module))
In-Reply-To: <1091649831.26267.527.camel@athlon>
References: <E1Brz79-0004QH-S5@sc8-pr-cvs1.sourceforge.net> 
	<200408031427.i73ERn330790@guido.python.org> 
	<Pine.GSO.4.51L2.0408031703580.7455@koeberg.lysator.liu.se> 
	<20040803213121.C9777@ActiveState.com>
	<Pine.GSO.4.51L2.0408041551070.16055@koeberg.lysator.liu.se>
	<1091631453.26267.390.camel@athlon>
	<200408041520.i74FKm001469@guido.python.org>
	<1091636838.26267.406.camel@athlon>
	<Pine.GSO.4.51L2.0408042054130.18090@koeberg.lysator.liu.se>
	<1091649831.26267.527.camel@athlon>
Message-ID: <Pine.GSO.4.51L2.0408052025140.26994@koeberg.lysator.liu.se>


> In popen5, "communicate" is terminal.  It calls select until there's no
> more data to get back and then unconditionally waits for the subprocess
> to finish, blocking the entire time.  This isn't useful for the type of

Yes, I agree. Other people has request this as well.

But, even though this could be useful, I do not consider it a showstopper
which needs to be solved before the module can be included in the stdlib.
Windows support without need for win32all is a more major issue, I think.

A non-terminal "communicate" can be added later.


/Peter ?strand <astrand@lysator.liu.se>

From mwh at python.net  Thu Aug  5 20:30:27 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug  5 20:30:29 2004
Subject: [Python-Dev] New-style exceptions
In-Reply-To: <1f7befae0408051123581f5f43@mail.gmail.com> (Tim Peters's
	message of "Thu, 5 Aug 2004 14:23:08 -0400")
References: <200408050133.i751XOiL031491@cosc353.cosc.canterbury.ac.nz>
	<41119C3E.5060509@prescod.net> <4111E598.5020302@egenix.com>
	<2md62596kl.fsf@starship.python.net>
	<2mzn597m02.fsf_-_@starship.python.net>
	<1f7befae04080508286f6bcae9@mail.gmail.com>
	<2mr7ql7g9b.fsf@starship.python.net>
	<1f7befae0408051123581f5f43@mail.gmail.com>
Message-ID: <2misbx77zw.fsf@starship.python.net>

Tim Peters <tim.peters@gmail.com> writes:

> [Michael Hudson]
>> Sorry, was too obscure.  One of the doctests pickles a couple
>> instances of PicklingError and disassembles the pickle.  That's not
>> going to stay the same past an old-style/new-style transition.
>
> That's no problem.  The real point of that part of the test is to
> exercise the different pickle protocols on an instance of a "foreign"
> (not defined in the same module) class.  Any foreign class would do as
> well  Creating an instance of pickle.PicklingError was just
> convenient; there was no intent to pick on an exception class.  IOW,
> if this goes forward, it's no problem to change the test.

Uhh, sure.  It was just to tell people what to expect if they were so
foolish as to download the patch and try it :-)

I probably didn't choose my words very well.

Cheers,
mwh

-- 
  This is the fixed point problem again; since all some implementors
  do is implement the compiler and libraries for compiler writing, the
  language becomes good at writing compilers and not much else!
                                 -- Brian Rogoff, comp.lang.functional
From niemeyer at conectiva.com  Thu Aug  5 20:34:44 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Aug  5 20:32:26 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805112318.B13062@ActiveState.com>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
Message-ID: <20040805183444.GA29796@burma.localdomain>

> [Gustavo Niemeyer wrote]
> > How can I help defining something superior? You said you don't want
> > voting, and that this is not a democracy, and that you don't care
> > if the current syntax is ugly. I'm worried about this issue.
> 
> You are being disingenuous, Gustavo. Guido said he doesn't want voting,
> **he wants reasoned arguments**. He cares more about **usability
> issues** than about the prettiness of the syntax.

I've read that in his mail. Do you have something to add,
besides personal insults, or just happen to have a lot of
free time?

-- 
Gustavo Niemeyer
http://niemeyer.net
From pje at telecommunity.com  Thu Aug  5 20:41:38 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug  5 20:37:33 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051714.i75HEmE03796@guido.python.org>
References: <Your message of "Thu, 05 Aug 2004 12:52:28 EDT."
	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com>

At 10:14 AM 8/5/04 -0700, Guido van Rossum wrote:
>[Phillip]
> > Does this mean that the C#-style syntax has a chance if it's got a
> > __future__?  :)
>
>I don't see how that would change the arguments against it.

I thought the primary argument against it was that it changes the meaning 
of (potentially existing) Python code, and that you had rejected the "hard 
to learn" argument on the basis that people learn by pattern-matching 
rather than principle.

I guess this is another reason to update the PEP...  :)


>No, but I suggest that the proponents of syntax alternatives will have
>to agree amongst themselves on a single alternative that they can
>present to me.

I think that will pretty much guarantee that it's either @ or nothing: it 
appears that the two biggest (or at least most vocal) camps are:

1. people who want a "simpler" syntax that doesn't support arguments (who 
seem to mostly favor 'def classmethod foo()')

2. people who think that decorators without arguments are pointless, and 
don't agree amongst themselves on the proper syntax, but don't necessarily 
care that much as long as there *is* one.  (But there may be a slight 
leaning towards either of the C#-inspired variants.)

Personally, I think the C# list-before-function syntax is, on balance, the 
best of all the choices explored throughout the whole tangled history of 
PEP 318.  It's visually attractive, doesn't *break* the parsing of existing 
tools (that may see '@' as a syntax error), and doesn't interfere with 
grepping for functions.  The "learning" argument was done to death before 
you even invented the syntax: *any* decorator choice results in having to 
learn *something*.

Of course, I suppose that nearly all of these can be turned into opposite 
arguments, for example that '@' causing breakage is a good thing, because 
it means you'll get an error sooner (e.g. if you run the program under an 
older Python).  And round and round the argument goes.  And every so often, 
somebody invents a new syntax, for example:

      foo = classmethod(
          def foo(cls):
              return 42
      )

and then some people love it, and some hate it, and there we go back on the 
merry-go-round.  Sigh.  Just Pronounce on *something* and put us all out of 
your misery already.  :)

From skip at pobox.com  Thu Aug  5 20:38:52 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug  5 20:39:08 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051714.i75HEmE03796@guido.python.org>
References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<200408051714.i75HEmE03796@guido.python.org>
Message-ID: <16658.32444.554073.223400@montanaro.dyndns.org>


    >> With regard to the PEP, I thought there were two volunteers who
    >> mentioned an intent to work on it in the past week; if they are not
    >> still doing so, I'd be happy to at least add the issues with "def
    >> decorator functionname()" that I remember (visual confusion for
    >> decorators w/arguments, tool confusion for existing tools).

    Guido> Please do (or coordinate with Skip who seems to have attracted
    Guido> this volunteer position).

I volunteered simply because my name is currently on the PEP as one of its
authors.  One of the loudest cries on c.l.py came from someone (Edward Ream
I think) complaining that the PEP as it currently stands doesn't really
describe the changes that were just checked in.  I can't really fault him
for it.  Had I (or associated with the PEP in the past) been alerted, I
think that particular complaint could have been avoided.

I do have Anthony's latest updates and will try to factor them into the PEP
this evening.  If others have editing suggestions, getting them to me
earlier is better than later.

Skip
From eppstein at ics.uci.edu  Thu Aug  5 20:39:10 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Thu Aug  5 20:39:15 2004
Subject: [Python-Dev] Re: Plea for simpler decorator syntax,
	in addition to pie-shaped syntax
References: <4110F829.2000609@zope.com>
	<20040805174619.GB27820@burma.localdomain>
Message-ID: <eppstein-206D17.11391005082004@sea.gmane.org>

In article <20040805174619.GB27820@burma.localdomain>,
 Gustavo Niemeyer <niemeyer@conectiva.com> wrote:

> > IMO, the most common uses of decorators will be to define properties,
> > and class and static methods.  IMO, these uses would be better served
> > by a simpler syntax:
> > 
> >   def classmethod foo(cls, ...):
> >       ...
> > 
> > This simplified syntax only allows names to specify decorators.  It
> > could allow multiple names, although I'm not sure it should,
> > 
> > I find this *far* more readable and obvious than any of the other syntaxs
> > I've seen propsed.
> 
> Agreed.

Disagreed.  It works fine when the decorator is as short as 
"classmethod" and the function signature is as short as "foo(cls, ...)".
It breaks down when those are long enough that the whole thing doesn't 
fit on a single line, which I'm expecting will happen a reasonably large 
fraction of the time.

I like pie.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From niemeyer at conectiva.com  Thu Aug  5 20:48:45 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Aug  5 20:46:30 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <20040805184845.GB29796@burma.localdomain>

Hello Ronald,

> I'm in favor of @decorators. It's easy to notice them when the are 
> present and they are clearly special. The '[decorator] def ...' and 

Why are they special? Why should they be more important than any other
part of the function definition?

> 'decorate(decorator) def ...' are very magic, and are IMHO unfriendly 
> to newbies (you must metion them in any introduction to python, because 
> otherwise users would complety mis the signicance of the decorations).
[...]
> 	@objc.signature("v@:@i")
> 	def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, 
> returnCode, contextInfo):
> 		pass
> 
> The argument to objc.signature is fairly magic and I do try to abstract 
> it away (such as with the introduction of objc.accessor), but that's 
> not always possible. If decorators were not allowed to have arguments 
> I'd have to introduce temporary functions that wouldn't help in 
> readability.

Is special syntax in the language really required in this case,
considering you're already doing something "fairly magic" anyways?

What is objc.signature() doing?

> The generic example PJE is introducing in PEAK also seem a good usecase 
> for decorators with arguments.

Any pointers?

I'd really like to "see the light" of complex decorators, as oposed
to KISS, and use the current language features to implement that.

-- 
Gustavo Niemeyer
http://niemeyer.net
From bob at redivi.com  Thu Aug  5 20:59:04 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Aug  5 20:59:13 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805184845.GB29796@burma.localdomain>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
Message-ID: <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>


On Aug 5, 2004, at 2:48 PM, Gustavo Niemeyer wrote:

> Hello Ronald,
>
>> I'm in favor of @decorators. It's easy to notice them when the are
>> present and they are clearly special. The '[decorator] def ...' and
>
> Why are they special? Why should they be more important than any other
> part of the function definition?

Because they take a function object as input and can do whatever they 
want with it and return something else.

>> 'decorate(decorator) def ...' are very magic, and are IMHO unfriendly
>> to newbies (you must metion them in any introduction to python, 
>> because
>> otherwise users would complety mis the signicance of the decorations).
> [...]
>> 	@objc.signature("v@:@i")
>> 	def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet,
>> returnCode, contextInfo):
>> 		pass
>>
>> The argument to objc.signature is fairly magic and I do try to 
>> abstract
>> it away (such as with the introduction of objc.accessor), but that's
>> not always possible. If decorators were not allowed to have arguments
>> I'd have to introduce temporary functions that wouldn't help in
>> readability.
>
> Is special syntax in the language really required in this case,
> considering you're already doing something "fairly magic" anyways?

The alternative would be (current syntax and current PyObjC) this:

def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, 
returnCode, contextInfo):
	pass
saveSheetDidDismiss_returnCode_contextInfo_ = 
objc.selector(saveSheetDidDismiss_returnCode_contextInfo_, 
signature='v@:@i')

Yes, we pretty much do need special syntax (without using a hack like 
PJE's).  Code like this is pretty commonplace in PyObjC projects.

>> def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet,

> What is objc.signature() doing?

objc.signature wraps the function object with an objc.selector that 
specifies specific return and argument types.  In this particular case, 
it declares that the selector 
saveSheetDidDismiss:returnCode:contextInfo: returns void and takes an 
object and an integer as arguments.  Without this, the selector can not 
be bridged correctly to the Objective C runtime and the program would 
crash.

The ctypes package behaves similarly and would use decorators for the 
same thing.  I imagine that other runtime/language bridges would also 
benefit from similar techniques (Jython, IronPython, Python.NET, 
JPython.. or whatever else).  I can also imagine it being used for 
things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar manner.

-bob
From niemeyer at conectiva.com  Thu Aug  5 21:05:49 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Aug  5 21:03:33 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805183444.GA29796@burma.localdomain>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
Message-ID: <20040805190549.GC29796@burma.localdomain>

> I've read that in his mail. Do you have something to add,
> besides personal insults, or just happen to have a lot of
> free time?

Michael Hudson asked me to calm down, so here I am. I don't
want to start a fight here, so I'm sorry if I took this too
strongly. I just was a little bit disappointed to receive
this kind of answer as the first comment to an email I sent
as a way to try to collaborate.

Unfortunately, everyone seems to be sick about this matter, and
I'm still disappointed with the introduced syntax. I just
wanted to be sure that such an important change won't be
a reason to regret soon, and that being sick of something
won't be a reason to introduce changes in the language
without extensive debate (even if it takes longer than what
is usually acceptable).

Is there something to be done that could change the current
decision? Voting? Collecting pros/cons and classifying them?

If there's nothing to be done, I'll just give up.

-- 
Gustavo Niemeyer
http://niemeyer.net
From ronaldoussoren at mac.com  Thu Aug  5 21:07:18 2004
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu Aug  5 21:07:53 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805184845.GB29796@burma.localdomain>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
Message-ID: <AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>


On 5-aug-04, at 20:48, Gustavo Niemeyer wrote:

> Hello Ronald,
>
>> I'm in favor of @decorators. It's easy to notice them when the are
>> present and they are clearly special. The '[decorator] def ...' and
>
> Why are they special? Why should they be more important than any other
> part of the function definition?
>
>> 'decorate(decorator) def ...' are very magic, and are IMHO unfriendly
>> to newbies (you must metion them in any introduction to python, 
>> because
>> otherwise users would complety mis the signicance of the decorations).
> [...]
>> 	@objc.signature("v@:@i")
>> 	def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet,
>> returnCode, contextInfo):
>> 		pass
>>
>> The argument to objc.signature is fairly magic and I do try to 
>> abstract
>> it away (such as with the introduction of objc.accessor), but that's
>> not always possible. If decorators were not allowed to have arguments
>> I'd have to introduce temporary functions that wouldn't help in
>> readability.
>
> Is special syntax in the language really required in this case,
> considering you're already doing something "fairly magic" anyways?
>
> What is objc.signature() doing?

The argument is fairly magic, in that most people wouldn't know how the 
interpret it. The function itself is easy enough: it creates a custom 
method object. The meta-class for Objective-C classes extracts the 
method signature from that method object and uses it build the right 
method description on the Objective-C side of the bridge.

A special syntax for descriptors is not stricly necessary, we are after 
all already using them with Python 2.2 and Python 2.3 code. It does 
make life a lot easier, the method names used in my example are not 
particularly long, I sometimes uses much longer names (because I have 
to [*]). Having 3 copies of such a name increases the likelyhood of 
errors, and makes it harder to spot that all 3 names are the same.

To stretch a point: class syntax is also not necessary, instead of a 
class definition you can call type() with the right arguments. Does 
that mean we should do away with classes?
>
>> The generic example PJE is introducing in PEAK also seem a good 
>> usecase
>> for decorators with arguments.
>
> Any pointers?
>
> I'd really like to "see the light" of complex decorators, as oposed
> to KISS, and use the current language features to implement that.

Whoops, I used the wrong word, I meant 'generic functions' instead of 
'generic example'. He's doing something like this:

@when("isinstance(db, OracleDB")
def storeInDB(db, object):
	pass

@when("isinstance(db, BerkelyDB")
def storeInDB(db, object):
	pass

The 'when' decorator converts the function into a generic function that 
uses the expressions to dispatch to the right function implementation 
at runtime.

Ronald

[*] "have too" because I haven't found a good way to automaticly 
convert Objective-C method names in sensible shorter names. Objective-C 
has segmented method names, like smalltalk. This looks a little like 
keyword arguments:

@interface MyModel : NSObject
{}
-(int)sheetDidEnd:(NSSheet*)sheet returnCode:(int)code 
contextInfo:(void*)context;
@end

This looks like a method with 2 keyword arguments, but really isn't. 
The method name is 'sheetDid:returnCode:contextInfo:'. I convert that 
to a python identifier by replacing colons by underscores. That's an 
easy to remember rule, which means I don't have to replicate the vast 
amount of Cocoa documentation, but you do end up with some ungainly 
method names.

The only way I see to fix this is to introduce segmented method in 
Python, but I don't think that would fly :-) :-)

From martin at v.loewis.de  Thu Aug  5 21:07:55 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 21:08:10 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805184845.GB29796@burma.localdomain>
References: <200408051636.i75Gaak03654@guido.python.org>	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
Message-ID: <4112858B.90406@v.loewis.de>

Gustavo Niemeyer wrote:
> Is special syntax in the language really required in this case,
> considering you're already doing something "fairly magic" anyways?

The syntax is not special for the specific application (Objective-C),
but special for a class of applications (modifications of/annotations
to the subsequent function).

> What is objc.signature() doing?

I'm only guessing here: it declares an Objective-C signature (i.e.
a type) of the function. This is probably necessary for thunking.

I can't see a way to do that in a comprehensible way without a
decorator. It applies to the function, and it probably needs to
wrap the function. So without a special syntax, it would have to
go after the function. This is counter-intuitive, as the Objective-C
signature has to correspond (I guess) with the Python signature,
so it is desirable that they are closely together.

It might be possible to put that annotation into the __doc__ string,
but that would be abusing doc strings, and might be more error-prone.

Regards,
Martin
From FBatista at uniFON.com.ar  Thu Aug  5 21:05:37 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Aug  5 21:09:50 2004
Subject: [Python-Dev] Call for defense of @decorators
Message-ID: <A128D751272CD411BC9200508BC2194D053C775F@escpl.tcp.com.ar>

[Gustavo Niemeyer]

#- Is there something to be done that could change the current
#- decision? Voting? Collecting pros/cons and classifying them?

Guido said it. All the people that doesn't like the actual state of things
could choose together an alernative syntax and propose it to him.

Then, I guess, he'll choose. Be aware that he wants good reasons, ;)

Anyway, I think that administering a voting/brainstorm to choose another
syntax is a good idea. Maybe in this list, maybe in another list, maybe on
IRC, as you like.

.	Facundo
From niemeyer at conectiva.com  Thu Aug  5 21:17:10 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Aug  5 21:14:56 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
Message-ID: <20040805191710.GD29796@burma.localdomain>

Hi Bob,

> >Why are they special? Why should they be more important than any other
> >part of the function definition?
> 
> Because they take a function object as input and can do whatever they 
> want with it and return something else.

This seems extremely powerful. OTOH, perl is also powerful.

> >>def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet,
> 
> >What is objc.signature() doing?
> 
> objc.signature wraps the function object with an objc.selector that 
> specifies specific return and argument types.  In this particular case, 
> it declares that the selector 
> saveSheetDidDismiss:returnCode:contextInfo: returns void and takes an 
> object and an integer as arguments.  Without this, the selector can not 
> be bridged correctly to the Objective C runtime and the program would 
> crash.

Isn't metaclass usage helpful in this case?

Or a perhaps a dictionary?

   __signature__ = {"funcname": "v@:@i"}

or

   def funcname(...):
      ...
   funcname.signature = "v@:@i"

and a postprocessor like:

objc.register(classname)

> The ctypes package behaves similarly and would use decorators for the 
> same thing.  I imagine that other runtime/language bridges would also 
> benefit from similar techniques (Jython, IronPython, Python.NET, 
> JPython.. or whatever else).  I can also imagine it being used for 
> things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar manner.

Is this something good? I mean, having function wrappers all
around the place? Wouldn't it be introducing unobvious code (magic?)
in packages which are working fine as they are?

Should I just throw away my t-shirt with the "zen of python" text? :-)

-- 
Gustavo Niemeyer
http://niemeyer.net
From ixokai at gmail.com  Thu Aug  5 21:24:14 2004
From: ixokai at gmail.com (IxokaI)
Date: Thu Aug  5 21:24:20 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
Message-ID: <f8ef9ae404080512245b3e62a0@mail.gmail.com>

So, I have been following Python-dev fairly closely ever since my
company decided to rewrite everything in Python, and are now very
interested in the future. Keeping track of the decorator debate has
been a bit of a pain, and with the outcry against @decorators now and
everyone throwing out the current proposals-- well, I thought maybe I
could help, since I was keeping notes anyways.

Someone said the PEP is out of date, and asked it to be updated with
Pro's and Cons of the various ideas out there. So I added what I have
to the Python Wiki; personally I think it'd be a good place for people
to edit in their pro-con thoughts. I left out all the proposals on the
PEP because they appear to all be rejected, although I don't remember
why. I just got through devouring the hundred or so messages on
@decorators made in the last week or so to find the various
information I appended.

http://www.python.org/moin/PythonDecorators

I added "with", although I havn't seen it. "using" would work too. I
personally do not like punctuation syntax-- I'd really like to see
something english, even with the hated addition of a keyword. I
understand Guido's disapproval for "as" so wanted to think of an
alternate.

--Stephen
From pje at telecommunity.com  Thu Aug  5 21:32:04 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug  5 21:28:01 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805191710.GD29796@burma.localdomain>
References: <8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
Message-ID: <5.1.1.6.0.20040805152516.02a983c0@mail.telecommunity.com>

At 04:17 PM 8/5/04 -0300, Gustavo Niemeyer wrote:
> > The ctypes package behaves similarly and would use decorators for the
> > same thing.  I imagine that other runtime/language bridges would also
> > benefit from similar techniques (Jython, IronPython, Python.NET,
> > JPython.. or whatever else).  I can also imagine it being used for
> > things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar manner.
>
>Is this something good? I mean, having function wrappers all
>around the place? Wouldn't it be introducing unobvious code (magic?)
>in packages which are working fine as they are?
>
>Should I just throw away my t-shirt with the "zen of python" text? :-)

This special syntax would be entirely unnecessary if Python function 
definitions (other than lambda) were expressions.  Apart from that 
syntactic issue, decorators are nothing new, and people already use them.

Is it better to bury the magic *after* the function body, or put it 
before?  In every other programming language I know of where functions are 
first class objects, one applies decorators  *before* the function body.

The advantage of a specialized decorator syntax over simply wrapping 
decorator calls around function bodies, is that it's possible to reduce the 
nestedness of the expression, e.g.:

     @binding.Make(uponAssembly=True)
     @events.taskFactory
     def some_method(self):
         ...

versus:

     binding.Make(events.taskFactory(
         def some_method(self):
             ...
         ),
         uponAssembly=True
     )

or today's:

     def some_method(self):
         ...
     some_method = 
binding.Make(events.TaskFactory(some_method),uponAssembly=True)

Given that '...' can be arbitrarily long, and "flat is better than nested", 
this looks like a net win for this style of programming.

Metaprogramming is an important part of Python, that has always been in 
Python and always should be.  Given that, it is better to have a visible 
syntax that precedes the function body, and this is the fundamental basis 
for the existence of PEP 318.




From ronaldoussoren at mac.com  Thu Aug  5 21:28:13 2004
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu Aug  5 21:28:34 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805191710.GD29796@burma.localdomain>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
Message-ID: <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>


On 5-aug-04, at 21:17, Gustavo Niemeyer wrote:

> Hi Bob,
>
>>> Why are they special? Why should they be more important than any 
>>> other
>>> part of the function definition?
>>
>> Because they take a function object as input and can do whatever they
>> want with it and return something else.
>
> This seems extremely powerful. OTOH, perl is also powerful.

They should IMHO be a part of the function definition because they are 
part of the specification of what the function will do. In the current 
situation decorators are not part of the function definition.

>
>>>> def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet,
>>
>>> What is objc.signature() doing?
>>
>> objc.signature wraps the function object with an objc.selector that
>> specifies specific return and argument types.  In this particular 
>> case,
>> it declares that the selector
>> saveSheetDidDismiss:returnCode:contextInfo: returns void and takes an
>> object and an integer as arguments.  Without this, the selector can 
>> not
>> be bridged correctly to the Objective C runtime and the program would
>> crash.
>
> Isn't metaclass usage helpful in this case?
>
> Or a perhaps a dictionary?
>
>    __signature__ = {"funcname": "v@:@i"}

This moves the signature away from the function definition, meaning you 
have to take care to keep them synchronized.
>
> or
>
>    def funcname(...):
>       ...
>    funcname.signature = "v@:@i"

That should be workable for this specific example. It wouldn't work for 
the objc.accessor example. The objc.accessor function/decorator deduces 
the right kind of signature from the name and arguments of the 
function.

>
> and a postprocessor like:
>
> objc.register(classname)

We already use a metaclass, which is working just fine ;)
>
>> The ctypes package behaves similarly and would use decorators for the
>> same thing.  I imagine that other runtime/language bridges would also
>> benefit from similar techniques (Jython, IronPython, Python.NET,
>> JPython.. or whatever else).  I can also imagine it being used for
>> things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar 
>> manner.
>
> Is this something good? I mean, having function wrappers all
> around the place? Wouldn't it be introducing unobvious code (magic?)
> in packages which are working fine as they are?

Nobody is saying anything about modifying existing modules. Decorators 
might make it easier to write objects that implement a COM or 
AppleScript interface, that doesn't mean we have to convert existing 
modules into COM objects.

>
> Should I just throw away my t-shirt with the "zen of python" text? :-)
Please don't, people need to be reminded of the zen sometimes :-)

Ronald

From bob at redivi.com  Thu Aug  5 21:28:58 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Aug  5 21:29:08 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805191710.GD29796@burma.localdomain>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
Message-ID: <B28EAFE2-E715-11D8-93D5-000A95686CD8@redivi.com>


On Aug 5, 2004, at 3:17 PM, Gustavo Niemeyer wrote:

> Hi Bob,
>
>>> Why are they special? Why should they be more important than any 
>>> other
>>> part of the function definition?
>>
>> Because they take a function object as input and can do whatever they
>> want with it and return something else.
>
> This seems extremely powerful. OTOH, perl is also powerful.

List comprehensions, generators, and generator expressions are also 
very powerful.  Let's get rid of those too.  What good is *args, 
**kwargs syntax if we can do it with apply?  Why do we need a syntax to 
build dictionaries if we can just dict() and then set all of the keys 
one at a time like in most other languages?

>>>> def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet,
>>
>>> What is objc.signature() doing?
>>
>> objc.signature wraps the function object with an objc.selector that
>> specifies specific return and argument types.  In this particular 
>> case,
>> it declares that the selector
>> saveSheetDidDismiss:returnCode:contextInfo: returns void and takes an
>> object and an integer as arguments.  Without this, the selector can 
>> not
>> be bridged correctly to the Objective C runtime and the program would
>> crash.
>
> Isn't metaclass usage helpful in this case?
>
> Or a perhaps a dictionary?
>
>    __signature__ = {"funcname": "v@:@i"}
>
> or
>
>    def funcname(...):
>       ...
>    funcname.signature = "v@:@i"
>
> and a postprocessor like:
>
> objc.register(classname)

The signature is tightly bound to the arguments that the function 
takes.  The farther you move it away from the arguments the easier it 
is to make a difficult to diagnose mistake (in the case of PyObjC, a 
segfault hopefully either upon call or return, but quite possibly some 
time later).  Metaclasses also do not always play very well with each 
other.  Decorators are much simpler than metaclasses in concept and 
implementation.

>> The ctypes package behaves similarly and would use decorators for the
>> same thing.  I imagine that other runtime/language bridges would also
>> benefit from similar techniques (Jython, IronPython, Python.NET,
>> JPython.. or whatever else).  I can also imagine it being used for
>> things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar 
>> manner.
>
> Is this something good? I mean, having function wrappers all
> around the place? Wouldn't it be introducing unobvious code (magic?)
> in packages which are working fine as they are?

I think you're mistaken if you think that all of these sorts of things 
work just great as-is.  It should be pretty obvious what a decorator 
with a sensible name does, just like when you subclass something or 
raise a particular kind of exception it should be pretty obvious what 
behavior that is going to have.  I know what "raise SystemExit" does, 
just like I know what @objc.signature(...) or @classmethod would do.

-bob
From colanderman at gmail.com  Thu Aug  5 21:31:40 2004
From: colanderman at gmail.com (Chris King)
Date: Thu Aug  5 21:31:41 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <875c7e07040805123136fe96de@mail.gmail.com>

On Thu, 5 Aug 2004 21:07:18 +0200, Ronald Oussoren
<ronaldoussoren@mac.com> wrote:
> 
> On 5-aug-04, at 20:48, Gustavo Niemeyer wrote:
> > What is objc.signature() doing?
> 
> The argument is fairly magic, in that most people wouldn't know how the
> interpret it. The function itself is easy enough: it creates a custom
> method object. The meta-class for Objective-C classes extracts the
> method signature from that method object and uses it build the right
> method description on the Objective-C side of the bridge.

Isn't this more a use case for function attributes, rather than
decorators?  Decorators seem like overkill in this case.
From guido at python.org  Thu Aug  5 21:32:15 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug  5 21:32:22 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: Your message of "Thu, 05 Aug 2004 14:41:38 EDT."
	<5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com> 
References: <Your message of "Thu, 05 Aug 2004 12:52:28 EDT."
	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com> 
	<5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com> 
Message-ID: <200408051932.i75JWFj04174@guido.python.org>

> At 10:14 AM 8/5/04 -0700, Guido van Rossum wrote:
> >[Phillip]
> > > Does this mean that the C#-style syntax has a chance if it's got a
> > > __future__?  :)
> >
> >I don't see how that would change the arguments against it.
> 
> I thought the primary argument against it was that it changes the meaning 
> of (potentially existing) Python code, and that you had rejected the "hard 
> to learn" argument on the basis that people learn by pattern-matching 
> rather than principle.

No, the reason I decided to drop that was was the ambiguity in
people's heads.

> I guess this is another reason to update the PEP...  :)

Indeed.

> >No, but I suggest that the proponents of syntax alternatives will
> >have to agree amongst themselves on a single alternative that they
> >can present to me.
> 
> I think that will pretty much guarantee that it's either @ or
> nothing: it appears that the two biggest (or at least most vocal)
> camps are:
> 
> 1. people who want a "simpler" syntax that doesn't support arguments (who 
> seem to mostly favor 'def classmethod foo()')

Tough beans.  They should have a look at how decorators are used in C#
and Java 1.5.

> 2. people who think that decorators without arguments are pointless, and 
> don't agree amongst themselves on the proper syntax, but don't necessarily 
> care that much as long as there *is* one.  (But there may be a slight 
> leaning towards either of the C#-inspired variants.)

So they should defend @ because it's there.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug  5 21:35:58 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug  5 21:36:05 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: Your message of "Thu, 05 Aug 2004 16:05:37 -0300."
	<A128D751272CD411BC9200508BC2194D053C775F@escpl.tcp.com.ar> 
References: <A128D751272CD411BC9200508BC2194D053C775F@escpl.tcp.com.ar> 
Message-ID: <200408051935.i75JZw304222@guido.python.org>

> Anyway, I think that administering a voting/brainstorm to choose another
> syntax is a good idea. Maybe in this list, maybe in another list, maybe on
> IRC, as you like.

Ah yes, IRC, the most democratic medium. :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)
From nidoizo at yahoo.com  Thu Aug  5 21:39:56 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Thu Aug  5 21:40:02 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
In-Reply-To: <f8ef9ae404080512245b3e62a0@mail.gmail.com>
References: <f8ef9ae404080512245b3e62a0@mail.gmail.com>
Message-ID: <ceu2ec$ugo$1@sea.gmane.org>

IxokaI wrote:
> I added "with", although I havn't seen it. "using" would work too. I
> personally do not like punctuation syntax-- I'd really like to see
> something english, even with the hated addition of a keyword. I
> understand Guido's disapproval for "as" so wanted to think of an
> alternate.

Is there ambiguities with the following? (I don't want to propose them, 
but I would like to at least add them to Wiki, even to specify why they 
don't work or why we hate them):

def decorator1(decoratorN(foo))(arg1, argN): pass
def decorator1(decoratorN(foo(arg1, argN))): pass
def(decorator1, decoratorN) foo(arg1, argN): pass

And is the [decorator] list syntax before or after the "def"?

Regards,
Nicolas

From tim.peters at gmail.com  Thu Aug  5 21:40:05 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug  5 21:40:08 2004
Subject: [Python-Dev] Re: Plea for simpler decorator syntax,
	in addition to pie-shaped syntax
In-Reply-To: <eppstein-206D17.11391005082004@sea.gmane.org>
References: <4110F829.2000609@zope.com>
	<20040805174619.GB27820@burma.localdomain>
	<eppstein-206D17.11391005082004@sea.gmane.org>
Message-ID: <1f7befae04080512401906a0db@mail.gmail.com>

[Gustavo Niemeyer]
>>> IMO, the most common uses of decorators will be to define properties,
>>> and class and static methods.  IMO, these uses would be better served
>>> by a simpler syntax:
>>>
>>>   def classmethod foo(cls, ...):
>>>       ...

I've seen this example several times today, and I have to say that
every time I've seen it, my unstoppable gut reaction was "wait, why
are they defining their own classmethod function here?!".  I've had
that problem since the first time this syntax vairant was suggested
(loooooong ago), and it's not going away.  Maybe it's 10+ years of "in
Python, the name of the function comes after the 'def'" and I just
can't adjust that to qualify "but is the *last* name after a 'def'
preceding the first left paren following the 'def'"; or maybe it's
because I've written God-only-knows how many tools that believe the
same thing (the Emacs python-mode parser; the IDLE parser; any number
of one-shot cheap-ass regexps).  Whatever, I can't get used to it.

So, sorry, but I like

@classmethod
def foo(cls, ...):

unboundedly better than that.  For that matter, I like it at least as
well as any alternative to date, and better than most.  I actively
hate putting stuff between 'the def' and the function name.

Then again, I'm old <wink>.
From pje at telecommunity.com  Thu Aug  5 21:45:35 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug  5 21:41:28 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out
  there?
In-Reply-To: <f8ef9ae404080512245b3e62a0@mail.gmail.com>
Message-ID: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com>

At 12:24 PM 8/5/04 -0700, IxokaI wrote:

>I added "with", although I havn't seen it.

Guido's reserving "with" for this purpose in some future Python:

     with x.y:
         .z = spam    # set x.y.z = spam
         print .q.r   # print x.y.q.r

I updated the Wiki page, and added another syntax, the original "def 
function() [decorator]:" syntax.  Interestingly, it looks like one of the 
options with the most pluses and fewest minuses of any syntax you've listed 
on that page; it's only ugly for long decorator definitions, and Guido said 
he didn't care if a syntax was ugly.  :)

From nas at arctrix.com  Thu Aug  5 21:45:23 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu Aug  5 21:45:27 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <20040805194523.GA20561@mems-exchange.org>

On Thu, Aug 05, 2004 at 09:28:13PM +0200, Ronald Oussoren wrote:
> On 5-aug-04, at 21:17, Gustavo Niemeyer wrote:
> >   def funcname(...):
> >      ...
> >   funcname.signature = "v@:@i"
> 
> That should be workable for this specific example.

Even nicer if '_' is bound to the last function defined.

    class SomeClass(objc):
        def funcname(...):
           ...
        _.signature = "v@:@i"

The objc metaclass could take the 'signature' function attribute and
transform the function.

> It wouldn't work for the objc.accessor example. The objc.accessor
> function/decorator deduces the right kind of signature from the
> name and arguments of the function.

Can't the metaclass do that?

  Neil
From ixokai at gmail.com  Thu Aug  5 21:45:42 2004
From: ixokai at gmail.com (IxokaI)
Date: Thu Aug  5 21:45:45 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
In-Reply-To: <ceu2ec$ugo$1@sea.gmane.org>
References: <f8ef9ae404080512245b3e62a0@mail.gmail.com>
	<ceu2ec$ugo$1@sea.gmane.org>
Message-ID: <f8ef9ae404080512456955d241@mail.gmail.com>

On Thu, 05 Aug 2004 15:39:56 -0400, Nicolas Fleury <nidoizo@yahoo.com> wrote:
> IxokaI wrote:
> > I added "with", although I havn't seen it. "using" would work too. I
> > personally do not like punctuation syntax-- I'd really like to see
> > something english, even with the hated addition of a keyword. I
> > understand Guido's disapproval for "as" so wanted to think of an
> > alternate.
> 
> Is there ambiguities with the following? (I don't want to propose them,
> but I would like to at least add them to Wiki, even to specify why they
> don't work or why we hate them):
> 
> def decorator1(decoratorN(foo))(arg1, argN): pass
> def decorator1(decoratorN(foo(arg1, argN))): pass
> def(decorator1, decoratorN) foo(arg1, argN): pass
> 

  Ack. I'd *never* use those syntaxes :) They are not ambigious, but
just... complicated. The function name is too hidden, and really, that
is in most cases the most important piece of information on that line.

> And is the [decorator] list syntax before or after the "def"?

  Sorry, i was refeering to the C# like before. I'll fix.

--Stephen
From barry at python.org  Thu Aug  5 21:47:28 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Aug  5 21:47:21 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805194523.GA20561@mems-exchange.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
	<20040805194523.GA20561@mems-exchange.org>
Message-ID: <1091735248.8540.81.camel@localhost>

On Thu, 2004-08-05 at 15:45, Neil Schemenauer wrote:

> Even nicer if '_' is bound to the last function defined.

Except, that doesn't play nicely with i18n/gettext. ;/

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/6971a015/attachment.pgp
From edreamleo at charter.net  Thu Aug  5 21:47:45 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Thu Aug  5 21:47:58 2004
Subject: [Python-Dev] About pep 318--Intro
Message-ID: <002a01c47b25$144a5a40$6700a8c0@computer>

I received the following from GvR in a private communication:

> If good arguments against @ are presented it may be removed in 2.4b1 (that
was always the plan but apparently not publicized).

I am going to take this as an invitation to discuss @ syntax again.  Given
the controversy, a reexamination of the major issues seems not only
reasonable, but desirable.

Please feel free to send me links to relevant discussion, but given the
apparent lack of consensus at present I would not like to be told to read
the entire py-dev archives.

> Please bring up the Leo issues in Python-dev.

I'll be posting on separate threads soon.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From niemeyer at conectiva.com  Thu Aug  5 21:52:01 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Aug  5 21:49:44 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <20040805195201.GE29796@burma.localdomain>

> >This seems extremely powerful. OTOH, perl is also powerful.
> 
> They should IMHO be a part of the function definition because they are 
> part of the specification of what the function will do. In the current 
> situation decorators are not part of the function definition.

Ok, I understand that for objc case, extra magic is needed to make
the system work. OTOH, is this kind of magic usual enough to
justify adding extra syntax to the language? Won't it encourage
usage of this facility abusively (as was just mentioned with SOAP,
XML-RPC, Jython, IronPython, ...), making it inherently less
readable?

For example, someone mentioned the following URL:

http://rafb.net/paste/results/bKBUem36.html

Which provides:

def magicprop(f): return property(*f())
class Foo(object):
    @magicprop
    def bar():
        def get(self):
            ...
        def set(self):
            ...
        return get, set

Notice that bar() is actually *called* inside the decorator.

> >Or a perhaps a dictionary?
> >
> >   __signature__ = {"funcname": "v@:@i"}
> 
> This moves the signature away from the function definition, meaning you 
> have to take care to keep them synchronized.

This might be spelled as:

class C:
    signature = {}

    signature["f"] = "@v:@p"
    def f(self):
        pass

> >or
> >
> >   def funcname(...):
> >      ...
> >   funcname.signature = "v@:@i"
> 
> That should be workable for this specific example. It wouldn't work for 
> the objc.accessor example. The objc.accessor function/decorator deduces 
> the right kind of signature from the name and arguments of the 
> function.

So why does it need to be run together with the function definition?

> >and a postprocessor like:
> >
> >objc.register(classname)
> 
> We already use a metaclass, which is working just fine ;)

Cool! Glad to know about another interesting metaclass usage.

[...]
> >>JPython.. or whatever else).  I can also imagine it being used for
> >>things like XML-RPC, SOAP, Apple Events, COM, etc. in a similar 
> >>manner.
> >
> >Is this something good? I mean, having function wrappers all
> >around the place? Wouldn't it be introducing unobvious code (magic?)
> >in packages which are working fine as they are?
> 
> Nobody is saying anything about modifying existing modules. Decorators 
> might make it easier to write objects that implement a COM or 
> AppleScript interface, that doesn't mean we have to convert existing 
> modules into COM objects.

Yes, that's what I was talking about. Current modules already work
fine, and I haven't seen usage cases that would benefit from the
complex (or, more powerful) decorator syntax in a way that couldn't
be easily spelled some other way.

-- 
Gustavo Niemeyer
http://niemeyer.net
From pje at telecommunity.com  Thu Aug  5 21:54:09 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug  5 21:50:02 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <875c7e07040805123136fe96de@mail.gmail.com>
References: <AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
	<200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <5.1.1.6.0.20040805155039.022d4cc0@mail.telecommunity.com>

At 03:31 PM 8/5/04 -0400, Chris King wrote:
>On Thu, 5 Aug 2004 21:07:18 +0200, Ronald Oussoren
><ronaldoussoren@mac.com> wrote:
> >
> > On 5-aug-04, at 20:48, Gustavo Niemeyer wrote:
> > > What is objc.signature() doing?
> >
> > The argument is fairly magic, in that most people wouldn't know how the
> > interpret it. The function itself is easy enough: it creates a custom
> > method object. The meta-class for Objective-C classes extracts the
> > method signature from that method object and uses it build the right
> > method description on the Objective-C side of the bridge.
>
>Isn't this more a use case for function attributes, rather than
>decorators?  Decorators seem like overkill in this case.

That argument has been done to death several times in the last year 
here.  Function attributes aren't a replacement for decorators.

From ronaldoussoren at mac.com  Thu Aug  5 21:54:17 2004
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu Aug  5 21:54:41 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805194523.GA20561@mems-exchange.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
	<20040805194523.GA20561@mems-exchange.org>
Message-ID: <3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com>


On 5-aug-04, at 21:45, Neil Schemenauer wrote:

> On Thu, Aug 05, 2004 at 09:28:13PM +0200, Ronald Oussoren wrote:
>> On 5-aug-04, at 21:17, Gustavo Niemeyer wrote:
>>>   def funcname(...):
>>>      ...
>>>   funcname.signature = "v@:@i"
>>
>> That should be workable for this specific example.
>
> Even nicer if '_' is bound to the last function defined.
>
>     class SomeClass(objc):
>         def funcname(...):
>            ...
>         _.signature = "v@:@i"
>
> The objc metaclass could take the 'signature' function attribute and
> transform the function.
>
>> It wouldn't work for the objc.accessor example. The objc.accessor
>> function/decorator deduces the right kind of signature from the
>> name and arguments of the function.
>
> Can't the metaclass do that?

Not every method is a property accessor. The metaclass does calculate a 
default signature (all arguments are objects and the result is an 
object) when it doesn't have better information (such as the signature 
of a method with the same name in a superclass). Sadly enough that 
isn't good enough for property setters. Some parts of Cocoa require 
that a setter returns 'void'.

I'm trying to push the automatic detection of the right signature as 
far as I can get to avoid having to use explicit annotations in user 
code, but sometimes you just cannot avoid adding annotations.

Ronald
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173

From ronaldoussoren at mac.com  Thu Aug  5 21:56:14 2004
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu Aug  5 21:56:41 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <5.1.1.6.0.20040805155039.022d4cc0@mail.telecommunity.com>
References: <AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
	<200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
	<5.1.1.6.0.20040805155039.022d4cc0@mail.telecommunity.com>
Message-ID: <8190C1A5-E719-11D8-89DE-000D93AD379E@mac.com>


On 5-aug-04, at 21:54, Phillip J. Eby wrote:
>
>> Isn't this more a use case for function attributes, rather than
>> decorators?  Decorators seem like overkill in this case.
>
> That argument has been done to death several times in the last year 
> here.  Function attributes aren't a replacement for decorators.

I couldn't agree more.

Ronald

From ronaldoussoren at mac.com  Thu Aug  5 21:59:19 2004
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu Aug  5 21:59:31 2004
Subject: [Python-Dev] About pep 318--Intro
In-Reply-To: <002a01c47b25$144a5a40$6700a8c0@computer>
References: <002a01c47b25$144a5a40$6700a8c0@computer>
Message-ID: <F01C66C8-E719-11D8-89DE-000D93AD379E@mac.com>


On 5-aug-04, at 21:47, Edward K. Ream wrote:

>
> Please feel free to send me links to relevant discussion, but given the
> apparent lack of consensus at present I would not like to be told to 
> read
> the entire py-dev archives.

Only the hunderds of messages about decorator syntax would do. That 
would
avoid rehashing the same points over and over and over again without 
adding
new information.

Ronald

From pje at telecommunity.com  Thu Aug  5 22:04:56 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug  5 22:00:49 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805194523.GA20561@mems-exchange.org>
References: <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
	<200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <5.1.1.6.0.20040805155650.02a99c60@mail.telecommunity.com>

At 03:45 PM 8/5/04 -0400, Neil Schemenauer wrote:
>On Thu, Aug 05, 2004 at 09:28:13PM +0200, Ronald Oussoren wrote:
> > On 5-aug-04, at 21:17, Gustavo Niemeyer wrote:
> > >   def funcname(...):
> > >      ...
> > >   funcname.signature = "v@:@i"
> >
> > That should be workable for this specific example.
>
>Even nicer if '_' is bound to the last function defined.
>
>     class SomeClass(objc):
>         def funcname(...):
>            ...
>         _.signature = "v@:@i"
>
>The objc metaclass could take the 'signature' function attribute and
>transform the function.
>
> > It wouldn't work for the objc.accessor example. The objc.accessor
> > function/decorator deduces the right kind of signature from the
> > name and arguments of the function.
>
>Can't the metaclass do that?

Arguing that decorators shouldn't exist isn't going to get much traction; 
the function attribute and metaclass objections have been argued to death 
numerous times previously.

If you have to raise these arguments again, please make a Wiki page so they 
can get answered at *most* once more, and then we can copy the answers from 
the Wiki into the PEP.  (This probably should have happened the first, 
second, or third times these arguments came around, and next time I will at 
least try to remember that the second time I answer the same argument I 
should darn well do so in either the PEP or the Wiki.)

From nidoizo at yahoo.com  Thu Aug  5 22:02:44 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Thu Aug  5 22:02:49 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
In-Reply-To: <f8ef9ae404080512456955d241@mail.gmail.com>
References: <f8ef9ae404080512245b3e62a0@mail.gmail.com>	<ceu2ec$ugo$1@sea.gmane.org>
	<f8ef9ae404080512456955d241@mail.gmail.com>
Message-ID: <ceu3p4$2ea$1@sea.gmane.org>

IxokaI wrote:
>   Ack. I'd *never* use those syntaxes :) They are not ambigious, but
> just... complicated. The function name is too hidden, and really, that
> is in most cases the most important piece of information on that line.

Ok, now I understand.
1.If the syntax is "before def", it is outside the definition.
2.If the syntax is "just after def keyword", it hides the function name.
3.If the syntax is "after definition", it cause ugly line wraps.

Maybe it's me, but I expect that me and beginners from C++ would use 
mostly use things like staticmethod and that makes me prefer #2 and #3 
solutions.

Regards,
Nicolas

From barry at python.org  Thu Aug  5 22:03:16 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Aug  5 22:03:07 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051932.i75JWFj04174@guido.python.org>
References: <Your message of "Thu, 05 Aug 2004 12:52:28 EDT."
	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com>
	<200408051932.i75JWFj04174@guido.python.org>
Message-ID: <1091736196.8540.91.camel@localhost>

On Thu, 2004-08-05 at 15:32, Guido van Rossum wrote:

> > 2. people who think that decorators without arguments are pointless, and 
> > don't agree amongst themselves on the proper syntax, but don't necessarily 
> > care that much as long as there *is* one.  (But there may be a slight 
> > leaning towards either of the C#-inspired variants.)
> 
> So they should defend @ because it's there.

I hate repeating myself, but I will anyway. :)

I'm in camp 2, but now that pie decorators are in, and I've had a chance
to convert my code to use them, I'm strongly +1 in favor of this
syntax.  It stands out nicely, and to me indicates a stronger affinity
to the def that follows it than the C# syntax.

I was never in favor of C# syntax, and I'm glad it wasn't chosen.  I
strongly disliked that it subtly changed the semantics of currently
valid Python.  I like that pie decorators code cannot run in older
Pythons, because if it /could/ it certainly wouldn't work.

'scuze-me-while-i-eat-the-pie-ly y'rs,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/af59ded4/attachment.pgp
From guido at python.org  Thu Aug  5 22:11:12 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug  5 22:11:18 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: Your message of "Thu, 05 Aug 2004 16:04:56 EDT."
	<5.1.1.6.0.20040805155650.02a99c60@mail.telecommunity.com> 
References: <9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
	<200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com> 
	<5.1.1.6.0.20040805155650.02a99c60@mail.telecommunity.com> 
Message-ID: <200408052011.i75KBCc04451@guido.python.org>

> If you have to raise these arguments again, please make a Wiki page
> so they can get answered at *most* once more, and then we can copy
> the answers from the Wiki into the PEP.  (This probably should have
> happened the first, second, or third times these arguments came
> around, and next time I will at least try to remember that the
> second time I answer the same argument I should darn well do so in
> either the PEP or the Wiki.)

Right.  That's what a PEP is *supposed* to do.  But we've been
slacking on the PEP...

--Guido van Rossum (home page: http://www.python.org/~guido/)
From nas at arctrix.com  Thu Aug  5 22:11:40 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu Aug  5 22:11:43 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
	<20040805194523.GA20561@mems-exchange.org>
	<3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <20040805201139.GA20768@mems-exchange.org>

On Thu, Aug 05, 2004 at 09:54:17PM +0200, Ronald Oussoren wrote:
> Not every method is a property accessor. The metaclass does calculate a 
> default signature (all arguments are objects and the result is an 
> object) when it doesn't have better information (such as the signature 
> of a method with the same name in a superclass).

Sorry I wasn't clearer.  What I meant was if we have a nice syntax
for setting function attributes then the metaclass to transform the
functions just like a decorator could.

  Neil  
From nidoizo at yahoo.com  Thu Aug  5 22:14:59 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Thu Aug  5 22:15:04 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
In-Reply-To: <ceu3p4$2ea$1@sea.gmane.org>
References: <f8ef9ae404080512245b3e62a0@mail.gmail.com>	<ceu2ec$ugo$1@sea.gmane.org>	<f8ef9ae404080512456955d241@mail.gmail.com>
	<ceu3p4$2ea$1@sea.gmane.org>
Message-ID: <ceu4g4$45d$1@sea.gmane.org>

Nicolas Fleury wrote:
> 1.If the syntax is "before def", it is outside the definition.
> 2.If the syntax is "just after def keyword", it hides the function name.
> 3.If the syntax is "after definition", it cause ugly line wraps.

I just discovered a #4 on the Wiki: "at beginning of function body 
syntax".  Best to me...

From ronaldoussoren at mac.com  Thu Aug  5 22:16:16 2004
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu Aug  5 22:16:35 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805201139.GA20768@mems-exchange.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
	<20040805194523.GA20561@mems-exchange.org>
	<3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com>
	<20040805201139.GA20768@mems-exchange.org>
Message-ID: <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com>


On 5-aug-04, at 22:11, Neil Schemenauer wrote:

> On Thu, Aug 05, 2004 at 09:54:17PM +0200, Ronald Oussoren wrote:
>> Not every method is a property accessor. The metaclass does calculate 
>> a
>> default signature (all arguments are objects and the result is an
>> object) when it doesn't have better information (such as the signature
>> of a method with the same name in a superclass).
>
> Sorry I wasn't clearer.  What I meant was if we have a nice syntax
> for setting function attributes then the metaclass to transform the
> functions just like a decorator could.

And how would that solve this: 
<http://www.eby-sarna.com/pipermail/peak/2004-July/001598.html>

Ronald

From niemeyer at conectiva.com  Thu Aug  5 22:22:45 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Aug  5 22:20:27 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <1091736196.8540.91.camel@localhost>
References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com>
	<200408051932.i75JWFj04174@guido.python.org>
	<1091736196.8540.91.camel@localhost>
Message-ID: <20040805202245.GH29796@burma.localdomain>

> > So they should defend @ because it's there.
> 
> I hate repeating myself, but I will anyway. :)

Thanks.

> I'm in camp 2, but now that pie decorators are in, and I've had a chance
> to convert my code to use them, I'm strongly +1 in favor of this
> syntax.  It stands out nicely, and to me indicates a stronger affinity
> to the def that follows it than the C# syntax.

I'm glad to see people happy with the current implementation.

> I was never in favor of C# syntax, and I'm glad it wasn't chosen.  I
> strongly disliked that it subtly changed the semantics of currently
> valid Python.  I like that pie decorators code cannot run in older
> Pythons, because if it /could/ it certainly wouldn't work.

I'm strongly against "I'm sick of it" and "I don't care [anymore]"
implemented features. Seeing everyone complaining and no positive
reactions to the current implementation certainly feels like
something is wrong.

-- 
Gustavo Niemeyer
http://niemeyer.net
From colanderman at gmail.com  Thu Aug  5 22:21:53 2004
From: colanderman at gmail.com (Chris King)
Date: Thu Aug  5 22:21:55 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
	<20040805194523.GA20561@mems-exchange.org>
	<3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com>
	<20040805201139.GA20768@mems-exchange.org>
	<4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <875c7e0704080513213cc4add0@mail.gmail.com>

On Thu, 5 Aug 2004 22:16:16 +0200, Ronald Oussoren
<ronaldoussoren@mac.com> wrote:

> And how would that solve this:
> <http://www.eby-sarna.com/pipermail/peak/2004-July/001598.html>

It wouldn't.  Pattern matching would, though.  But that's an argument
for another day ;) (Read: I don't know which side of that argument I'm
on)
From pf_moore at yahoo.co.uk  Thu Aug  5 22:23:28 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Thu Aug  5 22:23:24 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org>
Message-ID: <y8ktmj0f.fsf@yahoo.co.uk>

Guido van Rossum <guido@python.org> writes:

> It seems a public outcry against @decorators has started.  When I let
> Anthony check it in for 2.4a2, the plan was to see how it fares in a2
> and a3, and possibly take it out in 2.4b1.  If you don't want that to
> happen, you might want to say something in public in defense of
> @decorators.

OK then, they get +1 from me, FWIW.

I don't intend to try to drown out the outcry - there have been far
too many messages on this subject already. However, I will say this
much:

I believe that the functionality is worthwhile. Decorators add a new
type of expressiveness which will take a while to show its full
benefits. Until people get over the syntax issue, we won't see the
real benefits coming through - and that won't be classmethod and the
like, but more creative ideas that will take time to gel. Sure, we'll
see some abuses, but my feeling is that there will be some genuinely
useful decorators, which quite possibly would never have been
identified without the language support directing people's thoughts to
the idea of decorators.

All the arguments are over syntax, but in my view, syntax is not the
issue.

Paul.
-- 
Instant gratification takes too long -- Carrie Fisher

From edreamleo at charter.net  Thu Aug  5 22:32:59 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Thu Aug  5 22:33:08 2004
Subject: [Python-Dev] Questions about '@' in pep 318
Message-ID: <004a01c47b2b$65d19260$6700a8c0@computer>

The "mandate" for this post is the following from GvR:

> If good arguments against @ are presented it may be removed in 2.4b1 (that
was always the plan but apparently not publicized).

So I would like seriously to consider whether removing '@' might be a good
idea.

Note: my major concern with pep 318 has always been with syntax.  I am not
interested or qualified to discuss whether annotation in general is a good
idea...

My main questions:

1.  Is '@' syntax required to get the job done cleanly?

If the answer is clearly Yes, then consensus ought to be fairly easy, it
seems to me.

2.  If '@' is not required, why isn't it forbidden :-)

There have been various statements to the effect that "almost every
conceivable syntax alternative has been considered".  I'd like to know if
the following has been considered, and if so, what the objections to it
were/are.

Instead of:

@accepts(int, (int,float))

Do:

from __future__ import annotation as annotate # or a or whatever looks good.

annotate.accepts(int, (int,float))

Clearly, with enough work the Python compiler could recognize this.  Yes,
it's a special case, but so what?  This would avoid most problems with
reserved words or keywords.  And it would be in the spirit of letting
modules encapsulate most major features.  In other words, what ever happened
to namespaces?

3. How "reliable" are the following design goals from pep 318?

* be easy to type; programs that use it are expected to use it very
frequently

* not make it more difficult to scan through code quickly. It should still
be easy to search for all definitions, a particular definition, or the
arguments that a function accepts.

- If visibility of code is a concern, use import annotation as XXXXX.

- If brevity is wanted, use import annotation as a.

I wonder why these design goals are even present.

4. Assuming that there are advantages to '@', why do they outweigh the
disadvantages of changing the following paragraph of the Reference Manual?

http://docs.python.org/ref/delimiters.html

[quote]
The following printing ASCII characters are not used in Python. Their
occurrence outside string literals and comments is an unconditional error:
@       $       ?
[end quote]

I regard any change to this paragraph as a major change to Python, in spite
of its innocent appearance.   Indeed, this paragraph is an excellent "line
in the sand" for other tools.  The pep contains the following design goal:

* not needlessly complicate secondary support tools such as
language-sensitive editors and other "toy parser tools out there.

How is this not going to happen with the '@' syntax?

All answers to these questions would be most appreciated.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From pje at telecommunity.com  Thu Aug  5 22:48:09 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug  5 22:44:04 2004
Subject: [Python-Dev] A usability argument for list-after-def
Message-ID: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>

Guido has stated he'll accept one community syntax proposal as an 
alternative to "@".  He's also ruled out his own previous proposal 
(list-before-def), and a variety of new-keyword alternatives involving such 
things as 'as' and 'with'.

There doesn't appear to be a downside for decorator supporters to put forth 
another syntax, if they like it better than "@"; it seems the worst case is 
that we still end up with "@", if we end up with any syntax at all.

Of the options on the Wiki page ( at 
http://www.python.org/moin/PythonDecorators ), this seems to leave only 
list-after-def, the previous community favorite and the first PEP 318 
syntax to have been implemented.

Guido's principal argument against list-after-def, if I recall correctly, 
was that it is ugly when multiple or lengthy decorators are involved.  But, 
"ugly" isn't an argument any more, so that shouldn't rule out 
list-after-def.  :)

I only have one usability argument to favor list-after-def over "@": the 
former looks more like executable pseudocode than the latter.  If I pretend 
that I do not know Python, that I am a programmer who has never written in 
Python and is reading a Python program found in a magazine or downloaded 
from somewhere, then if I look at:

     @events.taskFactory
     def monitorProcess(self,process):
         ...

It is not immediately obvious that the previous line is part of the 
function definition; I am put off reading this code, because how do I know 
what "@" does?

Brackets are normally used in English to indicate an annotation or "aside" 
of some sort; such as a commentary *about* the surrounding material.  When 
I see this:

     def monitorProcess(self,process) [events.taskFactory]:
         ...

My impression is that the decorator is somehow annotating or subscripting 
the function.  In this specific example, I might wonder if this is how 
Python specifies a function return type.  But if I see:

     def do_something(cls,data) [classmethod]:
         ...

I am quickly alerted to this being some kind of annotation about the method 
itself, probably akin to public/private/static/friend etc. in other 
languages.  And, if I look for the definition of 'events.taskFactory' or 
'classmethod', I will quickly realize that this is an extensible annotation 
facility, as well.  By contrast:

     @classmethod
     def do_something(cls,data):
         ...

is also suggestive, but I personally find it far less conclusive, even with 
'classmethod' as a semantic cue.  Drawing on my experience of other 
languages and typical uses of "@", I think:

   * Is this some sort of array thing? (Perl)

   * Is this some sort of documentation tag? (Java...  but then I think no, 
it's not in a comment, so that can't be it)

   * Is this some sort of escape character for literate programming? (e.g. 
FunnelWeb and other such tools)

   * Is it being used in place of the word "at"?  (But what does it mean to 
be "at" a class method?  It's on a different line; surely if it were 
related there would be some sort of syntactic indication like indentation 
or a colon or braces or a comma or *something* between it and the definition?)

After some weighing of the evidence, I think I would most likely guess that 
this was some sort of documentation markup, rather than semantic 
markup.  But my train of thought in reading the code has now been quite 
disrupted, and if I was reading this code casually I might be discouraged 
from continuing, unless I had some reason for wanting to learn Python 
besides just reading the code in question or hacking up a quick fix to 
something.

Now, I don't know if my thought process here is at all typical or 
representative of anything, other than my own idiosyncratic self.  But, it 
is the one nagging thing that I have against "@", and I suspect that it may 
be implicitly behind a lot of the general outcry against it.  In other 
words, I think that "ugly" is just a pejorative term for the emotional 
reaction engendered by being forced to go through the above mental process 
for figuring it out.  And, to a lesser extent, I think list-before-def 
required some of the same mental processing overhead, although I personally 
think it was somewhat less than that of "@".

So there it is, my personal argument for list-after-def as better than "@", 
even if the list is over multiple lines: it's still much more clearly part 
of the function definition, and that IMO is the most important thing the 
syntax should convey, apart from listing the decorators themselves.

From tjreedy at udel.edu  Thu Aug  5 22:48:06 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu Aug  5 22:48:12 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org><FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com><20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
Message-ID: <ceu6e8$8sj$1@sea.gmane.org>

"Ronald Oussoren"
>My particular use-case is PyObjC, which sometimes suffers from haveing
>veryLong_andSometimesAlsoVeryUgly_methodNames_. Having to repeat those
>three times when using a decorator is no fun.

I did not get this at first (see below).  I believe you mean more
specifically "Having to repeat those three times when using the old
no-special-deco-syntax post call is no fun."

"Bob Ippolito" followed with
> The alternative would be (current syntax and current PyObjC) this:
>
> def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet,
> returnCode, contextInfo):
> pass
> saveSheetDidDismiss_returnCode_contextInfo_ =
> objc.selector(saveSheetDidDismiss_returnCode_contextInfo_,
> signature='v@:@i')

Aha!  Now I get it (perhaps for the second time in an extended discussion).

One argument for extended assignment was and is that not writing names,
especially long names, twice makes for easier reading and fewer bugs.  The
argument for not writing names *three* times, is even stronger.  If this
rationale against the status quo (which many objectors on c.l.py favor) is
not currently in the pep, I think it should be.

Terry J. Reedy





From edreamleo at charter.net  Thu Aug  5 22:48:48 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Thu Aug  5 22:49:02 2004
Subject: [Python-Dev] The impact of '@' on Leo
Message-ID: <005e01c47b2d$9c0dee80$6700a8c0@computer>

The mandate for this post is the following from GvR:

> Please bring up the Leo issues in Python-dev.

The major concern I have with '@' syntax as it relates to Leo is the
potential for ongoing and never-ending conflicts between Leo's directives
and @x constructs in Python.

Leo presently supports the following directives:

@ (followed by one or more whitespace characters)
@all, @asis,
@c, @code, @color, @comment,
@delims, @doc,
@encoding, @end_raw,
@file, @first,
@header,
@ignore,
@killcolor,
@language, @last, @lineending,
@nocolor, @noheader, @noref, @nosent, @nowrap,
@others,
@pagewidth, @path,
@quiet,
@raw,
@root, @root-code, @root-doc,
@silent,
@tabwidth, @terse, @thin,
@unit,
@verbose,
@wrap

Worse, Leo allows plugins to define their own directives.  There is already
a plugin that defines @wiki.  Furthermore, @run, @test and @suite are also
used in special contexts.

Leo's users could probably live with a small, fixed set of @x constructs in
Python.  But if the Python programmer can create new @x identifiers then
they have a big, big problem.  What happens when another Python @x construct
conflicts with one of these directives or some other directive?

These directives act as a markup meta-language.  Many of these directives
are found interspersed in the code: a conflict with a construct in Python
would have most unhappy consequences.

To make matters worse, one of the _best_ choices I ever made in Leo was to
remove all "escape" conventions regarding '@' (and other noweb constructs).
This took a bit of courage, but it has turned out to be clearly the right
choice.  But if Python can now generate arbitrary @x strings Leo will have
to do something really drastic.

There are a number of markup languages that use '@' to from the underlying
language (Python) to the markup language.  I was always under the impression
that the clear statement in the Reference Manual that at signs are invalid
everywhere (except in comments and strings) was a clear signal of an
intention to keep those symbols for other valid purposes.  I do hope this
won't change.

Edward

P.S.  I recently wrote a pretty-printer for Python using the tokenize
module.  What will be the status of the '@' token?  Is it still an
"errortoken" ?  Sure, this is a minor point, but it _will_ affect other
tools.

EKR
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From martin at v.loewis.de  Thu Aug  5 22:49:21 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 22:49:26 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <8190C1A5-E719-11D8-89DE-000D93AD379E@mac.com>
References: <AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>	<200408051636.i75Gaak03654@guido.python.org>	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>	<20040805184845.GB29796@burma.localdomain>	<AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>	<5.1.1.6.0.20040805155039.022d4cc0@mail.telecommunity.com>
	<8190C1A5-E719-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <41129D51.6000300@v.loewis.de>

Ronald Oussoren wrote:

>> That argument has been done to death several times in the last year 
>> here.  Function attributes aren't a replacement for decorators.
> 
> 
> I couldn't agree more.

While I agree with that statement, I think it was unhelpful. Chris
did not want to know whether function attributes can, in general,
replace decorators. Instead, he wanted to know whether the *specific*
decorator could have been replaced with a function attribute.

I don't really know the answer to that question, but my guess is:
It could have.

Regards,
Martin
From nas at arctrix.com  Thu Aug  5 22:53:16 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu Aug  5 22:53:19 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com>
References: <200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>
	<20040805191710.GD29796@burma.localdomain>
	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
	<20040805194523.GA20561@mems-exchange.org>
	<3C1C962D-E719-11D8-89DE-000D93AD379E@mac.com>
	<20040805201139.GA20768@mems-exchange.org>
	<4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <20040805205316.GA20997@mems-exchange.org>

On Thu, Aug 05, 2004 at 10:16:16PM +0200, Ronald Oussoren wrote:
> On 5-aug-04, at 22:11, Neil Schemenauer wrote:
> >What I meant was if we have a nice syntax for setting function
> >attributes then the metaclass to transform the functions just
> >like a decorator could.
> 
> And how would that solve this: 
> <http://www.eby-sarna.com/pipermail/peak/2004-July/001598.html>

Is that supposed to be a good example how to use decorators?
Anyhow, obviously a metaclass won't work for module level functions
(unless you had a way of setting the metaclass of the module).

  Neil
From martin at v.loewis.de  Thu Aug  5 22:53:28 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 22:53:31 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <875c7e07040805123136fe96de@mail.gmail.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>	<20040805184845.GB29796@burma.localdomain>	<AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
	<875c7e07040805123136fe96de@mail.gmail.com>
Message-ID: <41129E48.7050607@v.loewis.de>

Chris King wrote:
> Isn't this more a use case for function attributes, rather than
> decorators?  Decorators seem like overkill in this case.

I don't fully understand the issues, but I believe that the
declaration is more like a statement than a mere declaration
(very similar to all other "declarations" in Python).

In the specific case, I believe declaring the Objective-C
signature also performs some kind of registration of the
function with the Objective-C run-time. This cannot be done
with function attributes; you would still need to invoke
a do_register()/generate_wrappers() functions at some point.

Even if this is a mere declaration, function attributes would
harm readability: they would have to go after the function, whereas
you want the Objective-C signature close to the Python signature.

Regards,
Martin
From niemeyer at conectiva.com  Thu Aug  5 22:55:59 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Aug  5 22:54:02 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <004a01c47b2b$65d19260$6700a8c0@computer>
References: <004a01c47b2b$65d19260$6700a8c0@computer>
Message-ID: <20040805205559.GA31288@burma.localdomain>

> The "mandate" for this post is the following from GvR:
> 
> > If good arguments against @ are presented it may be removed in 2.4b1
> > (that was always the plan but apparently not publicized).
> 
> So I would like seriously to consider whether removing '@' might be a
> good idea.

I'd be glad to find a good way to do so.

The point is that people agree to disagree with most proposals (and
I'm no exception here).

> Note: my major concern with pep 318 has always been with syntax.  I am
> not interested or qualified to discuss whether annotation in general
> is a good idea...

I'm concerned about both, but there're people with good usage cases
for it, and it has been decided that decorators are going in, so
syntax is what is left to be discussed.

> My main questions:
> 
> 1.  Is '@' syntax required to get the job done cleanly?
> 
> If the answer is clearly Yes, then consensus ought to be fairly easy,
> it seems to me.

Unfortunately, it's not.

> 2.  If '@' is not required, why isn't it forbidden :-)

Because it's not clearly "no". :-)

[...]
> Instead of:
> 
> @accepts(int, (int,float))
> 
> Do:
> 
> from __future__ import annotation as annotate # or a or whatever looks
> good.
> 
> annotate.accepts(int, (int,float))

I don't understand. How are you changing/annotating the function?

[...]
> - If visibility of code is a concern, use import annotation as XXXXX.
> 
> - If brevity is wanted, use import annotation as a.
> 
> I wonder why these design goals are even present.

We're unable to agree into more obvious issues, I belive it'll be
mostly impossible to agree in such highly subjective issues.

> 4. Assuming that there are advantages to '@', why do they outweigh the
> disadvantages of changing the following paragraph of the Reference
> Manual?
> 
> http://docs.python.org/ref/delimiters.html
> 
> [quote]
> The following printing ASCII characters are not used in Python. Their
> occurrence outside string literals and comments is an unconditional error:
> @       $       ?
> [end quote]

Good point. This paragraph should be changed, if it wasn't yet.

> I regard any change to this paragraph as a major change to Python, in
> spite of its innocent appearance.   Indeed, this paragraph is an
> excellent "line in the sand" for other tools.  The pep contains the
> following design goal:
> 
> * not needlessly complicate secondary support tools such as
> language-sensitive editors and other "toy parser tools out there.
> 
> How is this not going to happen with the '@' syntax?

Every tool which deals with Python's syntax must be adapted
to the new syntax, no matter which new syntax is adopted.

-- 
Gustavo Niemeyer
http://niemeyer.net
From martin at v.loewis.de  Thu Aug  5 23:01:13 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 23:01:21 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805202245.GH29796@burma.localdomain>
References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>	<5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com>	<200408051932.i75JWFj04174@guido.python.org>	<1091736196.8540.91.camel@localhost>
	<20040805202245.GH29796@burma.localdomain>
Message-ID: <4112A019.3080000@v.loewis.de>

Gustavo Niemeyer wrote:
> I'm strongly against "I'm sick of it" and "I don't care [anymore]"
> implemented features. Seeing everyone complaining and no positive
> reactions to the current implementation certainly feels like
> something is wrong.

Well, I am happy that some sort of syntax was implemented. I don't
consider syntax enormously important, so it does not matter much to
me which syntax is chosen.

Regards,
Martin

From colanderman at gmail.com  Thu Aug  5 23:01:38 2004
From: colanderman at gmail.com (Chris King)
Date: Thu Aug  5 23:01:43 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <41129E48.7050607@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>	<20040805184845.GB29796@burma.localdomain>	<AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
	<875c7e07040805123136fe96de@mail.gmail.com>
	<41129E48.7050607@v.loewis.de>
Message-ID: <875c7e07040805140174141409@mail.gmail.com>

On Thu, 05 Aug 2004 22:53:28 +0200, "Martin v. L?wis"
<martin@v.loewis.de> wrote:
> Chris King wrote:
> > Isn't this more a use case for function attributes, rather than
> > decorators?  Decorators seem like overkill in this case.
> 
> In the specific case, I believe declaring the Objective-C
> signature also performs some kind of registration of the
> function with the Objective-C run-time. This cannot be done
> with function attributes; you would still need to invoke
> a do_register()/generate_wrappers() functions at some point.

I held off posting that at first because of this (since I am not at
all familiar with PyObjC), but then Ronald mentioned something about
the registration and processing being done via a metaclass, rather
than by the decorators themselves.

> Even if this is a mere declaration, function attributes would
> harm readability: they would have to go after the function, whereas
> you want the Objective-C signature close to the Python signature.

I should have been more specific -- I meant function attributes of
this form (proposed and subsequently rejected somewhere else a while
ago):

def foo(a,b,c):
    .signature = 'v@:@i'
    pass

I only bring this up because most (though I realize not all) use cases
for decorators (at least in their current form) seem to involve some
type of attribute or metadata.
From tjreedy at udel.edu  Thu Aug  5 23:03:17 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu Aug  5 23:03:24 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
References: <f8ef9ae404080512245b3e62a0@mail.gmail.com>
	<ceu2ec$ugo$1@sea.gmane.org>
Message-ID: <ceu7an$b2m$1@sea.gmane.org>


"Nicolas Fleury" <nidoizo@yahoo.com> wrote in message
news:ceu2ec$ugo$1@sea.gmane.org...
> IxokaI wrote:
> Is there ambiguities with the following?
>
> def decorator1(decoratorN(foo))(arg1, argN): pass
> def decorator1(decoratorN(foo(arg1, argN))): pass
> def(decorator1, decoratorN) foo(arg1, argN): pass

If you name the decos decorator1, etc, not really ambigous, mearly hard to
find the foo in the soup.  But try

def bar(baz(foo))(a1,aN): pass

and it becomes easily ambigous.  Anyway, by current Python syntax,
decoN(foo) look like a function call on an *existing* object foo.  The
current situation is that name(arg) is always a func call except when name
*immediately* follows def.  Like Tim, I really don't want that to change.

Terry J. Reedy



From barry at python.org  Thu Aug  5 23:06:51 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Aug  5 23:06:44 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805202245.GH29796@burma.localdomain>
References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com>
	<200408051932.i75JWFj04174@guido.python.org>
	<1091736196.8540.91.camel@localhost>
	<20040805202245.GH29796@burma.localdomain>
Message-ID: <1091740011.8540.130.camel@localhost>

On Thu, 2004-08-05 at 16:22, Gustavo Niemeyer wrote:

> I'm strongly against "I'm sick of it" and "I don't care [anymore]"
> implemented features. Seeing everyone complaining and no positive
> reactions to the current implementation certainly feels like
> something is wrong.

Let me be clear.  I want decorator support in Python 2.4.  I would have
accepted any of the syntaxes that made Guido's final cut, and I trust
Guido to do the right thing.  By choosing pie decorators, I think he
has, and I will happily use them.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/74c8d9d5/attachment.pgp
From martin at v.loewis.de  Thu Aug  5 23:07:25 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 23:07:35 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805190549.GC29796@burma.localdomain>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
Message-ID: <4112A18D.5090801@v.loewis.de>

Gustavo Niemeyer wrote:
> Is there something to be done that could change the current
> decision? Voting? Collecting pros/cons and classifying them?

Depends on in what direction you want to make a change. It
appears you want to avoid introducing any kind of syntax
change. In that case, you should explain people how to spell
classmethod and synchronized in a more convenient way, because
that is what the stated motivation of PEP 318 is - you would
have to explain why this motive is bad, irrelevant, or otherwise
not a worthy goal.

Or you could argue on a procedural basis: regardless of whether
the feature is good or bad, the current implementation is
unacceptable, as the PEP does not correspond with the
implementation, the syntax is undocumented, the code has no test
cases, and so on. I'm actually going to do that, because I do
think the process is unacceptable, and should be either corrected
or reversed (of course, this says nothing about the feature itself,
or the code implementing it).

Regards,
Martin
From pje at telecommunity.com  Thu Aug  5 23:14:45 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug  5 23:10:37 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <875c7e07040805140174141409@mail.gmail.com>
References: <41129E48.7050607@v.loewis.de>
	<200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
	<875c7e07040805123136fe96de@mail.gmail.com>
	<41129E48.7050607@v.loewis.de>
Message-ID: <5.1.1.6.0.20040805171011.02a662e0@mail.telecommunity.com>

At 05:01 PM 8/5/04 -0400, Chris King wrote:
>I only bring this up because most (though I realize not all) use cases
>for decorators (at least in their current form) seem to involve some
>type of attribute or metadata.

They may seem to, but that's only because most decorator proponents mostly 
refrain from using their current decorators in examples, for the sake of 
brevity.  Decorators in PEAK, for example, create specialized descriptors 
for managing things such as asynchronous tasks, lazy attribute 
initialization, wrapping methods in transactions, automatic type 
conversions on assignment to attributes, etc.  They're useful enough to 
motivate usage even without a decorator syntax, but they have little to do 
with "metadata" in the usual sense, since they *implement* what is 
specified by the "metadata", rather than simply holding the metadata for 
some class or other object to read and interpret.

From bac at OCF.Berkeley.EDU  Thu Aug  5 23:15:17 2004
From: bac at OCF.Berkeley.EDU (Brett Cannon)
Date: Thu Aug  5 23:15:17 2004
Subject: [Python-Dev] OK to lower timeout for test_timeout's
	testConnectTimeout test?
Message-ID: <4112A365.1000203@ocf.berkeley.edu>

I am actually getting failures from test_timeout's testConnectTimeout 
since the Net connection I have at school lets me connect to Google in 
under the .001 connection timeout value.  If I push it to .00000001 (a 
hundred-millionth of a second) it then fails consistently.

Now the problem is that the second part of the test uses this and a fuzz 
value (currently at 2) to see if the test timed out within the proper 
amount of time.  The comparison is basically if the amount of time it 
took to do the timed out failure is less than timeout + fuzz.  So 
lowering this number could possibly affect the test, although at .001 
seconds, I am doubting that will occur.  But since these types of timing 
tests can be touchy I thought I would double-check.

So if anyone thinks it is bad to lower the value to .00000001 then 
please let me know.  Otherwise I will lower the value before the next 
alpha goes out.

-Brett

P.S.: the only other failures I have on OS X right now is test_curses 
(because I used -uall) and test__locale .  Should we disable 
test__locale on OS X to shut it up since Martin seems to think the test 
isn't all that useful and won't work for OS X ever?
From fdrake at acm.org  Thu Aug  5 23:16:09 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Aug  5 23:16:27 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <20040805205559.GA31288@burma.localdomain>
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<20040805205559.GA31288@burma.localdomain>
Message-ID: <200408051716.09292.fdrake@acm.org>

On Thursday 05 August 2004 04:55 pm, Gustavo Niemeyer wrote:
 > Good point. This paragraph should be changed, if it wasn't yet.

Now changed in CVS.

 > Every tool which deals with Python's syntax must be adapted
 > to the new syntax, no matter which new syntax is adopted.

I expect many tools won't be affected by @decorations.  Import checkers and 
function/method identifying tools probably won't care about the decorations.  
In the end, it very much depends on the assumptions of the tool and how it 
implements scanning and parsing.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From barry at python.org  Thu Aug  5 23:19:53 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Aug  5 23:19:44 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112A18D.5090801@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
Message-ID: <1091740792.8546.133.camel@localhost>

On Thu, 2004-08-05 at 17:07, "Martin v. L?wis" wrote:

> Or you could argue on a procedural basis: regardless of whether
> the feature is good or bad, the current implementation is
> unacceptable, as the PEP does not correspond with the
> implementation, the syntax is undocumented, the code has no test
> cases, and so on. I'm actually going to do that, because I do
> think the process is unacceptable, and should be either corrected
> or reversed (of course, this says nothing about the feature itself,
> or the code implementing it).

Martin makes a good point.  Guido could threaten to remove the feature
by beta 1 (and thus for 2.4 final) if the PEP is not brought up to date.

not-that-i'm-volunteering-'cause-i'd-rather-see-it-done-ly y'rs,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/e8230d4a/attachment.pgp
From martin at v.loewis.de  Thu Aug  5 23:20:57 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 23:21:00 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <004a01c47b2b$65d19260$6700a8c0@computer>
References: <004a01c47b2b$65d19260$6700a8c0@computer>
Message-ID: <4112A4B9.4010907@v.loewis.de>

Edward K. Ream wrote:
> 2.  If '@' is not required, why isn't it forbidden :-)

Because it is better than all alternatives. To see that, you would have
to produce a list of all alternatives, and compare them one-by-one.

> Instead of:
> 
> @accepts(int, (int,float))
> 
> Do:
> 
> from __future__ import annotation as annotate # or a or whatever looks good.
> 
> annotate.accepts(int, (int,float))

That requires all annotations to be in the __future__ module. This is
not acceptable, as there is a desire to provide user-defined annotations
as well.

> * be easy to type; programs that use it are expected to use it very
> frequently
> 
> * not make it more difficult to scan through code quickly. It should still
> be easy to search for all definitions, a particular definition, or the
> arguments that a function accepts.
> 
> - If visibility of code is a concern, use import annotation as XXXXX.
> 
> - If brevity is wanted, use import annotation as a.
> 
> I wonder why these design goals are even present.

The primary application for this is staticmethod, classmethod, and
an imaginary synchronized declarator. They need to be easy to type
because that is the intended improvement over the status quo, where
you need to write

    foo = staticmethod(foo)

after the function. They also should not be in the way of reading
the function proper, hence the second requirement.

> 4. Assuming that there are advantages to '@', why do they outweigh the
> disadvantages of changing the following paragraph of the Reference Manual?
> 
> http://docs.python.org/ref/delimiters.html
> 
> [quote]
> The following printing ASCII characters are not used in Python. Their
> occurrence outside string literals and comments is an unconditional error:
> @       $       ?
> [end quote]

I believe @ was chosen precisely for that reason - it is a character
that is not yet used, so it cannot be confused with something else.

> I regard any change to this paragraph as a major change to Python, in spite
> of its innocent appearance.

No doubt about that. This is a significant change.

> * not needlessly complicate secondary support tools such as
> language-sensitive editors and other "toy parser tools out there.
> 
> How is this not going to happen with the '@' syntax?

It's of course hard to tell how tools will react to that syntax, or
with any other syntax change, for that matter. This specific is likely
easy to implement, because it likely has only minimal impact on people's
parsers: one can essentially treat a line starting with @ like a
statement, e.g. by making @ a keyword. Tools that only look for parts
of a Python code are likely interested in lines starting with a proper
Python keyword (e.g. def, class, import) - those tools are not affected
at all.

Regards,
Martin
From bob at redivi.com  Thu Aug  5 23:23:07 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Aug  5 23:23:21 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <1091740011.8540.130.camel@localhost>
References: <5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<5.1.1.6.0.20040805124817.029142e0@mail.telecommunity.com>
	<5.1.1.6.0.20040805142708.0335a7a0@mail.telecommunity.com>
	<200408051932.i75JWFj04174@guido.python.org>
	<1091736196.8540.91.camel@localhost>
	<20040805202245.GH29796@burma.localdomain>
	<1091740011.8540.130.camel@localhost>
Message-ID: <A5349945-E725-11D8-9D9A-000A95686CD8@redivi.com>


On Aug 5, 2004, at 5:06 PM, Barry Warsaw wrote:

> On Thu, 2004-08-05 at 16:22, Gustavo Niemeyer wrote:
>
>> I'm strongly against "I'm sick of it" and "I don't care [anymore]"
>> implemented features. Seeing everyone complaining and no positive
>> reactions to the current implementation certainly feels like
>> something is wrong.
>
> Let me be clear.  I want decorator support in Python 2.4.  I would have
> accepted any of the syntaxes that made Guido's final cut, and I trust
> Guido to do the right thing.  By choosing pie decorators, I think he
> has, and I will happily use them.

My sentiments exactly.

-bob

From martin at v.loewis.de  Thu Aug  5 23:24:35 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 23:24:38 2004
Subject: [Python-Dev] The impact of '@' on Leo
In-Reply-To: <005e01c47b2d$9c0dee80$6700a8c0@computer>
References: <005e01c47b2d$9c0dee80$6700a8c0@computer>
Message-ID: <4112A593.7080608@v.loewis.de>

Edward K. Ream wrote:
> The major concern I have with '@' syntax as it relates to Leo is the
> potential for ongoing and never-ending conflicts between Leo's directives
> and @x constructs in Python.

Can you give an example of a file that would be difficult to understand
under that change? If you allow @foo inside a Python file, it is not
Python anymore, is it?

Regards,
Martin
From jjl at pobox.com  Thu Aug  5 23:25:59 2004
From: jjl at pobox.com (John J Lee)
Date: Thu Aug  5 23:26:06 2004
Subject: [Python-Dev] No handlers could be found for logger "cookielib"
In-Reply-To: <20040805094437.32296.qmail@web25403.mail.ukl.yahoo.com>
References: <20040805094437.32296.qmail@web25403.mail.ukl.yahoo.com>
Message-ID: <Pine.LNX.4.58.0408052222570.15102@alice>

On Thu, 5 Aug 2004, [iso-8859-1] Vinay Sajip wrote:

> I just made a change to test_logging to clean up after
> itself, so that it removes all handlers that it
> creates. test_urllib2 seems to have handler creation
> commented out  in NetworkTests.setUp(). A combination
> of the two may explain why the message shows up now.

FWLIW:

The handler creation in test_urllib2.py has been 'if 0'-ed out since that
code was added to test_urllib2.py on the last bug day.  It's there for
debugging purposes, as the comment in that TestCase method points out.

I just ran the entire test suite, so that Tim doesn't feel too lonely ;-)


John
From skip at pobox.com  Thu Aug  5 23:29:21 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug  5 23:29:37 2004
Subject: [Python-Dev] use of #pragma GCC system_header to suppress
	_XOPEN_SOURCE warnings
Message-ID: <16658.42673.542930.749915@montanaro.dyndns.org>


I really hate to disrupt this scintillating function decorator discussion...

At work the goal for our C++ build environment is "no warnings with gcc
-Wall".  We are not there yet (and thankfully I am not a participant in that
particular holy grail), but those involved in that effort encountered a
warning the other day generated because _XOPEN_SOURCE is defined in
pyconfig.h.  Unfortunately GCC also defines it, so a "macro redefined"
warning is emitted when compiling with GCC and including Python.h.

I'm not about to suggest that the _XOPEN_SOURCE definition be removed from
pyconfig.h.in.  I trust that Martin v. L?wis and others involved have good
reasons to leave it in.  Perhaps adding

    #pragma GCC system_header

to pyconfig.h.in would be acceptable though.  This tells GCC that the file
it appears in is a system header file and to not bitch about trivial stuff
like macro redefinitions.  This suggestion comes from one of the C++ experts
here.  I would never have figured it out myself.  The scope of the pragma is
only the file in which it appears, and it doesn't take effect for even that
file until it's first encountered.  This suggests that perhaps a slight
restructuring of pyconfig.h.in would help to reduce its effect even more:

    <all macros which don't start with "_">

    #pragms GCC system_header

    <all macros which do start with "_">

(Pyconfig.h.in is almost there now.)

Some exceptions to the above structure might be necessary if the definition
of a macro in the first group is conditioned on the value of a macro from
the second group.  I don't believe this is the case, at least not at the
moment.  It's unlikely to ever be the case since any smarts for such tests
more appropriately belong in the configure script (or would be hand-coded on
non-configure-type platforms).

I've adjusted my local copy of pyconfig.h.in and am building CVS HEAD and
the 2.3 maintenance branch on both Solaris and MacOSX to see if any problems
arise.  I'd appreciate some feedback on whether or not this is deemed a
reasonable idea though.

Thx,

Skip
From martin at v.loewis.de  Thu Aug  5 23:29:35 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 23:29:40 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <875c7e07040805140174141409@mail.gmail.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>	<20040805184845.GB29796@burma.localdomain>	<AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>	<875c7e07040805123136fe96de@mail.gmail.com>	<41129E48.7050607@v.loewis.de>
	<875c7e07040805140174141409@mail.gmail.com>
Message-ID: <4112A6BF.4040004@v.loewis.de>

Chris King wrote:
> I should have been more specific -- I meant function attributes of
> this form (proposed and subsequently rejected somewhere else a while
> ago):

I see. So you would suggest to make a different extension *instead*
of the currently proposed one.

> def foo(a,b,c):
>     .signature = 'v@:@i'
>     pass
> 
> I only bring this up because most (though I realize not all) use cases
> for decorators (at least in their current form) seem to involve some
> type of attribute or metadata.

Yes. Remember that Python has no declarations, though - there is, in
general, no separation between the definition of a property, and its
"enforcement". For example, classmethod, staticmethod are not considered
by the class, but declarations that have behaviour themselves. If the
container (e.g. the class) would have to evaluate the attribute, it
would need to know all attributes in advance - something which is not
really possible.

Many other languages have taken the path of making some of these
attributes built-in (which is what you are essentially proposing)
only to find out that this is insufficient, and needed to be extended
by a more general mechanism later.

Regards,
Martin
From nidoizo at yahoo.com  Thu Aug  5 23:29:52 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Thu Aug  5 23:29:57 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
In-Reply-To: <f8ef9ae404080512245b3e62a0@mail.gmail.com>
References: <f8ef9ae404080512245b3e62a0@mail.gmail.com>
Message-ID: <ceu8sh$6j9$1@sea.gmane.org>

IxokaI wrote:
> I added "with", although I havn't seen it. "using" would work too. I
> personally do not like punctuation syntax-- I'd really like to see
> something english, even with the hated addition of a keyword. I
> understand Guido's disapproval for "as" so wanted to think of an
> alternate.

Other crazy ideas (in case it inspires anyone):

accepts(int,int,def)
returns(float,def)
def bar(low,high):

for def accepts(int,int)
for def returns(float)
def bar(low,high):

equivalent to:
@accepts(int,int)
@returns(float)
def bar(low,high):

From martin at v.loewis.de  Thu Aug  5 23:31:50 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 23:31:59 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <1091740792.8546.133.camel@localhost>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>
	<1091740792.8546.133.camel@localhost>
Message-ID: <4112A746.6010501@v.loewis.de>

Barry Warsaw wrote:
> Martin makes a good point.  Guido could threaten to remove the feature
> by beta 1 (and thus for 2.4 final) if the PEP is not brought up to date.
> 
> not-that-i'm-volunteering-'cause-i'd-rather-see-it-done-ly y'rs,

I'm atleast volunteering to take the code out if no proper
implementation (including documentation and test cases) is delivered by
some deadline.

Regards,
Martin
From barry at python.org  Thu Aug  5 23:32:41 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Aug  5 23:32:32 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <4112A4B9.4010907@v.loewis.de>
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de>
Message-ID: <1091741561.8541.144.camel@localhost>

On Thu, 2004-08-05 at 17:20, "Martin v. L?wis" wrote:

> > How is this not going to happen with the '@' syntax?
> 
> It's of course hard to tell how tools will react to that syntax, or
> with any other syntax change, for that matter. This specific is likely
> easy to implement, because it likely has only minimal impact on people's
> parsers: one can essentially treat a line starting with @ like a
> statement, e.g. by making @ a keyword.

Not that it should have any impact on the decision in any way, but the
regular expression python-mode.el uses to syntax color pie decorators is
much simpler (to figure out and to read) than the previous decorator
syntax support of list-before-colon.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/85adf199/attachment.pgp
From martin at v.loewis.de  Thu Aug  5 23:34:07 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 23:34:09 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
In-Reply-To: <ceu8sh$6j9$1@sea.gmane.org>
References: <f8ef9ae404080512245b3e62a0@mail.gmail.com>
	<ceu8sh$6j9$1@sea.gmane.org>
Message-ID: <4112A7CF.3030700@v.loewis.de>

Nicolas Fleury wrote:
> Other crazy ideas (in case it inspires anyone):
> 
> accepts(int,int,def)
> returns(float,def)
> def bar(low,high):

That doesn't work. If accepts and returns are callables
(as they should be), then this already means something
in current Python. So this would not be backwards
compatible.

> for def accepts(int,int)
> for def returns(float)
> def bar(low,high):

This is probably very confusing to existing tools,
which expect an identifier after the for, and a colon
somewhere on the same line.

Regards,
Martin
From edreamleo at charter.net  Thu Aug  5 23:34:52 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Thu Aug  5 23:35:02 2004
Subject: [Python-Dev] Questions about '@' in pep 318
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<20040805205559.GA31288@burma.localdomain>
Message-ID: <003601c47b34$0b4bf660$6700a8c0@computer>

> > annotate.accepts(int, (int,float))
>
> I don't understand. How are you changing/annotating the function?

I'm glad you asked.  My suggestion was that the compiler turn

annotate.accepts(int, (int,float))

into exactly what it would have turned

@accepts(int, (int,float))

into.

I hope I am not being incredibly stupid, but I am thinking only of syntax
here.  So wherever the community would have decided to put:

@accepts <<whatever>>

the new "syntax" would be:

annotate.accepts <<whatever>>

Do you see? The annotate "module" probably doesn't exist at all.  I think of
it as a pseudo-module.  That's why I wrote "from __future__ import
annotation as annotate" in the original post.

My thinking is this:  apparently people like the @ syntax.  Why not do some
magic and compile module references to the annotation pseudo-module just as
it is presently proposed to compile the @ syntax.  That way we don't change
the apparent syntax at all, we use the annotation namespace to denote what
is happening, and we use annotate.x exactly as we would have used @x.

Again, my thinking is that this would be completely independent of the other
choices about where to use @.  Wherever the consensus is, then use the
annotation pseudo-module there.

Have I made myself clear?

Edward

P.S.  I wrote:

from __future__ import annotation as annotate

as a hint that one could write:

from __future__ import annotation as a

if brevity is important.  Then we would write a.accepts instead of @accepts.

EKR
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From aahz at pythoncraft.com  Thu Aug  5 23:37:13 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Aug  5 23:37:16 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
Message-ID: <20040805213713.GA28782@panix.com>

On Thu, Aug 05, 2004, Phillip J. Eby wrote:
>
> So there it is, my personal argument for list-after-def as better than "@", 
> even if the list is over multiple lines: it's still much more clearly part 
> of the function definition, and that IMO is the most important thing the 
> syntax should convey, apart from listing the decorators themselves.

+1

I've been staying out of this because I thought my preference had no
chance to make it in (i.e. I don't care between "no decorators" and "@
decorators"), but if it's getting some revival, I want to make clear
that I think putting decorators *after* ``def``, name, and parameter
list but clearly *with* the function definition makes the most sense for
readability -- in the simple cases.  Who cares about the complex cases,
because it's always going to be ugly?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From martin at v.loewis.de  Thu Aug  5 23:40:56 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug  5 23:41:00 2004
Subject: [Python-Dev] use of #pragma GCC system_header to
	suppress	_XOPEN_SOURCE warnings
In-Reply-To: <16658.42673.542930.749915@montanaro.dyndns.org>
References: <16658.42673.542930.749915@montanaro.dyndns.org>
Message-ID: <4112A968.2030401@v.loewis.de>

Skip Montanaro wrote:
> At work the goal for our C++ build environment is "no warnings with gcc
> -Wall".  We are not there yet (and thankfully I am not a participant in that
> particular holy grail), but those involved in that effort encountered a
> warning the other day generated because _XOPEN_SOURCE is defined in
> pyconfig.h.  Unfortunately GCC also defines it, so a "macro redefined"
> warning is emitted when compiling with GCC and including Python.h.

That is not true. GCC does not define that macro, on any platform.

Some system header may define it, though. I believe this is a bug in the
system - the macro *should* be defined in the application. What system
are you referring to, and where does that define _XOPEN_SOURCE? Is that
definition conditional?

> Perhaps adding
> 
>     #pragma GCC system_header
> 
> to pyconfig.h.in would be acceptable though.

I'd prefer to solve the problem properly. If the system defines
something, and we define it differently, this *is* a problem, and
deserves a warning. We should find out why that is, and what
can be done about. Perhaps we should not define _XOPEN_SOURCE
on that platform until the vendor fixes the platform bug, or
we should aim at defining it the same way the vendor does.

Silencing errors is a risky thing.

Regards,
Martin
From jjl at pobox.com  Thu Aug  5 23:47:01 2004
From: jjl at pobox.com (John J Lee)
Date: Thu Aug  5 23:47:10 2004
Subject: httplib patch for 2.4 (was: Re: [Python-Dev] 2.4a2 ...)
In-Reply-To: <e8bf7a53040802104616e42b40@mail.gmail.com>
References: <410DDFE3.7010006@interlink.com.au>
	<Pine.WNT.4.58.0408021425420.1720@vernon>
	<e8bf7a53040802104616e42b40@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0408052244000.15102@alice>

On Mon, 2 Aug 2004, Jeremy Hylton wrote:
> On Mon, 2 Aug 2004 14:30:39 +0100 (GMT Daylight Time), John J Lee
> <jjl@pobox.com> wrote:
[...]
> > Would somebody mind checking this in:
> >
> > http://www.python.org/sf/997626
> >
> > It fixes a bug in httplib which badly breaks urllib2 now that urllib2 is
> > using httplib.HTTPConnection instead of httplib.HTTP (POSTs don't work).
> >
> > It includes patch and a new test, and does not require doc updates, but
> > nobody has reviewed it yet.
>
> I'll take a look, probably tomorrow morning.
>
> Jeremy
[...]

It don't think it's in especial need of reviewing by an expert.  It just
needs to get checked in!


John
From niemeyer at conectiva.com  Thu Aug  5 23:53:31 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Aug  5 23:51:14 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112A18D.5090801@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
Message-ID: <20040805215331.GB31288@burma.localdomain>

> Depends on in what direction you want to make a change. It
> appears you want to avoid introducing any kind of syntax
> change. In that case, you should explain people how to spell
> classmethod and synchronized in a more convenient way, because
> that is what the stated motivation of PEP 318 is - you would
> have to explain why this motive is bad, irrelevant, or otherwise
> not a worthy goal.
> 
> Or you could argue on a procedural basis: regardless of whether
> the feature is good or bad, the current implementation is
> unacceptable, as the PEP does not correspond with the
> implementation, the syntax is undocumented, the code has no test
> cases, and so on. I'm actually going to do that, because I do
> think the process is unacceptable, and should be either corrected
> or reversed (of course, this says nothing about the feature itself,
> or the code implementing it).

Ok, I'll try to summarize the current status of the feature
so that I (and others) can understand if there's something to
be done:

- Decorators are going in on 2.4.

- There are two obvious usage cases for decorators: static and
  class methods;

- There are more complex usage cases for decorators such as
  PyObjC which was already agreed to be something necessarily
  supported in the implementation;

- People which want the powerful decorators don't care about
  the syntax, as far as the feature is implemented;

- Adapting external tools should not be used as an argument;

- We're clearly unable to get into a consensus about syntax;

- All current syntax vs. problems to solve have been discussed
  extensively;

- There are bizarre usage cases of the current decorator
  implementation, but this is something which is considered
  abusive and won't affect decisions since should be casual;

- The @ character is used in at least two tools (Leo, IPython),
  and this is being considered as something bad, but not a
  show stopper;

- The perlish argument is non-sense;

I belive that either some different syntax which most
people agree upon is raised, or we're done.

-- 
Gustavo Niemeyer
http://niemeyer.net
From gregory.lielens at fft.be  Thu Aug  5 23:52:26 2004
From: gregory.lielens at fft.be (Gregory Lielens)
Date: Thu Aug  5 23:52:42 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
Message-ID: <001b01c47b36$8216f6d0$0f00000a@chikubi>


> Of the options on the Wiki page ( at 
> http://www.python.org/moin/PythonDecorators ), this seems to 
> leave only 
> list-after-def, the previous community favorite and the first PEP 318 
> syntax to have been implemented.
> 
> Guido's principal argument against list-after-def, if I 
> recall correctly, 
> was that it is ugly when multiple or lengthy decorators are 
> involved.  But, 
> "ugly" isn't an argument any more, so that shouldn't rule out 
> list-after-def.  :)

On a cosmetic point of view, I also prefer the list-after-def syntax,
first time I saw it it was the one than seemed the easiest (the "def f()
as [decorator]" variant was event clearer, but the argument that as is
only for renaming has mitigaed that somewhat ;-) )
"Cosmetic" does not sound like a very convincing argument, but I feel
that this whole discussion is purely on a cosmetic level anyway, it is
just a way to express more clearly something that is already
possible...and as feedback as been asked for... ;-)

But does it means that the list-after-def behaves like all lists, so
that these are possible and equivalent?

def foo() [decorator1, decorator2(bar)]:
   ...

def foo() [
    decorator1, 
    decorator2(bar)]:
   ...

my_complex_decorator=[decorator1, decorator2(bar)]
def foo() my_complex_decorator :
   ...

If it is, I am +1 on list-after-def (and, because of the last example,
+2 on a variant with a keyword or pseudo-keyword between the def and the
list, like ("with" is only an example) :

def foo() with [decorator1, decorator2(bar)]:
   ... 
my_complex_decorator=[decorator1, decorator2(bar)]
def foo() with my_complex_decorator :
   ...
def foo2() with my_complex_decorator :
   ...


If not (i.e. the brackets are just a syntax hint), I am -1, I prefer the
the pie syntax cause it would break consistency with other lists
usage...
For example, 
  for j in [1,2]:
    ...
  for j in ["a","b",
	       1,2,3,4]:
     ...
  for j in mylist:
     ...
are all possible, so it would be very surprising if the def syntax would
not be similar, it would mean that the brackets after a def are not
really a list and that would feel too much like special-casing to me...

Best regards,

Greg.


From nidoizo at yahoo.com  Thu Aug  5 23:55:00 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Thu Aug  5 23:55:08 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
In-Reply-To: <4112A7CF.3030700@v.loewis.de>
References: <f8ef9ae404080512245b3e62a0@mail.gmail.com>	<ceu8sh$6j9$1@sea.gmane.org>
	<4112A7CF.3030700@v.loewis.de>
Message-ID: <ceuabl$9t3$1@sea.gmane.org>

Martin v. L?wis wrote:
> Nicolas Fleury wrote:
>> accepts(int,int,def)
>> returns(float,def)
>> def bar(low,high):
> 
> That doesn't work. If accepts and returns are callables
> (as they should be), then this already means something
> in current Python. So this would not be backwards
> compatible.

I'm sorry (I don't like that solution at all honestly), but what passing 
"def" to a callable means in current Python?  It is not accepted by my 
interactive shell...

Regards,
Nicolas

From ixokai at gmail.com  Fri Aug  6 00:00:24 2004
From: ixokai at gmail.com (IxokaI)
Date: Fri Aug  6 00:00:27 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <001b01c47b36$8216f6d0$0f00000a@chikubi>
References: <001b01c47b36$8216f6d0$0f00000a@chikubi>
Message-ID: <f8ef9ae404080515004c1f5e98@mail.gmail.com>

For what its worth, this lurker until now is strongly +1 on
list-after-def, regardless of how it turns out. Real list, fake list,
pseudo-keyword seperator.

I don't like @syntax because unless you're using whitespace between
def's in the 'right' way its just.. not completely clear what belongs
to what.

class incdec:
    def increment (arg):
       return arg +1
    increment = staticmethod(increment)
    @staticmethod
    def decrement(arg):
       return arg - 1

That looks like a mess. Python uses whitespace to group things, and
@syntax is not in any way grouped or connected to the function..  No,
its not ambigious, its just-- messy, unclear to a novice.

However:

class incdec:
    def increment(arg) [staticmethod]:
        return arg +1
    def decrement(arg) using [staticmethod]:
        return arg -1 

*clearly* shows that the decorator is decorating that specific
function. You may not know just what its doing, but you know its
directly related. I think that's very important.

--Stephen

On Thu, 5 Aug 2004 23:52:26 +0200, Gregory Lielens
<gregory.lielens@fft.be> wrote:
> 
> > Of the options on the Wiki page ( at
> > http://www.python.org/moin/PythonDecorators ), this seems to
> > leave only
> > list-after-def, the previous community favorite and the first PEP 318
> > syntax to have been implemented.
> >
> > Guido's principal argument against list-after-def, if I
> > recall correctly,
> > was that it is ugly when multiple or lengthy decorators are
> > involved.  But,
> > "ugly" isn't an argument any more, so that shouldn't rule out
> > list-after-def.  :)
> 
> On a cosmetic point of view, I also prefer the list-after-def syntax,
> first time I saw it it was the one than seemed the easiest (the "def f()
> as [decorator]" variant was event clearer, but the argument that as is
> only for renaming has mitigaed that somewhat ;-) )
> "Cosmetic" does not sound like a very convincing argument, but I feel
> that this whole discussion is purely on a cosmetic level anyway, it is
> just a way to express more clearly something that is already
> possible...and as feedback as been asked for... ;-)
> 
> But does it means that the list-after-def behaves like all lists, so
> that these are possible and equivalent?
> 
> def foo() [decorator1, decorator2(bar)]:
>    ...
> 
> def foo() [
>     decorator1,
>     decorator2(bar)]:
>    ...
> 
> my_complex_decorator=[decorator1, decorator2(bar)]
> def foo() my_complex_decorator :
>    ...
> 
> If it is, I am +1 on list-after-def (and, because of the last example,
> +2 on a variant with a keyword or pseudo-keyword between the def and the
> list, like ("with" is only an example) :
> 
> def foo() with [decorator1, decorator2(bar)]:
>    ...
> my_complex_decorator=[decorator1, decorator2(bar)]
> def foo() with my_complex_decorator :
>    ...
> def foo2() with my_complex_decorator :
>    ...
> 
> If not (i.e. the brackets are just a syntax hint), I am -1, I prefer the
> the pie syntax cause it would break consistency with other lists
> usage...
> For example,
>   for j in [1,2]:
>     ...
>   for j in ["a","b",
>                1,2,3,4]:
>      ...
>   for j in mylist:
>      ...
> are all possible, so it would be very surprising if the def syntax would
> not be similar, it would mean that the brackets after a def are not
> really a list and that would feel too much like special-casing to me...
> 
> Best regards,
> 
> Greg.
> 
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ixokai%40gmail.com
>
From niemeyer at conectiva.com  Fri Aug  6 00:07:43 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Fri Aug  6 00:05:26 2004
Subject: [Python-Dev] Similar syntax
Message-ID: <20040805220743.GC31288@burma.localdomain>

> Ok, I'll try to summarize the current status of the feature
> so that I (and others) can understand if there's something to
> be done:
> 
> - Decorators are going in on 2.4.
[...]
> - The @ character is used in at least two tools (Leo, IPython),
>   and this is being considered as something bad, but not a
>   show stopper;
> 
> - The perlish argument is non-sense;
> 
> I belive that either some different syntax which most
> people agree upon is raised, or we're done.

After writing this list, I thought about something which is
close to the current implementation, but avoids @. If it was
already suggested, please forgive me. At least I wasn't able
to find it in the PEP/Wiki:

    <static>
    def method(foo):
        pass

and also

    <accepts(int,int), returns(float)>
    def bar(low, high):
        ...

Advantages over current syntax:

- Avoids introducing a new meaningful character in the syntax;

- Leaves Leo and IPython people happy;

- ">" is already being used on "print >>", so special casing
  it wouldn't be something new;

- Not acceptable in the current syntax;

- Looks like a tag;

-- 
Gustavo Niemeyer
http://niemeyer.net
From exarkun at divmod.com  Fri Aug  6 00:07:25 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Fri Aug  6 00:07:30 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
In-Reply-To: <4112A7CF.3030700@v.loewis.de>
References: <f8ef9ae404080512245b3e62a0@mail.gmail.com>	<ceu8sh$6j9$1@sea.gmane.org>
	<4112A7CF.3030700@v.loewis.de>
Message-ID: <4112AF9D.1080202@divmod.com>

Martin v. L=F6wis wrote:
> Nicolas Fleury wrote:
> =

>> Other crazy ideas (in case it inspires anyone):
>>
>> accepts(int,int,def)
>> returns(float,def)
>> def bar(low,high):
> =

> =

> That doesn't work. If accepts and returns are callables
> (as they should be), then this already means something
> in current Python. So this would not be backwards
> compatible.

   Adding _any_ names to Python has this potential problem.  I don't =

think that's going to stop anyone from adding new modules to the =

standard library or new builtins (2.4 introduces 3 new builtins).

   To say this isn't backwards compatible is true, but not in a sense =

that strikes me as important.  "accepts" and "returns" don't even need =

to be builtins, they could be placed in a module with a handful of other =

useful common decorators.

   Jp
From edreamleo at charter.net  Fri Aug  6 00:09:32 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 00:09:41 2004
Subject: [Python-Dev] The impact of '@' on Leo
References: <005e01c47b2d$9c0dee80$6700a8c0@computer>
	<4112A593.7080608@v.loewis.de>
Message-ID: <000701c47b38$e2e526b0$6700a8c0@computer>

> > The major concern I have with '@' syntax as it relates to Leo is the
> > potential for ongoing and never-ending conflicts between Leo's
directives
> > and @x constructs in Python.
>
> Can you give an example of a file that would be difficult to understand
> under that change? If you allow @foo inside a Python file, it is not
> Python anymore, is it?

There are two files involved.  The .leo file (an xml file) contains the
"marked up" code.  Leo turns this file into a visual outline containing zero
or more "derived files" that could be written in any language.

Here is the entire top-level node in the outline that represents the derived
file leoApp.py:

[Node starts]

@first # -*- coding: utf-8 -*-

@language python
@tabwidth -4

import leoGlobals as g
import os
import sys

class LeoApp:

    """A class representing the Leo application itself.

    Ivars of this class are Leo's global variables."""

    @others

[Node ends]

- The [Node starts] and [Node ends] are _not_ part of the node, they delimit
the node in this email.
- The @first directive makes sure that the first line of leoApp.py is # -*-
coding: utf-8 -*-
- The @language python and @tabwidth directives tell Leo how to syntax color
the text and how to handle tabs.
- The @others directive tells Leo to insert all descendent nodes.  In this
case, each descendent node contains a single method of the LeoApp class.
- There are other details to Leo's markup, but this gives you the general
feel.

Everything else is plain Python.  When saving the .leo file, Leo produces
leoApp.py following the rules of the markup.  In particular, Leo writes all
descendent nodes in place of the @others directive, indented the same as the
@others directive.  Most of these descendent nodes will contain few or no
Leo directives: descendent nodes inherit most directives from their
ancestors.

The problem is that Leo doesn't really understand _any_ language, except to
colorize it.  Most Leo directives must start with @ in the leftmost column,
but the @others and @all directives may contain leading whitespace.  If @foo
is both a valid Python construct and a valid Leo directive Leo will treat
@foo as a Leo directive even if it is "supposed" to be Python.

If @foo is not a valid Leo directive, Leo creates a "verbatim" sentinel line
in the derived file to indicate that the next line isn't a directive, even
though it sorta looks like one.  Like this:

#@verbatim
@foo

This is basically how Leo handle's languages with @ signs.  There is no
problem until @x in the language conflicts with a Leo directive, in which
case Leo doesn't have any way of distinguishing which usage was intended.

As I said in another post, Leo doesn't have escape conventions involving @
signs, and that has turned out well.  There have been requests to allow
strings other than "@" to delimit Leo directives, and that will probably
happen, but in general Leo really would rather stick to "@"

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From pje at telecommunity.com  Fri Aug  6 00:15:09 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug  6 00:11:09 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <001b01c47b36$8216f6d0$0f00000a@chikubi>
References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040805181155.021c9910@mail.telecommunity.com>

At 11:52 PM 8/5/04 +0200, Gregory Lielens wrote:

> > Of the options on the Wiki page ( at
> > http://www.python.org/moin/PythonDecorators ), this seems to
> > leave only
> > list-after-def, the previous community favorite and the first PEP 318
> > syntax to have been implemented.
> >
> > Guido's principal argument against list-after-def, if I
> > recall correctly,
> > was that it is ugly when multiple or lengthy decorators are
> > involved.  But,
> > "ugly" isn't an argument any more, so that shouldn't rule out
> > list-after-def.  :)
>
>On a cosmetic point of view, I also prefer the list-after-def syntax,
>first time I saw it it was the one than seemed the easiest (the "def f()
>as [decorator]" variant was event clearer, but the argument that as is
>only for renaming has mitigaed that somewhat ;-) )
>"Cosmetic" does not sound like a very convincing argument, but I feel
>that this whole discussion is purely on a cosmetic level anyway, it is
>just a way to express more clearly something that is already
>possible...and as feedback as been asked for... ;-)
>
>But does it means that the list-after-def behaves like all lists, so
>that these are possible and equivalent?
>
>def foo() [decorator1, decorator2(bar)]:
>    ...
>
>def foo() [
>     decorator1,
>     decorator2(bar)]:
>    ...
>
>my_complex_decorator=[decorator1, decorator2(bar)]
>def foo() my_complex_decorator :
>    ...

The third example isn't allowed, but the first two are.  Remember that part 
of the point is that the [] are being used to visually signal 
annotation.  The third syntax loses this point.

Note that there is precedent for this kind of syntax restriction; a class' 
bases are a tuple, but that doesn't mean you can do this:

     yz = (Y,Z)

     class X yz:
         pass

From edreamleo at charter.net  Fri Aug  6 00:16:33 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 00:16:43 2004
Subject: [Python-Dev] Questions about '@' in pep 318
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de>
Message-ID: <000f01c47b39$de3bcd70$6700a8c0@computer>

> That requires all annotations to be in the __future__ module. This is not
acceptable, as there is a desire to provide user-defined annotations as
well.

Oh dear.  Does this mean that @x is a potential annotation for any
identifier x?  This would be very bad for Leo.

I was using __future__ by way of explanation.  I do hope namespaces could
somehow denote annotations.  My off-the-cuff suggestion was for
pseudo-modules, so maybe the normal module rules could be sidestepped?

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From skip at pobox.com  Fri Aug  6 00:19:16 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 00:19:33 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <20040805213713.GA28782@panix.com>
References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
	<20040805213713.GA28782@panix.com>
Message-ID: <16658.45668.851011.950341@montanaro.dyndns.org>


    >> So there it is, my personal argument for list-after-def as better
    >> than "@" ...

    aahz> +1

What he said.

+1.

I had simply given up arguing the point because it seemed pointless and I
was tired of arguing.  I suppose there are more than a few people in this
particular boat.

Skip
From edreamleo at charter.net  Fri Aug  6 00:19:29 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 00:19:38 2004
Subject: [Python-Dev] Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
Message-ID: <002301c47b3a$469bc5f0$6700a8c0@computer>

> Or you could argue on a procedural basis: regardless of whether
> the feature is good or bad, the current implementation is
> unacceptable, as the PEP does not correspond with the
> implementation, the syntax is undocumented, the code has no test
> cases, and so on. I'm actually going to do that, because I do
> think the process is unacceptable, and should be either corrected
> or reversed (of course, this says nothing about the feature itself,
> or the code implementing it).

Thank you, Martin.  I think Python will be the better for holding itself to
higher standards.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From eppstein at ics.uci.edu  Fri Aug  6 00:22:54 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Fri Aug  6 00:23:00 2004
Subject: [Python-Dev] Re: Questions about '@' in pep 318
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<20040805205559.GA31288@burma.localdomain>
	<003601c47b34$0b4bf660$6700a8c0@computer>
Message-ID: <eppstein-3BB6F5.15225405082004@sea.gmane.org>

In article <003601c47b34$0b4bf660$6700a8c0@computer>,
 "Edward K. Ream" <edreamleo@charter.net> wrote:

> I hope I am not being incredibly stupid, but I am thinking only of syntax
> here.  So wherever the community would have decided to put:
> 
> @accepts <<whatever>>
> 
> the new "syntax" would be:
> 
> annotate.accepts <<whatever>>
> 
> Do you see? The annotate "module" probably doesn't exist at all.  I think of
> it as a pseudo-module.  That's why I wrote "from __future__ import
> annotation as annotate" in the original post.
> 
> My thinking is this:  apparently people like the @ syntax.  Why not do some
> magic and compile module references to the annotation pseudo-module just as
> it is presently proposed to compile the @ syntax.

I'm not convinced this is possible in general (for all pythons, 
obviously it's possible by gross hacks in CPython) without making 
annotate into a keyword.

And, I don't like the choice of the word, either, since I think 
decorators are more likely to be used to change the meaning of the 
definition and not just attach some meaningless annotation to it.

Anyway, since decorators are quite different from normal control flow in 
Python I'd prefer to see them given a syntax that stands out, that makes 
it obvious that it's not just some slightly mysterious function call 
prior to the def.  @syntax does that for me and yours doesn't.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From skip at pobox.com  Fri Aug  6 00:24:27 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 00:24:42 2004
Subject: [Python-Dev] use of #pragma GCC system_header to
	suppress        _XOPEN_SOURCE warnings
In-Reply-To: <4112A968.2030401@v.loewis.de>
References: <16658.42673.542930.749915@montanaro.dyndns.org>
	<4112A968.2030401@v.loewis.de>
Message-ID: <16658.45979.672934.897388@montanaro.dyndns.org>


    >> At work the goal for our C++ build environment is "no warnings with
    >> gcc -Wall".  We are not there yet (and thankfully I am not a
    >> participant in that particular holy grail), but those involved in
    >> that effort encountered a warning the other day generated because
    >> _XOPEN_SOURCE is defined in pyconfig.h.  Unfortunately GCC also
    >> defines it, so a "macro redefined" warning is emitted when compiling
    >> with GCC and including Python.h.

    Martin> That is not true. GCC does not define that macro, on any
    Martin> platform.

    Martin> Some system header may define it, though.

Well, yeah, that's what I meant.  My apologies for the imprecise statement.

    Martin> I believe this is a bug in the system - the macro *should* be
    Martin> defined in the application. What system are you referring to,
    Martin> and where does that define _XOPEN_SOURCE? Is that definition
    Martin> conditional?

The platform is Solaris/Intel (2.8 and/or 2.9).  I'll have to get you more
complete details tomorrow.  The involved folks are gone for the day.

    Martin> Silencing errors is a risky thing.

Agreed.  I made them investigate the python-dev archives before
contributing.  They said, "Who is this Martin guy?" :-)

wasn't-this-a-pleasant-diversion-from-pie-syntax?-ly y'rs,

Skip
From barry at python.org  Fri Aug  6 00:25:17 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug  6 00:25:09 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805215331.GB31288@burma.localdomain>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
Message-ID: <1091744717.8545.153.camel@localhost>

On Thu, 2004-08-05 at 17:53, Gustavo Niemeyer wrote:

> - There are two obvious usage cases for decorators: static and
>   class methods;
> 
> - There are more complex usage cases for decorators such as
>   PyObjC which was already agreed to be something necessarily
>   supported in the implementation;

I'm not sure these two points are accurate.  There's definitely a large
middle ground between staticmethod/classmethod and PyObjC's use cases.

> - People which want the powerful decorators don't care about
>   the syntax, as far as the feature is implemented;

I care deeply about the syntax, but I can accept any of the choices that
have made Guido's final cut.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/8380bfdb/attachment.pgp
From martin at v.loewis.de  Fri Aug  6 00:26:02 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 00:26:08 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805215331.GB31288@burma.localdomain>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
Message-ID: <4112B3FA.3050106@v.loewis.de>

Gustavo Niemeyer wrote:
> - Decorators are going in on 2.4.

Yes, that's the current status. Of course, it is difficult to
predict, especially the future. Who knows what would happen
if people find flaws in the proposed syntax, and no alternative
is available?

> - There are two obvious usage cases for decorators: static and
>   class methods;

Correct.

> - There are more complex usage cases for decorators such as
>   PyObjC which was already agreed to be something necessarily
>   supported in the implementation;

Correct.

> - People which want the powerful decorators don't care about
>   the syntax, as far as the feature is implemented;

Some actually do care about the syntax as well.

> - Adapting external tools should not be used as an argument;

No, external tools need to be considered. Preferably in the
concrete, not in the abstract.

> - We're clearly unable to get into a consensus about syntax;

Yes. I believe this is not specific to this change - people
*always* argue about syntax.

> - All current syntax vs. problems to solve have been discussed
>   extensively;

Yes.

> - There are bizarre usage cases of the current decorator
>   implementation, but this is something which is considered
>   abusive and won't affect decisions since should be casual;

Yes.

> - The @ character is used in at least two tools (Leo, IPython),
>   and this is being considered as something bad, but not a
>   show stopper;

This is certainly bad. Whether it is a show-stopper, I don't know.
I don't understand the Leo issue; I thought it would use the @
syntax only within Python comments, in which case there would be
no conflicts.

> - The perlish argument is non-sense;

Yes. In a typical decorator, the only special characters on
the line will be the @ and possibly () (perhaps also commas
between arguments). This is completely unlike Perl which
often has large fractions on special characters on a line.

> I belive that either some different syntax which most
> people agree upon is raised, or we're done.

Perhaps. People really should use it to find potential flaws
in the semantics of decorators, e.g. things you ought to be
able to do with decorators but can't. Of course,
test_decorators.py already gives a good idea what is possible,
so it might be "good enough".

Regards,
Martin
From s.percivall at chello.se  Fri Aug  6 00:29:05 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Fri Aug  6 00:29:11 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051636.i75Gaak03654@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
Message-ID: <DC304556-E72E-11D8-8E6E-0003934AD54A@chello.se>

+1 on @decorator style for being clear, unambiguous and useful
(and not bad to look at once you get used to it).

clear: * it's more visible and less messy than any of the
          before-/after-/behind-def style decorations;
        * it stands out. this is a good thing.

unambiguous: unlike the other viable prefix notation it makes 2.4
              code special.

useful: just having a special decoration syntax makes it useful.


decorations can make the behaviour of a function or class seem more
magic than otherwise, but at least the @decoration style makes it
more obvious that something is going on.

On 2004-08-05, at 18.36, Guido van Rossum wrote:
> [...] you might want to say something in public in defense of
> @decorators.

From eppstein at ics.uci.edu  Fri Aug  6 00:29:09 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Fri Aug  6 00:29:16 2004
Subject: [Python-Dev] Re: A usability argument for list-after-def
References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
Message-ID: <eppstein-8A8FE9.15290905082004@sea.gmane.org>

In article <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>,
 "Phillip J. Eby" <pje@telecommunity.com> wrote:

> Guido has stated he'll accept one community syntax proposal as an 
> alternative to "@".  He's also ruled out his own previous proposal 
> (list-before-def), and a variety of new-keyword alternatives involving such 
> things as 'as' and 'with'.
> 
> There doesn't appear to be a downside for decorator supporters to put forth 
> another syntax, if they like it better than "@"; it seems the worst case is 
> that we still end up with "@", if we end up with any syntax at all.

It's clear from previous discussions that there can be no single 
community preference.  The downside to me is that all this brouhaha will 
prevent us from getting any syntax at all.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From eppstein at ics.uci.edu  Fri Aug  6 00:24:41 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Fri Aug  6 00:30:24 2004
Subject: [Python-Dev] Re: The impact of '@' on Leo
References: <005e01c47b2d$9c0dee80$6700a8c0@computer>
Message-ID: <eppstein-977D3E.15244105082004@sea.gmane.org>

In article <005e01c47b2d$9c0dee80$6700a8c0@computer>,
 "Edward K. Ream" <edreamleo@charter.net> wrote:

> The major concern I have with '@' syntax as it relates to Leo is the
> potential for ongoing and never-ending conflicts between Leo's directives
> and @x constructs in Python.

I don't know what Leo is, but the usual solution to this sort of problem 
is to introduce some kind of quoting convention, e.g. @@ gets passed 
through as a single @ to Python, any other @something gets treated as a 
Leo directive.  Is there some reason that wouldn't work?

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From tim.peters at gmail.com  Fri Aug  6 00:31:49 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug  6 00:31:52 2004
Subject: [Python-Dev] OK to lower timeout for test_timeout's
	testConnectTimeout test?
In-Reply-To: <4112A365.1000203@ocf.berkeley.edu>
References: <4112A365.1000203@ocf.berkeley.edu>
Message-ID: <1f7befae040805153126848c1@mail.gmail.com>

[Brett Cannon]
> I am actually getting failures from test_timeout's testConnectTimeout

It would have helped if you had been specific about the failures
you're seeing, and whether you get them all the time, or just some of
the time.

> since the Net connection I have at school lets me connect to Google in
> under the .001 connection timeout value.  If I push it to .00000001 (a
> hundred-millionth of a second) it then fails consistently.

What fails?  The test fails?  The socket fails to connect?

Note that .0000001 is the same as passing, e.g., 1e-200.  select()
only accepts arguments at microsecond granularity, so any positive
non-zero timeout value < 1e-6 gets chopped to exactly 0.0.

	tv.tv_sec = (int)s->sock_timeout;
	tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6);

The test really doesn't intend to pass a timeout of exactly 0 to
select(), so we can't change this to less than 1e-6.

> Now the problem is that the second part of the test uses this and a fuzz
> value (currently at 2) to see if the test timed out within the proper
> amount of time.  The comparison is basically if the amount of time it
> took to do the timed out failure is less than timeout + fuzz.  So
> lowering this number could possibly affect the test, although at .001
> seconds, I am doubting that will occur.  But since these types of timing
> tests can be touchy I thought I would double-check.

Na, 2.0 is gigantic compared to .001 already.  For the purposes of the
test, 2 isn't really more gigantic compared to 1e-6.

> So if anyone thinks it is bad to lower the value to .00000001 then
> please let me know.

Changing it to 1e-7 is out, for the reason explained earlier.  I'd
like to keep .001, because while the interface to select() *allows*
specifying microsecond resolution, there's no guarantee that it can
use that much precision.  Most (all?) implementations should be able
to deal with millisecond resolution, though.  Perhaps we could pick on
something other than www.google.com.  Maybe www.python.org (everyone
in America is far from that <wink>).
From martin at v.loewis.de  Fri Aug  6 00:33:24 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 00:33:29 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <002301c47b3a$469bc5f0$6700a8c0@computer>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<002301c47b3a$469bc5f0$6700a8c0@computer>
Message-ID: <4112B5B4.9@v.loewis.de>

Edward K. Ream wrote:
> Thank you, Martin.  I think Python will be the better for holding itself to
> higher standards.

I was somewhat too quick critizing the documentation, though:

http://www.python.org/dev/doc/devel/ref/function.html

does explain syntax and semantics, and there are testcases as well. So 
it's just the PEP that needs updating (a section in whatsnew also needs
to be drafted). Then, of course, the text saying that "@" is not used
needs to go.

I wonder whether replacing "@" with "!" or "%" would do any good...
In fact, any binary operator would work - Objective-C does use "+"
and "-" at the beginning of a line...

Regards,
Martin
From exarkun at divmod.com  Fri Aug  6 00:33:40 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Fri Aug  6 00:33:45 2004
Subject: [Python-Dev] Re: A usability argument for list-after-def
In-Reply-To: <eppstein-8A8FE9.15290905082004@sea.gmane.org>
References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
	<eppstein-8A8FE9.15290905082004@sea.gmane.org>
Message-ID: <4112B5C4.9070804@divmod.com>

David Eppstein wrote:
> In article <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>,
>  "Phillip J. Eby" <pje@telecommunity.com> wrote:
> 
> 
>>Guido has stated he'll accept one community syntax proposal as an 
>>alternative to "@".  He's also ruled out his own previous proposal 
>>(list-before-def), and a variety of new-keyword alternatives involving such 
>>things as 'as' and 'with'.
>>
>>There doesn't appear to be a downside for decorator supporters to put forth 
>>another syntax, if they like it better than "@"; it seems the worst case is 
>>that we still end up with "@", if we end up with any syntax at all.
> 
> 
> It's clear from previous discussions that there can be no single 
> community preference.  The downside to me is that all this brouhaha will 
> prevent us from getting any syntax at all.
> 

   This would be a major downside, if the feature were unsupportable 
without syntax additions.

   Jp
From martin at v.loewis.de  Fri Aug  6 00:38:20 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 00:38:22 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <000f01c47b39$de3bcd70$6700a8c0@computer>
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de>
	<000f01c47b39$de3bcd70$6700a8c0@computer>
Message-ID: <4112B6DC.4080605@v.loewis.de>

Edward K. Ream wrote:
> Oh dear.  Does this mean that @x is a potential annotation for any
> identifier x?  

Yes. Take a look at the documentation, and the test case.

> I was using __future__ by way of explanation.  I do hope namespaces could
> somehow denote annotations.  My off-the-cuff suggestion was for
> pseudo-modules, so maybe the normal module rules could be sidestepped?

I don't see how this would be possible. The plan is that arbitrary 
callables can be used as decorations, as long as they take a single
argument.

Regards,
Martin
From bob at redivi.com  Fri Aug  6 00:39:39 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Aug  6 00:39:46 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112B5B4.9@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de>
Message-ID: <55CC1464-E730-11D8-9D9A-000A95686CD8@redivi.com>


On Aug 5, 2004, at 6:33 PM, Martin v. L?wis wrote:

> Edward K. Ream wrote:
>> Thank you, Martin.  I think Python will be the better for holding 
>> itself to
>> higher standards.
>
> I was somewhat too quick critizing the documentation, though:
>
> http://www.python.org/dev/doc/devel/ref/function.html
>
> does explain syntax and semantics, and there are testcases as well. So 
> it's just the PEP that needs updating (a section in whatsnew also 
> needs
> to be drafted). Then, of course, the text saying that "@" is not used
> needs to go.
>
> I wonder whether replacing "@" with "!" or "%" would do any good...
> In fact, any binary operator would work - Objective-C does use "+"
> and "-" at the beginning of a line...

Except + and - would be valid Python syntax already ;)  For reference, 
in Objective-C, + means class method and - means instance method.  This 
is not decoration because they are in separate namespaces; +class 
methods are basically instance methods on an implicit metaclass.  In 
Objective-C, + and - are roughly equivalent to Python's "def" keyword.

Aesthetically I much prefer @ over ! or %.  !looks !bad !next !to !text 
!because !it !doesn't !stand !out and %makes %me %think %of %comments 
%in %another %language.

-bob
From martin at v.loewis.de  Fri Aug  6 00:41:44 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 00:41:50 2004
Subject: [Python-Dev] use of #pragma GCC system_header to suppress
	_XOPEN_SOURCE warnings
In-Reply-To: <16658.45979.672934.897388@montanaro.dyndns.org>
References: <16658.42673.542930.749915@montanaro.dyndns.org>
	<4112A968.2030401@v.loewis.de>
	<16658.45979.672934.897388@montanaro.dyndns.org>
Message-ID: <4112B7A8.5090100@v.loewis.de>

Skip Montanaro wrote:
> The platform is Solaris/Intel (2.8 and/or 2.9).  I'll have to get you more
> complete details tomorrow.  The involved folks are gone for the day.

Ok. I'll get back to a Solaris machine next week, so I can investigate
somewhat myself.

Of course, the rule is, strictly speaking, "no warnings on the major
platforms"; I'm uncertain whether Solaris still applies here. It is
clearly not feasible to have Python compile with no warning no matter
what the compiler or operating system is. The major platforms, of
course, are Windows, Linux, OS X (i.e. the ones we provide binaries
for).

Regards,
Martin
From edreamleo at charter.net  Fri Aug  6 00:43:42 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 00:43:51 2004
Subject: [Python-Dev] Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
Message-ID: <008901c47b3d$a89c17c0$6700a8c0@computer>

> - The @ character is used in at least two tools (Leo, IPython),
  and this is being considered as something bad, but not a show stopper;

I wonder how bad @ is, even for Leo.  Is there any consensus or opinions
about how many files will actually contain annotations?  If there aren't
many, people could change the delimiter for Leo directives from @ to @@ just
for those files :-)  Of course, Leo can't do this yet, but that's a nit...

Edward

P.S.  Oh yeah, just to make sure:  @@ won't be valid in annotations, will
it?

EKR
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From edreamleo at charter.net  Fri Aug  6 00:46:27 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 00:46:35 2004
Subject: [Python-Dev] Similar syntax
References: <20040805220743.GC31288@burma.localdomain>
Message-ID: <000701c47b3e$0b14d540$6700a8c0@computer>

>     <accepts(int,int), returns(float)>
>     def bar(low, high):
...
> - Leaves Leo and IPython people happy;

Yep.  No problem with Leo.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From marktrussell at btopenworld.com  Fri Aug  6 00:52:07 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Fri Aug  6 00:52:10 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112B5B4.9@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<002301c47b3a$469bc5f0$6700a8c0@computer>  <4112B5B4.9@v.loewis.de>
Message-ID: <1091746327.1419.18.camel@localhost>

On Thu, 2004-08-05 at 23:33, "Martin v. L?wis" wrote:
> it's just the PEP that needs updating (a section in whatsnew also needs
> to be drafted). Then, of course, the text saying that "@" is not used
> needs to go.

I have no experience in working on PEPs, but I'm willing to have a go if
nobody else has the time.

> I wonder whether replacing "@" with "!" or "%" would do any good...
> In fact, any binary operator would work - Objective-C does use "+"
> and "-" at the beginning of a line...

Good thought.  Or even something like:

	|staticmethod|
	def foo(): pass

I just tried converting test_decorators.py to this syntax, and it looks
quite nice to my eye.

Mark

From bob at redivi.com  Fri Aug  6 00:54:44 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Aug  6 00:54:51 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <008901c47b3d$a89c17c0$6700a8c0@computer>
References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<008901c47b3d$a89c17c0$6700a8c0@computer>
Message-ID: <71A2DB28-E732-11D8-9D9A-000A95686CD8@redivi.com>


On Aug 5, 2004, at 6:43 PM, Edward K. Ream wrote:

>> - The @ character is used in at least two tools (Leo, IPython),
>   and this is being considered as something bad, but not a show 
> stopper;
>
> I wonder how bad @ is, even for Leo.  Is there any consensus or 
> opinions
> about how many files will actually contain annotations?  If there 
> aren't
> many, people could change the delimiter for Leo directives from @ to 
> @@ just
> for those files :-)  Of course, Leo can't do this yet, but that's a 
> nit...

I think the idea is that Leo should escape the program code sort of 
like SQL escapes quotes.  What you have right now just sounds fragile.

@leo
@@pythondecorator
@leo

@leo
@@perlarray

> P.S.  Oh yeah, just to make sure:  @@ won't be valid in annotations, 
> will
> it?

No.

-bob
From nhodgson at bigpond.net.au  Fri Aug  6 00:58:16 2004
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Fri Aug  6 00:58:22 2004
Subject: [Python-Dev] Questions about '@' in pep 318
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de>
	<1091741561.8541.144.camel@localhost>
Message-ID: <028301c47b3f$b1b08bf0$034b8890@neil>

Barry Warsaw:

> Not that it should have any impact on the decision in any way, but
> the regular expression python-mode.el uses to syntax color pie
> decorators is much simpler (to figure out and to read) than the
> previous decorator syntax support of list-before-colon.

   I've been looking at how complex decorators can get and wondered what you
are actually highlighting as a decorator. Just the '@deco'? The line that
starts with '@' or a whole expression. Without an updated PEP it is hard to
work out what is possible but it seems that the @ can start a complex
multiline expression:

@NETSignature( \
    Package("Engine.Ignition"),
    ("Control.Argument.Tenuous.Debate",
    invisible("Marshalling.Infrastructure.Transport")))
def Reignite(debate):
    pass

   Neil

From tdelaney at avaya.com  Fri Aug  6 00:58:53 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Fri Aug  6 00:59:03 2004
Subject: [Python-Dev] A usability argument for list-after-def
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C481@au3010avexu1.global.avaya.com>

Aahz wrote:

> I've been staying out of this because I thought my preference had no
> chance to make it in (i.e. I don't care between "no decorators" and "@
> decorators"), but if it's getting some revival, I want to make clear
> that I think putting decorators *after* ``def``, name, and parameter
> list but clearly *with* the function definition makes the most sense
> for readability -- in the simple cases.  Who cares about the complex
> cases, because it's always going to be ugly?

+1

This has been my position all along. I understand the need for decorator
syntax (and even defended pie-syntax on c.l.py for that reason) but I
think that of all the alternatives, list after parameters, before colon
is the best alternative presented.

    def func (params) [decorators]:
        pass

To me this conveys the information I need clearly and in the order that
I need it - name is most important, parameters second, and the "type" of
function last.

The vast majority of functions are between 1 and 3 parameters.
Similarly, I doubt there will be many functions with more than a couple
of decorators.

Having everything on a single line in the majority of cases is IMO a big
win - both for readability and understandability.

Tim Delaney
From martin at v.loewis.de  Fri Aug  6 01:02:34 2004
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri Aug  6 01:02:37 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <55CC1464-E730-11D8-9D9A-000A95686CD8@redivi.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de>
	<55CC1464-E730-11D8-9D9A-000A95686CD8@redivi.com>
Message-ID: <4112BC8A.5030604@v.loewis.de>

> Except + and - would be valid Python syntax already ;)  

Ah, yes, the unary +. It's not clear to me why we need to have this -
the unary - is certainly necessary. So we still have * and / :-)

> Aesthetically I much prefer @ over ! or %.  !looks !bad !next !to !text 
> !because !it !doesn't !stand !out and %makes %me %think %of %comments 
> %in %another %language.

So we do need Unicode after all :-)

     ?classmethod
     def foo():
       pass

Regards,
Martin
From edreamleo at charter.net  Fri Aug  6 01:06:12 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 01:06:23 2004
Subject: [Python-Dev] Questions about '@' in pep 318
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de>
	<000f01c47b39$de3bcd70$6700a8c0@computer>
	<4112B6DC.4080605@v.loewis.de>
Message-ID: <000f01c47b40$cde05110$6700a8c0@computer>

> > I was using __future__ by way of explanation.  I do hope namespaces
could
> > somehow denote annotations.  My off-the-cuff suggestion was for
> > pseudo-modules, so maybe the normal module rules could be sidestepped?
>
> I don't see how this would be possible. The plan is that arbitrary
> callables can be used as decorations, as long as they take a single
> argument.

Ok.  Consider me dense.  But I'm just wanting something that _looks_ like a
module reference but isn't really.  What it is really is a stand-in for '@'.
Wouldn't this allow user-defined annotations, provided the compiler was in
on the joke?  In essence, what I am asking for is
just-another-name-for-at-sign.

So: just-another-name-for-at-sign.arbitrary-callable

Or maybe I should hope for <...> :-)

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From martin at v.loewis.de  Fri Aug  6 01:10:41 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 01:10:44 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
In-Reply-To: <ceuabl$9t3$1@sea.gmane.org>
References: <f8ef9ae404080512245b3e62a0@mail.gmail.com>	<ceu8sh$6j9$1@sea.gmane.org>	<4112A7CF.3030700@v.loewis.de>
	<ceuabl$9t3$1@sea.gmane.org>
Message-ID: <4112BE71.90402@v.loewis.de>

Nicolas Fleury wrote:
>>> accepts(int,int,def)
>>> returns(float,def)
>>> def bar(low,high):
>>
>>
>> That doesn't work. If accepts and returns are callables
>> (as they should be), then this already means something
>> in current Python. So this would not be backwards
>> compatible.
> 
> 
> I'm sorry (I don't like that solution at all honestly), but what passing 
> "def" to a callable means in current Python?  It is not accepted by my 
> interactive shell...

Right, I missed that point. In PEP 318, you can't define a decorator
that takes a Python keyword as an argument (what would that mean, 
anyway), so you don't need to find syntax for that, either.

If you meant to suggest that the last parameter indicates that this is
not a function call but a decorator, then you just had your
demonstration that this doesn't work: people are not going to recognize
the keyword in that place, even if the compiler could be talked into
accepting it.

Regards,
Martin
From bob at redivi.com  Fri Aug  6 01:13:14 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Aug  6 01:13:20 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112BC8A.5030604@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de>
	<55CC1464-E730-11D8-9D9A-000A95686CD8@redivi.com>
	<4112BC8A.5030604@v.loewis.de>
Message-ID: <07581E18-E735-11D8-9D9A-000A95686CD8@redivi.com>


On Aug 5, 2004, at 7:02 PM, Martin v. L?wis wrote:

>> Except + and - would be valid Python syntax already ;)
>
> Ah, yes, the unary +. It's not clear to me why we need to have this -
> the unary - is certainly necessary. So we still have * and / :-)
>
>> Aesthetically I much prefer @ over ! or %.  !looks !bad !next !to 
>> !text !because !it !doesn't !stand !out and %makes %me %think %of 
>> %comments %in %another %language.
>
> So we do need Unicode after all :-)
>
>     ?classmethod
>     def foo():
>       pass

In AppleScript-ese that might be

?classmethod?
def foo():
	pass

:)
From martin at v.loewis.de  Fri Aug  6 01:13:16 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 01:13:24 2004
Subject: [Python-Dev] Similar syntax
In-Reply-To: <20040805220743.GC31288@burma.localdomain>
References: <20040805220743.GC31288@burma.localdomain>
Message-ID: <4112BF0C.9050309@v.loewis.de>

Gustavo Niemeyer wrote:
> After writing this list, I thought about something which is
> close to the current implementation, but avoids @. If it was
> already suggested, please forgive me. At least I wasn't able
> to find it in the PEP/Wiki:
> 
>     <static>
>     def method(foo):
>         pass

This is discussed in the PEP:

The '<...>' alternative presents parsing problems because '<' and '>' 
already parse as un-paired. They present a further parsing ambiguity 
because a right angle bracket might be a greater than symbol instead of 
a closer for the decorators.

Regards,
Martin
From martin at v.loewis.de  Fri Aug  6 01:15:42 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 01:15:45 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <000f01c47b40$cde05110$6700a8c0@computer>
References: <004a01c47b2b$65d19260$6700a8c0@computer>	<4112A4B9.4010907@v.loewis.de>	<000f01c47b39$de3bcd70$6700a8c0@computer>	<4112B6DC.4080605@v.loewis.de>
	<000f01c47b40$cde05110$6700a8c0@computer>
Message-ID: <4112BF9E.40003@v.loewis.de>

Edward K. Ream wrote:
> Ok.  Consider me dense.  But I'm just wanting something that _looks_ like a
> module reference but isn't really.  What it is really is a stand-in for '@'.
> Wouldn't this allow user-defined annotations, provided the compiler was in
> on the joke?  In essence, what I am asking for is
> just-another-name-for-at-sign.

In essence, you are proposing to reserve a new keyword. Solutions 
creating new keywords have been ruled out because they might break
existing code.

Regards,
Martin
From guido at python.org  Fri Aug  6 01:24:59 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 01:25:06 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: Your message of "Fri, 06 Aug 2004 01:15:42 +0200."
	<4112BF9E.40003@v.loewis.de> 
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de>
	<000f01c47b39$de3bcd70$6700a8c0@computer>
	<4112B6DC.4080605@v.loewis.de>
	<000f01c47b40$cde05110$6700a8c0@computer> 
	<4112BF9E.40003@v.loewis.de> 
Message-ID: <200408052324.i75NOxA05160@guido.python.org>

> In essence, you are proposing to reserve a new keyword. Solutions 
> creating new keywords have been ruled out because they might break
> existing code.

Perhaps this could be addressed by requiring "from __future__ import
decorators", for one release, just like was done for "yield".  I
expect that this would be acceptable to the ObjC folks, too.  It
wouldn't be my favorite, but I won't rule it out just because of the
new keyword (and yes, this is a softening of my position on new
keywords).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From bac at OCF.Berkeley.EDU  Fri Aug  6 01:26:38 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug  6 01:26:48 2004
Subject: [Python-Dev] OK to lower timeout for test_timeout's
	testConnectTimeout test?
In-Reply-To: <1f7befae040805153126848c1@mail.gmail.com>
References: <4112A365.1000203@ocf.berkeley.edu>
	<1f7befae040805153126848c1@mail.gmail.com>
Message-ID: <4112C22E.6080805@ocf.berkeley.edu>

Tim Peters wrote:

> [Brett Cannon]
> 
>>I am actually getting failures from test_timeout's testConnectTimeout
> 
> 
> It would have helped if you had been specific about the failures
> you're seeing, and whether you get them all the time, or just some of
> the time.
> 

Sorry about that; almost filed a bug report and pasted it in there; just 
plain forgot to paste into here:

======================================================================
FAIL: testConnectTimeout (__main__.TimeoutTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "Lib/test/test_timeout.py", line 116, in testConnectTimeout
     self.addr_remote)
AssertionError: error not raised

----------------------------------------------------------------------

And it always fails.

> 
>>since the Net connection I have at school lets me connect to Google in
>>under the .001 connection timeout value.  If I push it to .00000001 (a
>>hundred-millionth of a second) it then fails consistently.
> 
> 
> What fails?  The test fails?  The socket fails to connect?
> 

The test.  No connection error is reported.

> Note that .0000001 is the same as passing, e.g., 1e-200.

I actually meant 1e-08, but apparently that is a moot point.

>  select()
> only accepts arguments at microsecond granularity, so any positive
> non-zero timeout value < 1e-6 gets chopped to exactly 0.0.
> 
> 	tv.tv_sec = (int)s->sock_timeout;
> 	tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6);
> 
> The test really doesn't intend to pass a timeout of exactly 0 to
> select(), so we can't change this to less than 1e-6.
> 

That's weird because if I do the test with sock.settimeout(0.0) and then 
sock.connect(('www.google.com', 80)) I get::

Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "<string>", line 1, in connect
socket.error: (36, 'Operation now in progress')

But if I do the exact same thing (new socket, etc.) after closing the 
previous one but with sock.settimeout(0.00000001) I get::

Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "<string>", line 1, in connect
socket.timeout: timed out

which is what the test expects.  Could this be a platform difference? 
Or is the conversion happening else where and thus not the equivalent of 
passing 0 to settimeout()?

> 
>>Now the problem is that the second part of the test uses this and a fuzz
>>value (currently at 2) to see if the test timed out within the proper
>>amount of time.  The comparison is basically if the amount of time it
>>took to do the timed out failure is less than timeout + fuzz.  So
>>lowering this number could possibly affect the test, although at .001
>>seconds, I am doubting that will occur.  But since these types of timing
>>tests can be touchy I thought I would double-check.
> 
> 
> Na, 2.0 is gigantic compared to .001 already.  For the purposes of the
> test, 2 isn't really more gigantic compared to 1e-6.
> 

I figured as much.

> 
>>So if anyone thinks it is bad to lower the value to .00000001 then
>>please let me know.
> 
> 
> Changing it to 1e-7 is out, for the reason explained earlier.  I'd
> like to keep .001, because while the interface to select() *allows*
> specifying microsecond resolution, there's no guarantee that it can
> use that much precision.  Most (all?) implementations should be able
> to deal with millisecond resolution, though.  Perhaps we could pick on
> something other than www.google.com.  Maybe www.python.org (everyone
> in America is far from that <wink>).

=)  I like that idea of changing of the site we hit.  Seems slightly 
rude to be eating Google's bandwidth with our regression suite when we 
have our own server we can pound (relatively speaking; obviously the 
test suite is not run by *that* many people).  I will go ahead and do that.

-Brett
From fumanchu at amor.org  Fri Aug  6 01:22:17 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Fri Aug  6 01:27:16 2004
Subject: [Python-Dev] A usability argument for list-after-def
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DE2@exchange.hqamor.amorhq.net>

Skip wrote:
>     >> So there it is, my personal argument for 
> list-after-def as better
>     >> than "@" ...
> 
>     aahz> +1
> 
> What he said.
> 
> +1.
> 
> I had simply given up arguing the point because it seemed 
> pointless and I
> was tired of arguing.  I suppose there are more than a few 
> people in this
> particular boat.

ObAOLMeToo.

+1

Good-thing-we're-NOT-voting-ly yours,


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From janssen at parc.com  Fri Aug  6 01:32:19 2004
From: janssen at parc.com (Bill Janssen)
Date: Fri Aug  6 01:32:41 2004
Subject: [Python-Dev] Call for defense of @decorators 
In-Reply-To: Your message of "Thu, 05 Aug 2004 11:26:42 PDT."
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com> 
Message-ID: <04Aug5.163225pdt."58612"@synergy1.parc.xerox.com>

> class ModelObject (NSObject):
> 
> 	@objc.accessor
> 	def messageLength(self):
> 		return "XIV"
> 
> 	@objc.signature("v@:@i")
> 	def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, 
> returnCode, contextInfo):
> 		pass

I'd prefer (and I don't remember what the issue about this was):

	def messageLength(self) as objc.accessor:
		return "XIV"

	def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet, 
returnCode, contextInfo) as objc.signature("v@:@i"):
		pass

or 

	def foo () as staticmethod:

I'm also happy with the

	def staticmethod foo():

proposal, but the point about breaking etags seems reasonable to me.
Maybe when the PEP captures the dscussion we can all read it and come
to a consensus, because we'll all understand what's wrong with the
non-pie proposals.

Bill
From shalabh at cafepy.com  Fri Aug  6 01:32:46 2004
From: shalabh at cafepy.com (Shalabh Chaturvedi)
Date: Fri Aug  6 01:32:54 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <20040805232522.A1B8E1E400A@bag.python.org>
References: <20040805232522.A1B8E1E400A@bag.python.org>
Message-ID: <52559.162.119.64.111.1091748766.squirrel@162.119.64.111>

I converted some code which makes good use decorators (an example follows):

    @paramnames(username='Username', password='Password')
    @webmethod(username=str, password=str)
    def login(self, username, password, return_to=None):
        request = current.request
        ...

Here is some usability feedback:

+ Easy to type. I found myself typing the function name before going up
one line and typing the decorators - @ decorator <ENTER> @ decorator
<ENTER>.

+ One line is one decorator - this preserves clarity while writing and
reading.

+ I look at the code and all decorators line up in a column just above the
function - very easily browsable and hard to miss. No searching or
scanning required.

+ Changing order or decorators (if it matters) is easy.

The only downside I found was:

- @ is a 'noisy' character and makes it look a little cluttered. Usually
def has an empty line before it but with decorators there is some funky
code sticking to it. Not a big deal, though.

Also, this symptom might get alleviated once editors know how to color
decorators nicely. An alternative character (as others have suggested)
might be good too. If so, I found '|' particularly nice :)

    |paramnames(username='Username', password='Password')
    |webmethod(username=str, password=str)
    def login(self, username, password, return_to=None):
        request = current.request
        ...

    |webmethod()
    def logout(self, return_to=None):
        current.request.session.user = None
        return current.request.redirect(return_to or self._q_location)

Appears accentuated, yet clean, specially for simple/single decorators.
Gives an appearance that the decorators are 'connected' to the function.

Conclusion: as a user I'm +1 on the @-decorator syntax, or any similar
syntax using an alternative character.

Cheers,
Shalabh

From fumanchu at amor.org  Fri Aug  6 01:35:18 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Fri Aug  6 01:40:18 2004
Subject: [Python-Dev] Call for defense of @decorators
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DE3@exchange.hqamor.amorhq.net>

Martin:
> So we do need Unicode after all :-)
> 
>      ?classmethod
>      def foo():
>        pass

And after that, we can replace 'def' with \u0402  >:)

  ?classmethod
  ? foo():
    pass


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From tdelaney at avaya.com  Fri Aug  6 01:48:29 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Fri Aug  6 01:48:45 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C4B4@au3010avexu1.global.avaya.com>

Shalabh Chaturvedi wrote:

>     |paramnames(username='Username', password='Password')
>     |webmethod(username=str, password=str)
>     def login(self, username, password, return_to=None):
>         request = current.request
>         ...
> 
>     |webmethod()
>     def logout(self, return_to=None):
>         current.request.session.user = None
>         return current.request.redirect(return_to or self._q_location)
> 
> Appears accentuated, yet clean, specially for simple/single
> decorators. 
> Gives an appearance that the decorators are 'connected' to the
> function. 

This looks remarkably nice to me for a syntax before the definition.
Astoundingly so. I still prefer list-before-colon, but the above would
alleviate much of my distate for @ which is that it doesn't look as
though the decorators are connected to the definition.

Hmm - perhaps a further variation ...

    |paramnames(username='Username', password='Password')
    |
    |webmethod(username=str, password=str)
    |
    def login(self, username, password, return_to=None):
        request = current.request

i.e. the decorators must be connected to the def, but you can use some
whitespace to make them look better (for decorators with long parameter
lists). Would be hell to parse though ...

Tim Delaney
From janssen at parc.com  Fri Aug  6 01:48:23 2004
From: janssen at parc.com (Bill Janssen)
Date: Fri Aug  6 01:48:49 2004
Subject: [Python-Dev] A usability argument for list-after-def 
In-Reply-To: Your message of "Thu, 05 Aug 2004 15:19:16 PDT."
	<16658.45668.851011.950341@montanaro.dyndns.org> 
Message-ID: <04Aug5.164830pdt."58612"@synergy1.parc.xerox.com>

> 
>     >> So there it is, my personal argument for list-after-def as better
>     >> than "@" ...
> 
>     aahz> +1
> 
> What he said.
> 
> +1.
> 
> I had simply given up arguing the point because it seemed pointless and I
> was tired of arguing.  I suppose there are more than a few people in this
> particular boat.
> 
> Skip

Ditto.

+1.

Bill
From barry at python.org  Fri Aug  6 01:54:53 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug  6 01:54:47 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <028301c47b3f$b1b08bf0$034b8890@neil>
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de> <1091741561.8541.144.camel@localhost>
	<028301c47b3f$b1b08bf0$034b8890@neil>
Message-ID: <1091750093.8545.162.camel@localhost>

Note: Reply-To set to python-mode@python.org

On Thu, 2004-08-05 at 18:58, Neil Hodgson wrote:

>    I've been looking at how complex decorators can get and wondered what you
> are actually highlighting as a decorator. Just the '@deco'? The line that
> starts with '@' or a whole expression.

It's damn simple: on a line where the first non-whitespace character
(specifically, not space or tab) is an '@', then everything from there
to the end of the line.

It's more difficult to specify multi-line font-lock regular expressions
in X/Emacs, and at least XEmacs has a hard time font-locking multiple
lines on the fly, so I don't intend to figure that out.  Patches are
welcome if they work on both Emacs and XEmacs.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040805/4c5193f9/attachment.pgp
From pje at telecommunity.com  Fri Aug  6 02:14:37 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug  6 02:10:34 2004
Subject: [Python-Dev] Call for defense of @decorators 
In-Reply-To: <04Aug5.163225pdt."58612"@synergy1.parc.xerox.com>
References: <Your message of "Thu, 05 Aug 2004 11:26:42 PDT."
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <5.1.1.6.0.20040805201247.01edcd50@mail.telecommunity.com>

At 04:32 PM 8/5/04 -0700, Bill Janssen wrote:
>I'd prefer (and I don't remember what the issue about this was):
>
>         def messageLength(self) as objc.accessor:
>                 return "XIV"
>
>         def saveSheetDidDismiss_returnCode_contextInfo_(self, sheet,
>returnCode, contextInfo) as objc.signature("v@:@i"):
>                 pass
>
>or
>
>         def foo () as staticmethod:
>
>I'm also happy with the
>
>         def staticmethod foo():
>
>proposal, but the point about breaking etags seems reasonable to me.
>Maybe when the PEP captures the dscussion we can all read it and come
>to a consensus, because we'll all understand what's wrong with the
>non-pie proposals.

See http://www.python.org/moin/PythonDecorators for a current pros-and-cons 
tally on the syntaxes you referenced.

From greg at cosc.canterbury.ac.nz  Fri Aug  6 02:25:59 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug  6 02:26:06 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <2mhdrh96nd.fsf@starship.python.net>
Message-ID: <200408060025.i760Pxn9000712@cosc353.cosc.canterbury.ac.nz>

> > Seems to me the point at which we start allowing new-style classes as
> > exceptions should also be the point at which we drop the idea of
> > string exceptions. Would that help?
> 
> It would probably make things a little simpler, but probably not in a
> major way.

I was thinking it might avoid the need to enforce a common
base class for exceptions, since it would remove the ambiguity
of whether 'raise "spam"' is raising a string exception or
an instance of class str.

But if it's considered a good idea to enforce a common
root anyway, I guess it doesn't make much difference.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From fperez528 at yahoo.com  Fri Aug  6 02:30:20 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Fri Aug  6 02:30:27 2004
Subject: [Python-Dev] Re: Similar syntax
References: <20040805220743.GC31288@burma.localdomain>
Message-ID: <ceujes$ql4$1@sea.gmane.org>

Gustavo Niemeyer wrote:

>     <static>
>     def method(foo):
>         pass
> 
> and also
> 
>     <accepts(int,int), returns(float)>
>     def bar(low, high):
>         ...
> 
> Advantages over current syntax:
> 
> - Avoids introducing a new meaningful character in the syntax;
> 
> - Leaves Leo and IPython people happy;

Yup (from the ipython side).

Best,

Fernando

ps.  I think I recall this havnig been discarded, but since it's not in the
PEP, I don't really remember where.  It may even be my memory playing tricks
on me.



From greg at cosc.canterbury.ac.nz  Fri Aug  6 02:32:32 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug  6 02:32:38 2004
Subject: [Python-Dev] Python in Unicode context
In-Reply-To: <20040805112928.GA5208@solar.trillke>
Message-ID: <200408060032.i760WWXw000745@cosc353.cosc.canterbury.ac.nz>

> Then i guess that searching down into a recursive structure and just 
> raising an "i found it" result object up doesn't count as a use case in
> your book, right?

You can always wrap the object you want to return in a suitable
subclass of Exception. I would prefer to write it that way anyway,
otherwise I'd need some sort of "catch anything which *isn't* a
subclass of Exception" statement, which is rather awkward to spell.

If this is considered a common enough requirement, a special built-in
Exception subclass could even be provided for it, to save people the
bother of defining their own...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Fri Aug  6 02:35:05 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug  6 02:35:13 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <2mvffx7lw3.fsf@starship.python.net>
Message-ID: <200408060035.i760Z5PE000754@cosc353.cosc.canterbury.ac.nz>

> >> class C(random.choice([dict, list])):
> >>     pass
> >>
> >> is my favourite example of this :-)
> >
> > Where is THAT monstrocity used!?
> 
> Well, it's not, I hope :-)

Hmmm... maybe if you had some algorithm that was more efficient
using either a list or a dict, depending on the input data, but
in a way that makes it difficult to tell ahead of time which
to use, and you want to improve the average behaviour...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From fperez528 at yahoo.com  Fri Aug  6 02:39:41 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Fri Aug  6 02:39:48 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
Message-ID: <ceuk0e$rut$1@sea.gmane.org>

Gustavo Niemeyer wrote:

> - The @ character is used in at least two tools (Leo, IPython),
>   and this is being considered as something bad, but not a
>   show stopper;

Just to record my comments here as the ipython author.  

I've held off on calling on the ipython lists for suggestions on what to
replace @ with, waiting for the dust to settle on this matter.

If @ ends up being accepted, ipython can obviously adapt.  I'll replace it with
alternative syntax, be it @@, %, or some other kind of special-casing trick.

It would not make me happy, and it worries me that ipython explicitly uses (for
good reasons, I like to think), the three characters explicitly avoided by
python:

@ -> ipython's 'magic' control system, which is fully user-extensible.

? -> ipython's object introspection system, with ?? showing additional info.

$ -> in ipython's shell extensions, a syntax for quick shell output capture,
either into strings ($astr=ls *.py) or lists ($$alist=ls *.py).

I worry that soon I'll end up rewriting other parts of ipython as soon as ? and
$ also become python characters.

But I accept that fact, since nowhere did the docs ever _promise_ that these
characters would remain off-limits forever.  I knew there was a risk involved,
and if that happens, I'll figure out a way to solve it.  But I'd much rather
not.

Best,

Fernando.

From greg at cosc.canterbury.ac.nz  Fri Aug  6 02:39:43 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug  6 02:39:51 2004
Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python in
	Unicode context)
In-Reply-To: <20040805153443.GB5208@solar.trillke>
Message-ID: <200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz>

> My point is that I like to regard try/except as a mechanism for
> "out-of-band" objects.  Guidos "should be shot" seems to indicate he
> sees try/except only useful/applicable to exception-handling.

If the root exception class were called something else,
such as 'Raisable', would that make you feel better?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From art at wiktorsadowski.com  Fri Aug  6 02:52:19 2004
From: art at wiktorsadowski.com (Wiktor Sadowski)
Date: Fri Aug  6 02:52:05 2004
Subject: [Python-Dev] Decorator syntax proposal
Message-ID: <04ac01c47b4f$a08c2b30$0201010a@kret>

FWIW:

[def][decorateMethod1(param)]
[def][decorateMethod2(param)]
def method(self,param):
        pass

or all in one line

[def][decorateFunc1(param)][decorateFunc2(param)]
def func(param):
        pass


[class][decorateClass(param)]
class toBeDecorated:
    pass

import hooks

[import][modify(mod)]
import foo

Regards
Wiktor Sadowski
From tim.peters at gmail.com  Fri Aug  6 03:22:28 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug  6 03:22:38 2004
Subject: [Python-Dev] OK to lower timeout for test_timeout's
	testConnectTimeout test?
In-Reply-To: <4112C22E.6080805@ocf.berkeley.edu>
References: <4112A365.1000203@ocf.berkeley.edu>
	<1f7befae040805153126848c1@mail.gmail.com>
	<4112C22E.6080805@ocf.berkeley.edu>
Message-ID: <1f7befae04080518224f886206@mail.gmail.com>

[Brett Cannon]
> That's weird because if I do the test with sock.settimeout(0.0) ...

That's very different.  If you set the socket timeout to exactly 0,
Python never calls select().  If you set it to anything > 0, Python
does call select(), but may truncate the timeout to exactly 0 in order
to call select.

> and then sock.connect(('www.google.com', 80)) I get::
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "<string>", line 1, in connect
> socket.error: (36, 'Operation now in progress')
>
> But if I do the exact same thing (new socket, etc.) after closing the
> previous one but with sock.settimeout(0.00000001) I get::
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "<string>", line 1, in connect
> socket.timeout: timed out

It will all be obvious <wink> if you read sock_connect().  In
particular, the message you got there is produced by

	if (timeout) {
		PyErr_SetString(socket_timeout, "timed out");
		return NULL;
	}

and you'll find that it's impossible for that to trigger if you set
the socket timeout to exactly 0.  I'd call that a bug, except it's not
worth anyone's time to fix ...
From anthony at interlink.com.au  Fri Aug  6 02:12:15 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 03:36:54 2004
Subject: [Python-Dev] trunk is no longer frozen
In-Reply-To: <41112923.8060505@interlink.com.au>
References: <41112923.8060505@interlink.com.au>
Message-ID: <4112CCDF.7030609@interlink.com.au>

I'm done with the trunk. Please feel free to check in again.

From anthony at interlink.com.au  Fri Aug  6 03:21:32 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 03:38:33 2004
Subject: [Python-Dev] PEP 318, and the PEP process
Message-ID: <4112DD1C.3060902@interlink.com.au>

I'm extremely averse to making the Python development
process any heavier than it has to be, but the complaint
that PEP 318 should have been updated to the state of
play before the checkin is a fair complaint. I'm not
sure what the best approach here is - should we make
a motherhood-and-apple-pie statement that, all things
being equal, a PEP should be updated before the code
it documents is checked in? Should we aim to have it
done before the first release including the code? Before
the first _final_ release that includes the code?

My answers would be "before checkin, where possible;
before first alpha/beta release, where really possible;
before first final release, absolutely".

Having said that, I don't think the lack of completed
PEP is a reason to back out the @ syntax from CVS. If
nothing else, it being present in a released alpha is
giving us very real experience with the use of the
feature. There's also the issue that there's a bunch
of other existing PEPs describing features that aren't
up to date at all.

Which brings up another related point: Many PEPs
contain "open issues" and the like, that are never
ever backfilled. Largely this is a matter of round
tuits - the PEP authors are generally drawn from a
very small group of developers. How can we encourage
other people to contribute to this? The obvious way
(to me) is some form of Wiki annotation. I don't
think we want to make the Wiki the primary copy of
the document, but I think having the PEPs annotatable
would be a win.


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Fri Aug  6 03:21:21 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 03:38:35 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
Message-ID: <4112DD11.1060006@interlink.com.au>

Phillip J. Eby wrote:
> Guido's principal argument against list-after-def, if I recall 
> correctly, was that it is ugly when multiple or lengthy decorators are 
> involved.  But, "ugly" isn't an argument any more, so that shouldn't 
> rule out list-after-def.  :)

I think you're confusing "ugly" (the @ form) with "potentially hidden
several lines down" (list-after-def). My beef with list-after-def is
that it's easy to overlook. I suspect in many cases, the decorating of
a function or method will fundamentally alter the behaviour of the
function or method, so in that case, I'd prefer to make it as obvious
as possible.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Fri Aug  6 03:21:54 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 03:38:46 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408051636.i75Gaak03654@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
Message-ID: <4112DD32.5080302@interlink.com.au>

My take on the @decorator syntax is that after initially
hating it, I've found it growing on me more and more.
It's extremely clear and very readable. Most importantly,
in my eyes, it's obviously grouped with the def that
follows it. From being a -0 (when Guido first mentioned
it) to a +0 (at the time of checkin) I'm now +1 on this
form of decorator.

(It's different in that way to "print >>", which I still
hate <wink>)

A couple of other arguments that I've seen:

"This is a backwards-incompatible change to Python's syntax."
So are generator expressions, and they're way harder to spot
in a piece of code. Python syntax changes over time, that's
always been how it is.

"The @ sign is used in tool X."
This is disappointing, but it can't be the sole reason to
not use @ - Python can't be constrained to not breaking any
other existing third-party tools. Hopefully there's a
solution to the Leo problem - perhaps doubling the @ sign
for Leo?

"The @ sign is used in Perl."
And?


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Fri Aug  6 03:21:47 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 03:38:55 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112B5B4.9@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>
Message-ID: <4112DD2B.5090403@interlink.com.au>

Martin v. L?wis wrote:
> does explain syntax and semantics, and there are testcases as well. So 
> it's just the PEP that needs updating (a section in whatsnew also needs
> to be drafted). 

Andrew's already done this, a couple of days ago.

As far as the rest of the documentation - yes, absolutely. But a change
this deep can't be expected to find all of the doc places that need
updating in a single go. :-/



-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Fri Aug  6 03:21:41 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 03:38:57 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112A18D.5090801@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de
Message-ID: <4112DD25.2010105@interlink.com.au>

Martin v. L?wis wrote:
> Or you could argue on a procedural basis: regardless of whether
> the feature is good or bad, the current implementation is
> unacceptable, as the PEP does not correspond with the
> implementation, the syntax is undocumented, the code has no test
> cases, and so on. I'm actually going to do that, because I do
> think the process is unacceptable, and should be either corrected
> or reversed (of course, this says nothing about the feature itself,
> or the code implementing it).

Note that @decorators are hardly unique in not having an up-to-date
PEP. Where it's different to the other cases is that they're rather
controversial, and therefore it's more obvious that there's a
problem.


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.


From dave at pythonapocrypha.com  Fri Aug  6 03:59:30 2004
From: dave at pythonapocrypha.com (Dave Brueck)
Date: Fri Aug  6 03:59:41 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <4112DD1C.3060902@interlink.com.au>
References: <4112DD1C.3060902@interlink.com.au>
Message-ID: <4112E602.4050009@pythonapocrypha.com>

Anthony Baxter wrote:

> Having said that, I don't think the lack of completed
> PEP is a reason to back out the @ syntax from CVS. If
> nothing else, it being present in a released alpha is
> giving us very real experience with the use of the
> feature.

If I may, I'd like to chime in with an "I agree". My understanding is 
that the decorator change was checked in for the very reason you state - 
so that people can actually try it out instead of just talking about it. 
If that is the case, then having the PEP up to date before an alpha 
release would certainly be nice but doesn't seem to be a hard requirement.

(FWIW, it seems that a lot of the uproar about the syntax change is that 
it is being interpreted as a permanent change to the language rather 
than just experimental functionality. I _think_ I read here on 
python-dev that it was semi-experimental, but if that's the case I'm not 
sure how well that message got out - e.g. the "What's New in Python 
2.4a2" section of NEWS.html doesn't reflect this)
From greg at cosc.canterbury.ac.nz  Fri Aug  6 04:07:42 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug  6 04:07:48 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com>
Message-ID: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz>

"Phillip J. Eby" <pje@telecommunity.com>:

> I updated the Wiki page, and added another syntax, the original "def 
> function() [decorator]:" syntax.  Interestingly, it looks like one of the 
> options with the most pluses and fewest minuses of any syntax you've listed 
> on that page; it's only ugly for long decorator definitions, and Guido said 
> he didn't care if a syntax was ugly.  :)

Moreover, it seems to me that the main use cases proposed for long
decorator expressions would be served just as well, or perhaps even
better, by a convenient way of specifying function attributes,
together with a suitable metaclass.

  def aMethodWithAnInt_andAString(self, x, y):
    @signature = "is"
    ...

To me, that seems like a more logical place to put something like a
signature for an external function interface. It's a detail of
interest to the implementor of the function, but not to its user, so
putting it above the "def" gives it too much prominence. 

Also, putting anything up there (especially something long) tends to
interfere with one's ability to scan down a list of defs looking for a
function name.

I hope Guido will see these arguments as being rational ones about
usability and not irrational ones about aesthetics. At the least,
perhaps they could be included in the PEP for posterity.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From jhylton at gmail.com  Fri Aug  6 04:09:07 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Fri Aug  6 04:09:08 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <4112DD11.1060006@interlink.com.au>
References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
	<4112DD11.1060006@interlink.com.au>
Message-ID: <e8bf7a5304080519096ee585c7@mail.gmail.com>

On Fri, 06 Aug 2004 11:21:21 +1000, Anthony Baxter
<anthony@interlink.com.au> wrote:
> Phillip J. Eby wrote:
> > Guido's principal argument against list-after-def, if I recall
> > correctly, was that it is ugly when multiple or lengthy decorators are
> > involved.  But, "ugly" isn't an argument any more, so that shouldn't
> > rule out list-after-def.  :)
> 
> I think you're confusing "ugly" (the @ form) with "potentially hidden
> several lines down" (list-after-def). My beef with list-after-def is
> that it's easy to overlook. I suspect in many cases, the decorating of
> a function or method will fundamentally alter the behaviour of the
> function or method, so in that case, I'd prefer to make it as obvious
> as possible.

Since this thread has become something of a poll anyway, I'll simply
add: I agree.

Jeremy
From djc at object-craft.com.au  Fri Aug  6 04:15:07 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Fri Aug  6 04:15:14 2004
Subject: [Python-Dev] trunk is no longer frozen
In-Reply-To: <4112CCDF.7030609@interlink.com.au>
References: <41112923.8060505@interlink.com.au>
	<4112CCDF.7030609@interlink.com.au>
Message-ID: <4112E9AB.9060003@object-craft.com.au>

Anthony Baxter wrote:
> I'm done with the trunk. Please feel free to check in again.

Although I have checkin permission on CVS I would appreciate it if 
someone could look at my socketpair() patch before I do check it in...

Could people who care about the addition of socketpair() to the socket 
module please have a look at patch #1003700.  The patch is relative to 
2.4a1 but applies cleanly to 2.4a2 with slight offsets.

- Dave

-- 
http://www.object-craft.com.au
From greg at cosc.canterbury.ac.nz  Fri Aug  6 04:15:30 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug  6 04:15:34 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <5.1.1.6.0.20040805155039.022d4cc0@mail.telecommunity.com>
Message-ID: <200408060215.i762FUeY000881@cosc353.cosc.canterbury.ac.nz>

"Phillip J. Eby" <pje@telecommunity.com>:

> That argument has been done to death several times in the last year 
> here.  Function attributes aren't a replacement for decorators.

Just to be clear, I wasn't arguing in my last post that decorators
should be replaced by function attributes. I was questioning the
assumption that "use cases exist for long decorators, therefore any
syntax for decorators needs to accommodate them".

In other words, a syntax for short decorators plus a syntax for long
function arguments might be sufficient.

There might even be a proof of sorts for this: arguments to the
decorator can be substituted with attributes on the function about to
be decorated, which the decorator extracts.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From python at rcn.com  Thu Aug  5 16:15:49 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug  6 04:17:39 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <4112DD1C.3060902@interlink.com.au>
Message-ID: <00e001c47af6$b594fc40$e841fea9@oemcomputer>

> I'm extremely averse to making the Python development
> process any heavier than it has to be, but the complaint
> that PEP 318 should have been updated to the state of
> play before the checkin is a fair complaint. I'm not
> sure what the best approach here is - should we make
> a motherhood-and-apple-pie statement that, all things
> being equal, a PEP should be updated before the code
> it documents is checked in? Should we aim to have it
> done before the first release including the code? Before
> the first _final_ release that includes the code?
> 
> My answers would be "before checkin, where possible;
> before first alpha/beta release, where really possible;
> before first final release, absolutely".

That sounds reasonable.  Also, it reflects an understanding that "where
possible" sometimes means that:
- the maintainers are temporarily out of cycles,
- the discussion is on-going and not fully resolved,
- the real docs are more important that the PEP, and
- at some point, the implementation leads rather than lags the PEP.

In the case decorators, it sounds like all of these apply.  Also, it
would seem that the primary interest in the pep is to frame on going
arguments.  IOW, this would not be the last update.

With genexps, the information for the final pep update was not even
available until the patch discussion and newsgroup discussion wound
down.  Brett can attest to the difficulty of summarizing a voluminous
discussion that has not reached resolution.  Interestingly, after the
genexp update, no one seemed to care about or even read it.  Everything
they wanted to know had already been discussed ad nauseum or written it
the docs.

One other thought is that peps implemented in phases seem to get
neglected.  I don't think either Sets or Unifying Int/Long is current.



Raymond

From jhylton at gmail.com  Fri Aug  6 04:18:53 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Fri Aug  6 04:18:55 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <4112DD1C.3060902@interlink.com.au>
References: <4112DD1C.3060902@interlink.com.au>
Message-ID: <e8bf7a5304080519182f7263f0@mail.gmail.com>

On Fri, 06 Aug 2004 11:21:32 +1000, Anthony Baxter
<anthony@interlink.com.au> wrote:
> I'm extremely averse to making the Python development
> process any heavier than it has to be, but the complaint
> that PEP 318 should have been updated to the state of
> play before the checkin is a fair complaint. I'm not
> sure what the best approach here is - should we make
> a motherhood-and-apple-pie statement that, all things
> being equal, a PEP should be updated before the code
> it documents is checked in? Should we aim to have it
> done before the first release including the code? Before
> the first _final_ release that includes the code?

My answer would be before checkin unless there is widespread consensus
or there's only one obvious way to do it.  Developer time is a scarce
resource.  Checking in code without having a decent PEP, particularly
for major changes or new features, just wastes everyone's time.  We
spend hundreds of hours, collectively, repeating old debates and
speculating about why something was done the way it was.  Since no one
(?) is doing Python development as a day job, it's particularly
important that we make effective use of our spare time.

A PEP makes the development process more heavyweight -- but that was
its primary goal.  We wanted to make sure the specification and
rationale got written ahead of time.  Then other developers would know
what's happening, the spec would actually provide the new words for
the language ref (do we have that for decorators or genexps?), and
we'd be sure that the developer had thought through all the issues.

Jeremy
From anthony at interlink.com.au  Fri Aug  6 04:18:55 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 04:19:10 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <4112E602.4050009@pythonapocrypha.com>
References: <4112DD1C.3060902@interlink.com.au>
	<4112E602.4050009@pythonapocrypha.com>
Message-ID: <4112EA8F.9030106@interlink.com.au>

Dave Brueck wrote:
> (FWIW, it seems that a lot of the uproar about the syntax change is that 
> it is being interpreted as a permanent change to the language rather 
> than just experimental functionality. I _think_ I read here on 
> python-dev that it was semi-experimental, but if that's the case I'm not 
> sure how well that message got out - e.g. the "What's New in Python 
> 2.4a2" section of NEWS.html doesn't reflect this)

I thought about adding something to the release announcement about
this, but decided against it, as I felt it wouldn't help the process,
but would instead lead to even more complaints. My take on this was
that the @syntax was put in 2.4a2, and was experimental only in that
if there was a better solution to come up, we'd use that. That is,
the default action was that it would stay in. Guido, or anyone else,
can chime in and let me know if I've mis-stated this.


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From greg at cosc.canterbury.ac.nz  Fri Aug  6 04:24:02 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug  6 04:24:07 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <200408060224.i762O2Vc000926@cosc353.cosc.canterbury.ac.nz>

Ronald Oussoren <ronaldoussoren@mac.com>:

> And how would that solve this: 
> <http://www.eby-sarna.com/pipermail/peak/2004-July/001598.html>

FWIW, in decorator-plus-function-attributes style, this
would be

  def do_something(x,y) [generic]:
    @when = "x in int and y in str"
    # body here will be executed when 'x'
    # is an integer and 'y' is a string

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From anthony at interlink.com.au  Fri Aug  6 04:29:21 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 04:29:37 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112DD2B.5090403@interlink.com.au>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>
Message-ID: <4112ED01.6010304@interlink.com.au>

I wrote:
> Martin v. L?wis wrote:
> 
>> does explain syntax and semantics, and there are testcases as well. So 
>> it's just the PEP that needs updating (a section in whatsnew also needs
>> to be drafted). 
> 
> 
> Andrew's already done this, a couple of days ago.

I meant to include this link:

http://www.python.org/dev/doc/devel/whatsnew/node5.html



-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From greg at cosc.canterbury.ac.nz  Fri Aug  6 04:35:08 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug  6 04:35:13 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
Message-ID: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz>

"Phillip J. Eby" <pje@telecommunity.com>:

> Now, I don't know if my thought process here is at all typical or 
> representative of anything, other than my own idiosyncratic self.

You're not alone. I agree with *everything* you said there.

All of that stuff should go in the PEP, if it's not there
already.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From niemeyer at conectiva.com  Fri Aug  6 04:37:34 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Fri Aug  6 04:35:16 2004
Subject: [Python-Dev] Similar syntax
In-Reply-To: <4112BF0C.9050309@v.loewis.de>
References: <20040805220743.GC31288@burma.localdomain>
	<4112BF0C.9050309@v.loewis.de>
Message-ID: <20040806023734.GA7230@burma.localdomain>

> Gustavo Niemeyer wrote:
> >After writing this list, I thought about something which is
> >close to the current implementation, but avoids @. If it was
> >already suggested, please forgive me. At least I wasn't able
> >to find it in the PEP/Wiki:
> >
> >    <static>
> >    def method(foo):
> >        pass
> 
> This is discussed in the PEP:
> 
> The '<...>' alternative presents parsing problems because '<' and '>' 
> already parse as un-paired. They present a further parsing ambiguity 
> because a right angle bracket might be a greater than symbol instead of 
> a closer for the decorators.

Gotcha! Thanks for pointing me this.

-- 
Gustavo Niemeyer
http://niemeyer.net
From bob at redivi.com  Fri Aug  6 04:38:25 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Aug  6 04:38:35 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz>
References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz>
Message-ID: <B1223238-E751-11D8-8338-000A95686CD8@redivi.com>


On Aug 5, 2004, at 10:07 PM, Greg Ewing wrote:

> "Phillip J. Eby" <pje@telecommunity.com>:
>
>> I updated the Wiki page, and added another syntax, the original "def
>> function() [decorator]:" syntax.  Interestingly, it looks like one of 
>> the
>> options with the most pluses and fewest minuses of any syntax you've 
>> listed
>> on that page; it's only ugly for long decorator definitions, and 
>> Guido said
>> he didn't care if a syntax was ugly.  :)
>
> Moreover, it seems to me that the main use cases proposed for long
> decorator expressions would be served just as well, or perhaps even
> better, by a convenient way of specifying function attributes,
> together with a suitable metaclass.
>
>   def aMethodWithAnInt_andAString(self, x, y):
>     @signature = "is"
>     ...
>
> To me, that seems like a more logical place to put something like a
> signature for an external function interface. It's a detail of
> interest to the implementor of the function, but not to its user, so
> putting it above the "def" gives it too much prominence.
>
> Also, putting anything up there (especially something long) tends to
> interfere with one's ability to scan down a list of defs looking for a
> function name.
>
> I hope Guido will see these arguments as being rational ones about
> usability and not irrational ones about aesthetics. At the least,
> perhaps they could be included in the PEP for posterity.

While using metaclasses makes a lot of things possible, I don't believe 
they are appropriate for this purpose.  Decorating functions and 
methods have nothing to do with the class, why should the class have to 
change in order to support it?  What about bare functions?  Metaclasses 
can become difficult to use and understand when you're using several of 
them at once.

If you look at it from another implementation standpoint, reasonable 
syntax for function attributes can be implemented rather easily with 
function decorators, but vice versa requires the use of metaclasses.  I 
think implementing decorators is the more reasonable choice.  Also 
consider how difficult it can be to integrate metaclasses into existing 
code, where decorators would not have this issue at all.

-bob
From greg at cosc.canterbury.ac.nz  Fri Aug  6 04:41:53 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug  6 04:41:57 2004
Subject: [Python-Dev] The impact of '@' on Leo
In-Reply-To: <005e01c47b2d$9c0dee80$6700a8c0@computer>
Message-ID: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz>

"Edward K. Ream" <edreamleo@charter.net>:

> I was always under the impression that the clear statement in the
> Reference Manual that at signs are invalid everywhere (except in
> comments and strings) was a clear signal of an intention to keep those
> symbols for other valid purposes.

I've only ever regarded that as a statement about the status
quo, not any kind of promise about the future.

But maybe it would be a good idea to explicitly reserve some
character (perhaps only at the beginning of a line).

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From adurdin at gmail.com  Fri Aug  6 04:46:57 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Fri Aug  6 04:47:00 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
Message-ID: <59e9fd3a04080519462bf57899@mail.gmail.com>

(This is a repeat of a message on python-list/c.l.p in the "Tweaking
@decorator syntax" thread, but I thought it was applicable to post it
here, based on how I understand Guido's message. However, this is not
a defense but an attack on @decorators, and based not on whether it
looks ugly or not but that it is unclear.)

On 5 Aug 2004 16:04:00 -0700, Sandy Norton <sandskyfly@hotmail.com>
wrote in c.l.p, thread "Tweaking @decorator syntax":
> Here are some more variations.
> 
> Please copy/paste the code below into your favourite python editor and see
> how each syntactical variation of the decorator code reads, and how quickly
> you grok its structure and meaning.

*snip examples*

Of the examples above, I find the "Nested, 'decorate' keyword syntax"
by far the most clear. It has the following features (in no particular
order) which I find attractive:

1. The nesting makes it abundantly clear what the decoration applies
to, namely the function definitions at the next indentation level. And
the colon at the end of the "decorate" line is consistent with all
other Python keywords which result in another indentation level (def,
if, class, etc.) -- which is one reason why I dislike the "nested
@decorator" and "nested |decorator|" options. The "Nested, minimal
decorator syntax" has indentation, but lacks clarity: it appears
merely to be one or a sequence of function calls followed inexplicably
by a nested block.

2. The nesting makes it convenient to apply the same decoration to
multiple functions, which I guess will be fairly common wherever
decorators are used.

3. It doesn't involve adding any additional punctuation to the
language. I strongly believe that the general lack of punctuation
contributes greatly towards Python's readability.

4. The comma-separated list of decorators allows for more compactness
without sacrificing legibility when multiple decorators are used.


Whereas I quite dislike the @ syntax, for almost opposite reasons:

5. In appearance it looks like a special kind of statement; and it is
not at all obvious to me that it is rather a kind of attribute or
modifier of the following function definition. I don't know of any
other construction in Python that performs a similar, unmarked (i.e.
non-explicit) modification of the following statement. So to me the
@syntax comes across as extremely un-Pythonic (by that I mean
radically inconsistent with the conventions of the rest of the
language). For this same reason, I dislike all the proposed syntaxes
which involve placing something before the "def" without a subsequent
indented block.

6. When using the same sets of decorators for multiple
functions/methods in the same module/class, the @ syntax is
inconvenient, as the decorators need to be repeated for each function.
(Of course, the current (2.3) syntax to achieve the same result is
similarly inconvenient; the @ syntax adds no benefit in this regard,
but the "decorate" syntax does).

7. The @ syntax adds more punctuation to Python's syntax: a Bad Thing.
Despite the risk of breaking compatibility with some older programs, I
firmly believe that adding new keywords is a much better idea in
general than adding new punctuation, because (a) keywords are more
readable (especially to those unfamiliar with them), and (b) there is
a very limited supply of punctuation marks on the keyboard, but a very
large supply of words.

8. When using multiple decorators, each appears on a separate line.
This not only confuses things with respect to point 5 above (e.g. does
the first decorator somehow decorate the second, or do both apply to
the function?), but also is IMO unnecessarily lengthy, as each line
has the same basic purpose; In analogy, the latter is similar to the
difference between
    x = 1
    y = 2
    z = 3
and
   x, y, z = 1, 2, 3
where a compact representation is possible and suitable where the
three variables are relating to the same purpose within the program.

-- Andrew Durdin
From bob at redivi.com  Fri Aug  6 04:47:25 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Aug  6 04:47:33 2004
Subject: [Python-Dev] The impact of '@' on Leo
In-Reply-To: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz>
References: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz>
Message-ID: <F28C03D1-E752-11D8-8338-000A95686CD8@redivi.com>


On Aug 5, 2004, at 10:41 PM, Greg Ewing wrote:

> "Edward K. Ream" <edreamleo@charter.net>:
>
>> I was always under the impression that the clear statement in the
>> Reference Manual that at signs are invalid everywhere (except in
>> comments and strings) was a clear signal of an intention to keep those
>> symbols for other valid purposes.
>
> I've only ever regarded that as a statement about the status
> quo, not any kind of promise about the future.
>
> But maybe it would be a good idea to explicitly reserve some
> character (perhaps only at the beginning of a line).

You mean like # ? :)

-bob

From skip at pobox.com  Fri Aug  6 04:48:00 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 04:48:33 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <4112DD1C.3060902@interlink.com.au>
References: <4112DD1C.3060902@interlink.com.au>
Message-ID: <16658.61792.917900.348835@montanaro.dyndns.org>


    Anthony> Which brings up another related point: Many PEPs contain "open
    Anthony> issues" and the like, that are never ever backfilled. Largely
    Anthony> this is a matter of round tuits - the PEP authors are generally
    Anthony> drawn from a very small group of developers. How can we
    Anthony> encourage other people to contribute to this? The obvious way
    Anthony> (to me) is some form of Wiki annotation. I don't think we want
    Anthony> to make the Wiki the primary copy of the document, but I think
    Anthony> having the PEPs annotatable would be a win.

For topics where a lot of user input is anticipated (or discovered) such as
for PEP 318, perhaps the initial round of the PEP should be a (set of) Wiki
page(s).  As the concepts it contains firm up a more static document can be
drawn up from that.

Skip
From mkc at mathdogs.com  Fri Aug  6 04:46:51 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Fri Aug  6 04:50:41 2004
Subject: [Python-Dev] pre-PEP: Complete,
	Structured Regular Expression Group Matching
Message-ID: <lkdekmlvv8k.fsf@panix2.panix.com>

If you've ever been frustrated that there wasn't an easy way to parse a string
or file, even though you could easily match it with re.match, then this PEP
may be for you.  If what's being proposed seems a little ghastly, well, I'll
just say that I've been wanting something like this for a long time and this
is the best I could come up with.  Your comments invited.

Mike

##############################################################################
PEP: XXX
Title: Complete, Structured Regular Expression Group Matching
Version: $Revision: 1.3 $
Last-Modified: $Date: 2004/08/02 05:18:31 $
Author: Mike Coleman <mkc@mathdogs.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 1-Aug-2004
Python-Version: 2.4
Post-History: 1-Aug-2004


Abstract
========

This proposal is to add a new method ``re.structmatch`` that fully
captures matches for groups that match repeatedly.  The method returns
a structured match tree whose shape is determined by the pattern
groups.  Ultimately this will allow a string that can be described by
a Python regular expressions (e.g., the contents of ``/etc/passwd`` or
``.ini`` files, or the output of ``diff``) to be parsed into the
obvious parse tree with a single call to ``re.structmatch``.


Motivation
==========

A notable limitation of the ``re.match`` method is that it fails to
capture all group match information for repeatedly matched groups.
For example, in a call like this ::

    m0 = re.match(r'([A-Z]|[a-z])*', 'XxxxYzz')

one would like to see that the group which matched four times matched
the strings ``'X'``, ``'xxx'``, ``'Y'`` and ``'zz'``.  In the current
implementation, only the last match for each group (``'zz'`` in this
case) is available, even though the other matches are calculated
implicitly during the matching process.  For simple cases, the missing
strings might be easily recovered by other means, but for more
complicated cases this is a significant annoyance.

The simplest extension would be to collect all of the matches for each
group in a list, so that in the call above for example ::

    m0.group(1) --> ['X', 'xxx', 'Y', 'zz']

(A mechanism like this is apparently to be added to Perl in version 6
[#PERL6]_, though that did not inspire this PEP.)

This immediately suggests the question of what is to be done for
nested repeats, like so ::

    m1 = re.match(r'("([A-Z]|[a-z])*"\s*)*', '"Xx" "yy" "ZzZ"')

We could have ::

    m1.group(2) --> ['X', 'x', 'yy', 'Z', 'z', 'Z' ]

but this flattened result would discard useful information about the
structure of the match.  A more useful result would be ::

    m1.group(2) --> [['X', 'x'], ['yy'], ['ZzZ']]

This is definitely an improvement.  Consider the following slightly
more complicated case, though ::

    m1 = re.match(r'("([A-Z]|[a-z])*"(\s*))*', '"Xx" "yy" "ZzZ"')

which would give ::

    m1.group(2) --> [['X', 'x'], ['yy'], ['Z', 'z', 'Z']]
    m1.group(3) --> [' ', ' ', '']

This is less useful than the putative conceptual structure of the
match, which would be something like ::

    [[['X', 'x'], ' '],
     [['yy'], ' '], 
     [['Z', 'z', 'Z'], '']]

In this simple case, this structure could be recovered using the
``zip`` function, but in the general case this is a non-trivial
transformation.

This PEP proposes a mechanism to address the general case directly.


Proposal
========

The new function ``re.structmatch`` has the same arguments as
``re.match`` and will match using exactly the same algorithm.  Instead
of returning a MatchObject upon success, ``re.structmatch`` will
return a "tree" (or rather, a forest) in the form of a Python list.

First we will define some terms.  Groups in the input pattern are
parenthesized subexpressions that "capture" strings when matched, and
are of the form ``'(abc)'`` or ``'(?P<name>abc)'``.  Leaf groups are
groups that do not contain any subgroups.  So, for example, the group
``'(abc)'`` is a leaf group and the group ``'(a(xyz)b)'`` is not a
leaf group (but it does contain the leaf group ``'(xyz)'``).  We shall
call parenthesized expressions that are not groups "nongroups"; these
include constructs like ``'(?:abc)'`` as well as lookahead and
lookbehind assertions.

The structure of the returned tree is determined from the grouping
tree present in the pattern in the following manner:

* Leaf groups not followed immediately by a repeat operator map to a
  single string::

    re.structmatch(r'(...)', 'abcdef') --> ['abc']
    re.structmatch(r'(...).(..)', 'abcdef') --> ['abc', 'ef']

* Leaf groups followed immediately by a repeat operator map to a list
  of strings::

    re.structmatch(r'([^d])*', 'abcdef') --> [['a', 'b', 'c']]
    re.structmatch(r'([^d])*(.)*', 'abcdef')
        --> [['a', 'b', 'c'], ['d', 'e', 'f']]
    re.structmatch(r'(..)*', 'abcdef') --> [['ab', 'cd', 'ef']]
    re.structmatch(r'(.){2}', 'abcdef') --> [['a', 'b']]

* Non-leaf groups not followed immediately by a repeat operator map to
  a list of the mappings of their subgroups::

    re.structmatch(r'(...)', 'abcdef') --> ['abc']
    re.structmatch(r'((...))', 'abcdef') --> [['abc']]
    re.structmatch(r'(((...)))', 'abcdef') --> [[['abc']]]
    re.structmatch(r'((...)())', 'abcdef') --> [['abc'], []]
    re.structmatch(r'(.(.)(.(.)).(.))', 'abcdef')
       --> ['a', ['b'], ['c', ['d']], 'e', ['f']]

* Non-leaf groups followed immediately by a repeat operator map to a
  list of the mappings of their repeated matches::

    re.structmatch(r'((.).(.))*', 'abcdef') --> [[['a', 'c'], ['d', 'f']]]
    re.structmatch(r'(([ab])*(x)*)*', 'baxbxx')
       --> [[['b', 'a'], ['x']], [['b'], ['x', 'x']]]

* In the case of alternation, only the matched groups appear in the
  output::

    re.structmatch(r'([^a])*|([^d])*', 'abcdef') --> [['a', 'b', 'c']]

  If it's important to know which alternative matched, named groups
  can be used.

* Named groups map to a pair where the first member is the name and
  the second member is what the unnamed group would have mapped to::

    re.structmatch(r'([^d])*(?P<rest>.)*', 'abcdef')
        --> [['a', 'b', 'c'], ('rest', ['d', 'e', 'f'])]

* Nongroups do not affect the output structure.  Compared to non-leaf
  groups, nongroups have the effect of "flattening" the output, like
  so::

    re.structmatch(r'((.).(.))', 'abcdef') --> [['a', 'c']]
    re.structmatch(r'(.).(.)', 'abcdef') --> ['a', 'c']
    re.structmatch(r'(?:(.).(.))', 'abcdef') --> ['a', 'c']

    re.structmatch(r'((.).(.))*', 'abcdef') 
        --> [[['a', 'c'], ['d', 'f']]]
    re.structmatch(r'(?:(.).(.))*', 'abcdef') 
        --> [['a', 'c', 'd', 'f']]

  (Nongroups that do not contain leaf groups have no effect whatsoever
  on the output structure.)


Additional Features
-------------------

In many cases in which ``'re.structmatch'`` fails to match, the cause will
be due to an unexpected error in the format of the string being
matched.  In order to assist the calling program in generating a more
meaningful possible error message, ``'re.structmatch'`` will return the
endpoint of the largest match against the searched string.  So, for
example ::

        re.structmatch('abcd', 'abxxx') --> 2
        re.structmatch('abcde|z', 'abxxx') --> 2
        re.structmatch('x*?y', 'xxx') --> 3

(This return value should be considered advisory rather than exact, as
future improvements in the match algorithm may make it difficult to
calculate the exact value.)


Examples
========

Parsing ``/etc/group``
----------------------

If ``/etc/group`` contains the following lines ::

    root:x:0:
    daemon:x:1:
    bin:x:2:
    sys:x:3:

then it can be parsed as follows ::

    p = r'''(?xm)           # VERBOSE, MULTILINE
            (
              (
                # one field, preceded by a ':' if
                # the field is not the line's first
                (?:^|:) ([^:\n]*)
              )*
              \n
            )*
         '''
    s = open('/etc/group').read()
    tree = re.structmatch(p, s)

which will give ``tree`` the following value::

    [['root', 'x', '0', ''],
     ['daemon', 'x', '1', ''],
     ['bin', 'x', '2', ''],
     ['sys', 'x', '3', '']]

Note that the above pattern is written to allow any ``/etc/group``
field to be empty.  The pattern won't work correctly for versions of
Python prior to 2.4 because of the possibility of repeating empty
matches.  This problem seems to have been fixed in Python 2.4.  (A
slightly more complicated pattern would work for pre-2.4 versions.)


Parsing ``.ini`` files
----------------------

The Microsoft ``.ini`` file format is found in various contexts (there
is one in the Python source distribution:
``Tools/freeze/extensions_win32.ini``).  Given a file with contents ::

    [singers]
    good=Annie Lennox
    bad=William Shatner

    [comedians]
    good=Monty Python

the file can be parsed with pattern ::

    r'''(?xm)           # VERBOSE, MULTILINE
        \s*             # leading whitespace
        (               # begin sections
          ^\[ ([^]]+) \] \s*  # section header
          (             # begin params
            ^([^=]+) =  # param name
            (.*) $      # param value
            \s*         # intervening whitespace
          )*            # end params
        )*              # end sections
        \Z              # assert that the entire
                        #   input was matched
     '''

to give ::

    [['singers',
      ['good', 'Annie Lennox'],
      ['bad', 'William Shatner']],
     ['comedians',
      ['good', 'Monty Python']]]

The pattern given omits support for ``.ini`` file comments for the
sake of simplicity, but this could be added.


Details
=======

The proposal states that ``re.structmatch`` will match using exactly
the same algorithm as ``re.match``.  This might be a little odd for a
pattern like ``r'(x|y|z)*aaa\1'``, where the algorithm will require
that the ``\1`` back-reference match (at most) one character.  It's
not obvious whether there is any better option, though, and there are
advantages of implementation and simplicity for keeping the match
algorithms of these two methods identical.  (A similar argument
applies to ``'(?P=NAME)'``.)


Discussion
==========

Part of the reason the proposed method would be so useful is that
``re.split`` currently does not split on empty matches.  If it had
this feature, one could split on lookahead and lookbehind assertions,
which would significantly ease parsing of strings with a recursive
regular structure (e.g., ``.ini`` files).  Patch `#988761`_ will
correct this ``re.split`` deficiency, if it is accepted.

.. _#988761: https://sourceforge.net/tracker/?func=detail&aid=988761&group_id=5470&atid=305470


For particularly complex patterns, the current 99 group limit might
actually become a practical problem.

One could imagine a variation in which subtrees of named groups might
be captured in dictionaries rather than lists, with the group names
used as keys.



Rejected Alternatives
=====================

Several simpler alternatives are rejected in the `Motivation`_ section
above.  Although these alternatives would be better than nothing, they
would not adequately satisfy the goal of this proposal, which is to
allow the programmer to extract the structure of a string in an
immediate manner.

It would be possible to use tuples for some substructures in the
return tree, rather than composing it strictly of lists.  This
alternative was rejected because it was thought useful to be able to
modify the resulting tree in place, perhaps to add decorations, etc.,
and tuples would make this more difficult.


References
==========

.. [#PERL6] Multiple regex matches in Perl Apocalypse 5
   (http://www.perl.com/pub/a/2002/06/04/apo5.html?page=22#rfc%20360:%20allow%20multiply%20matched%20groups%20in%20regexes%20to%20return%20a%20listref%20of%20all%20matches)


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:

From niemeyer at conectiva.com  Fri Aug  6 04:57:54 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Fri Aug  6 04:55:34 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <16658.45668.851011.950341@montanaro.dyndns.org>
References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
	<20040805213713.GA28782@panix.com>
	<16658.45668.851011.950341@montanaro.dyndns.org>
Message-ID: <20040806025754.GB7230@burma.localdomain>

> What he said.
> 
> +1.

+1

Given the discussed possibilities, I'm also more comfortable
with list-after-def.

> I had simply given up arguing the point because it seemed pointless
> and I was tired of arguing.  I suppose there are more than a few
> people in this particular boat.

That's exactly what I was afraid of.

-- 
Gustavo Niemeyer
http://niemeyer.net
From skip at pobox.com  Fri Aug  6 04:57:47 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 04:58:13 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112DD32.5080302@interlink.com.au>
References: <200408051636.i75Gaak03654@guido.python.org>
	<4112DD32.5080302@interlink.com.au>
Message-ID: <16658.62379.437323.980365@montanaro.dyndns.org>


    Anthony> From being a -0 (when Guido first mentioned it) to a +0 (at the
    Anthony> time of checkin) I'm now +1 on this form of decorator.

It's still not my first choice.  As others have observed, most functions
have no more than three parameters and it should be rare for a function to
have more than two decorators, so in the common case the list-after-def
works (decorators are prominent - on the def line), but don't obscure the
beginning of the definition ("def", function name, parameter list).

    Anthony> (It's different in that way to "print >>", which I still hate
    Anthony> <wink>)

Odd, I've always liked it.  I use it heavily for debugging.  The ">>" is
easy to grep for when I want to rip 'em out, and I can combine the
convenience of the print statement with redirection to stderr.

    Anthony> "The @ sign is used in Perl."
    Anthony> And?

I think the point there is that Perl has this history of using prefix
punctuation ($, @, %) to modify the meaning of the identifier that follows,
and more generally using punctuation in all sorts of non-mnemonic ways.  To
a certain degree @-decorators have that same feel.

Skip
From skip at pobox.com  Fri Aug  6 05:09:12 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 05:09:28 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <4112E602.4050009@pythonapocrypha.com>
References: <4112DD1C.3060902@interlink.com.au>
	<4112E602.4050009@pythonapocrypha.com>
Message-ID: <16658.63064.942060.30357@montanaro.dyndns.org>


    >> Having said that, I don't think the lack of completed PEP is a reason
    >> to back out the @ syntax from CVS. If nothing else, it being present
    >> in a released alpha is giving us very real experience with the use of
    >> the feature.

    Dave> (FWIW, it seems that a lot of the uproar about the syntax change
    Dave> is that it is being interpreted as a permanent change to the
    Dave> language rather than just experimental functionality. 

I early June I suggested that two different candidate syntaxes be added for
the alpha (Guido's previous before-the-def candidate and the list-after-def
syntax) precisely so people could experiment with both variants and decide
which one they liked better.  Guido shot it down immediately:

    http://mail.python.org/pipermail/python-dev/2004-June/045192.html

He was probably mostly averse to the two-at-once idea, but I also didn't
sense any support for the idea of any experimental addition.

I still think that finding a way to let users play with a couple syntax
variants would be a reasonable way to sort things out.

Skip
From greg at cosc.canterbury.ac.nz  Fri Aug  6 05:15:17 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug  6 05:15:25 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C4B4@au3010avexu1.global.avaya.com>
Message-ID: <200408060315.i763FHQV001012@cosc353.cosc.canterbury.ac.nz>

"Delaney, Timothy C (Timothy)" <tdelaney@avaya.com>:

> the above would alleviate much of my distate for @ which is that it
> doesn't look as though the decorators are connected to the definition.

For something which looks more attached to the definition,
how about

  staticmethod...
  def foo():
    ###

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From pete at shinners.org  Fri Aug  6 05:39:44 2004
From: pete at shinners.org (Pete Shinners)
Date: Fri Aug  6 05:39:13 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408051636.i75Gaak03654@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
Message-ID: <ceuugr$cej$1@sea.gmane.org>

Guido van Rossum wrote:
> It seems a public outcry against @decorators has started.  When I let
> Anthony check it in for 2.4a2, the plan was to see how it fares in a2
> and a3, and possibly take it out in 2.4b1.  If you don't want that to
> happen, you might want to say something in public in defense of
> @decorators.

All along I've not been too particular on the syntax for decorators. I 
see the need for tham, and still have sneaking suspicion this will end 
up changing the way people use Python in many contexts.

Now that the @ syntax is out, I am feeling it is the best of the many 
available options. All the proposals has pros and cons, but the @ syntax 
seems to always win with more pros, especially after more experimentation.

I look forward to taking advantage of the decorators very soon. I can't 
imagine there are any further syntax options remaining, although I 
wouldn't be opposed if the community could put something together.

Anyways, this hardly turns out to be a rallying defense. But the need 
for functionality is there, and the many syntax proposals have been 
weighed. The better option has already been committed in my eyes.

I'm just glad >> wasn't involved, heh.

From tanzer at swing.co.at  Fri Aug  6 05:43:14 2004
From: tanzer at swing.co.at (Christian Tanzer)
Date: Fri Aug  6 05:43:29 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: Your message of "Thu, 05 Aug 2004 17:22:45 -0300."
	<20040805202245.GH29796@burma.localdomain>
Message-ID: <E1BsvdS-0001ZJ-00@swing.co.at>


> > I'm in camp 2, but now that pie decorators are in, and I've had a chance
> > to convert my code to use them, I'm strongly +1 in favor of this
> > syntax.  It stands out nicely, and to me indicates a stronger affinity
> > to the def that follows it than the C# syntax.
>
> I'm glad to see people happy with the current implementation.
>
> > I was never in favor of C# syntax, and I'm glad it wasn't chosen.  I
> > strongly disliked that it subtly changed the semantics of currently
> > valid Python.  I like that pie decorators code cannot run in older
> > Pythons, because if it /could/ it certainly wouldn't work.
>
> I'm strongly against "I'm sick of it" and "I don't care [anymore]"
> implemented features. Seeing everyone complaining and no positive
> reactions to the current implementation certainly feels like
> something is wrong.

Sigh.

I'm just a lurker here and I don't really want to add to all the heat.

All that said, I'm +1 on the pie syntax (and being ugly means that it
makes decorations obvious).

To all proponents of list markup: that would be very hard to look up
in the documentation. `@` should be easy to find in the index.

To Gustavo in particular: people are sick of hearing endless repeats
of the same old tired arguments (I certainly am).

-- 
Christian Tanzer                                    http://www.c-tanzer.at/

From skip at pobox.com  Fri Aug  6 05:54:31 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 05:54:42 2004
Subject: [Python-Dev] Re: PEP-0318
In-Reply-To: <41122FC2.1080608@interlink.com.au>
References: <41122FC2.1080608@interlink.com.au>
Message-ID: <16659.247.791367.302786@montanaro.dyndns.org>


    Anthony> ... here's what I managed [to edit in PEP 318] before a tidal
    Anthony> wave of crap flowed in...

I checked in Anthony's changes to PEP 318 (sent in private email) with very
few modifications.  I simply didn't have enough time to do more this
evening.  I didn't even have the time to read back through the entire
document.

Anybody else who feels like taking a whack at it, feel free.  I'll be out of
Internet reach over the weekend, so it's unlikely I'll be able to do
anything more with it until next week.  Context diffs against v 1.15 would
be best on the off-chance I get inputs from multiple people.  A SourceForge
patch would be the best place to stick such things so other people can see
where your changes are.  Note that I'm not really interested in thirty-leven
new syntax proposals.  What would be helpful are pointers to other documents
and concise summaries of some of the discussions on c.l.py and python-dev.
Incorporation of all or part of

    http://www.python.org/moin/PythonDecorators

into the PEP would also be appreciated.  Try to write objectively.  A PEP
isn't Usenet. <wink>

Note how I'm being optimistic that multiple people will contribute.  Lots of
people have worn out their keyboards on this subject, so I know the energy
is out there.  It just needs to be focused a bit.

Thx,

Skip
From fdrake at acm.org  Fri Aug  6 05:56:29 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Aug  6 05:56:41 2004
Subject: [Python-Dev] The impact of '@' on Leo
In-Reply-To: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz>
References: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz>
Message-ID: <200408052356.29328.fdrake@acm.org>

On Thursday 05 August 2004 10:41 pm, Greg Ewing wrote:
 > But maybe it would be a good idea to explicitly reserve some
 > character (perhaps only at the beginning of a line).

Bob Ippolito responded:
 > You mean like # ?

Yeah, like that.  ;-)  That's always accepted as the first non-blank character 
on a non-continuation line, and the only thing Python ever cares about after 
that are the encoding turd and a newline.  Anything after the newline is, 
well, another line.

I've really only managed to look at Leo much once, and Edward was sitting next 
to me at the time.  Leo doesn't store Python code in a Leo document as any 
sort of embedded object that's labelled as Python code as far as I can tell 
(though it may in it's data model); it just more text, and can have 
@directives like any other text.  I think this is inherently a risky approach 
to the data; there needs to be at least a way to say that some portion of the 
data is some other text format embedded into the Leo document.

I'm sure Edward can find a way to make things work for Leo users, but I hope 
the .leo files have some sort of format version indicator to make upgrading 
relatively easy for Leo users.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From andrewm at object-craft.com.au  Fri Aug  6 06:04:10 2004
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Fri Aug  6 06:04:13 2004
Subject: [Python-Dev] trunk is no longer frozen 
In-Reply-To: <4112E9AB.9060003@object-craft.com.au> 
References: <41112923.8060505@interlink.com.au>
	<4112CCDF.7030609@interlink.com.au>
	<4112E9AB.9060003@object-craft.com.au>
Message-ID: <20040806040410.57C1E3C221@coffee.object-craft.com.au>

>Although I have checkin permission on CVS I would appreciate it if 
>someone could look at my socketpair() patch before I do check it in...

In the name of bringing this thread up out of the noise: we need a
"go/no go" on applying this patch to the tree prior to a3 (the patch
adds "socketpair" to the socket module. Can I ask if the gods can
spend a moment reviewing it (it's well localised and includes test and
documentation updates)?

>Could people who care about the addition of socketpair() to the socket 
>module please have a look at patch #1003700.  The patch is relative to 
>2.4a1 but applies cleanly to 2.4a2 with slight offsets.



-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/
From pje at telecommunity.com  Fri Aug  6 06:12:22 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug  6 06:08:22 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <200408060224.i762O2Vc000926@cosc353.cosc.canterbury.ac.nz>
References: <4E57E6F0-E71C-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <5.1.1.6.0.20040806001019.02ab6060@mail.telecommunity.com>

At 02:24 PM 8/6/04 +1200, Greg Ewing wrote:
>Ronald Oussoren <ronaldoussoren@mac.com>:
>
> > And how would that solve this:
> > <http://www.eby-sarna.com/pipermail/peak/2004-July/001598.html>
>
>FWIW, in decorator-plus-function-attributes style, this
>would be
>
>   def do_something(x,y) [generic]:
>     @when = "x in int and y in str"
>     # body here will be executed when 'x'
>     # is an integer and 'y' is a string

Which reduces composability of decorators by introducting the possibility 
of attribute collisions, while simultaneously reducing clarity by 
separating decorators from their arguments.

From niemeyer at conectiva.com  Fri Aug  6 06:16:14 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Fri Aug  6 06:13:54 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <E1BsvdS-0001ZJ-00@swing.co.at>
References: <20040805202245.GH29796@burma.localdomain>
	<E1BsvdS-0001ZJ-00@swing.co.at>
Message-ID: <20040806041614.GA7579@burma.localdomain>

> Sigh.
> 
> I'm just a lurker here and I don't really want to add to all the heat.

There's no heat here. There's just a discussion about something polemic.

> All that said, I'm +1 on the pie syntax (and being ugly means that it
> makes decorations obvious).
> 
> To all proponents of list markup: that would be very hard to look up
> in the documentation. `@` should be easy to find in the index.
> 
> To Gustavo in particular: people are sick of hearing endless repeats
> of the same old tired arguments (I certainly am).

I'm glad my tired arguments made you say your opinion about
the current syntax.

-- 
Gustavo Niemeyer
http://niemeyer.net
From nbastin at opnet.com  Fri Aug  6 07:01:46 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Fri Aug  6 07:02:16 2004
Subject: [Python-Dev] use of #pragma GCC system_header to suppress
	_XOPEN_SOURCE warnings
In-Reply-To: <4112B7A8.5090100@v.loewis.de>
References: <16658.42673.542930.749915@montanaro.dyndns.org>
	<4112A968.2030401@v.loewis.de>
	<16658.45979.672934.897388@montanaro.dyndns.org>
	<4112B7A8.5090100@v.loewis.de>
Message-ID: <B7B41599-E765-11D8-8FC8-000D932927FE@opnet.com>


On Aug 5, 2004, at 6:41 PM, Martin v. L?wis wrote:

> Skip Montanaro wrote:
>> The platform is Solaris/Intel (2.8 and/or 2.9).  I'll have to get you 
>> more
>> complete details tomorrow.  The involved folks are gone for the day.
>
> Ok. I'll get back to a Solaris machine next week, so I can investigate
> somewhat myself.
>
> Of course, the rule is, strictly speaking, "no warnings on the major
> platforms"; I'm uncertain whether Solaris still applies here. It is
> clearly not feasible to have Python compile with no warning no matter
> what the compiler or operating system is. The major platforms, of
> course, are Windows, Linux, OS X (i.e. the ones we provide binaries
> for).

I've been trying to keep the solaris build warning-free for the last 
few months, although I've been using SunPro and not GCC.  I would like 
to believe that Solaris is a 'major platform' (sparc, at least), but 
maybe I'm misguided.. :-)

--
Nick
From psoberoi at gmail.com  Fri Aug  6 07:02:26 2004
From: psoberoi at gmail.com (Paramjit Oberoi)
Date: Fri Aug  6 07:02:35 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz>
References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz>
Message-ID: <e443ad0e04080522023e4029e6@mail.gmail.com>

> > on that page; it's only ugly for long decorator definitions, and Guido said
> > he didn't care if a syntax was ugly.  :)

> I hope Guido will see these arguments as being rational ones about
> usability and not irrational ones about aesthetics. At the least,

I'm worried by the thought expressed multiple times today that ugly is
OK and aesthetics don't matter.  I don't think aesthetics can be
separated from usability, and FWIW, I would like to say that as far as
I am concerned, Aesthetics are what make Python what it is.

Yes, the development community is very nice, and so is the standard
library, but the real thing distinguishes Python in my mind is that
the programs are so incredibly nice to look at. It is the beauty of
the programs that has kept me faithful and fanatically devoted to
Python all these years.

Python has gotten more complex over the years, but most of the
additions---list/generator comprehensions, generators, and iterators
in particular---have enhanced aesthetics of the language. I do reaize
that there are some hard problems with decorator syntax, and that even
a somewhat ugly choice will vastly improve some people's lives, but
please don't say ugly is fine and aesthetics are unimportant.

As Tim Peters said in import this, "Beautiful is better than ugly" and
"Readability counts".

-param

PS1: This is just a general comment based on the atmosphere I have
observed today, and not a specific criticism of anyone. Yes, I realize
that when Greg said ,"...see these arguments as being rational ones
about usability and not irrational ones about aesthetics" he meant
that arguments about aesthetics are usually subjective. I apologise if
I'm offending anyone or stating the obvious.

PS2: I like the list-after-def syntax.
From tim.peters at gmail.com  Fri Aug  6 07:31:57 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug  6 07:32:00 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <e443ad0e04080522023e4029e6@mail.gmail.com>
References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz>
	<e443ad0e04080522023e4029e6@mail.gmail.com>
Message-ID: <1f7befae04080522314d818d8f@mail.gmail.com>

[Paramjit Oberoi]
> ...
> As Tim Peters said in import this, "Beautiful is better than ugly" and
> "Readability counts".

Indeed.  Luckily, since I was able to channel those words from the
depths of Guido's bottomless mind, I can also channel what beauty and
readability mean.
...

> PS2: I like the list-after-def syntax.

There's quite a peculiar split over that.  I find the @ gimmick most
readable of all, and especially because it forces one decoration per
physical line.  list-after-def looks like a train wreck piling up
after the arglist half the time.  But @ looks like a friendly little
womb, with a happy little birth canal, out of which is born a single
pure expression.  What's more beautiful than the miracle of birth? 
Certainly not a train wreck.

leave-the-miracle-of-death-for-lesser-languages-to-explore-ly y'rs  - tim
From ixokai at gmail.com  Fri Aug  6 08:03:22 2004
From: ixokai at gmail.com (IxokaI)
Date: Fri Aug  6 08:03:30 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <1f7befae04080522314d818d8f@mail.gmail.com>
References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz>
	<e443ad0e04080522023e4029e6@mail.gmail.com>
	<1f7befae04080522314d818d8f@mail.gmail.com>
Message-ID: <f8ef9ae40408052303384b55c@mail.gmail.com>

On Fri, 6 Aug 2004 01:31:57 -0400, Tim Peters <tim.peters@gmail.com> wrote:
> > PS2: I like the list-after-def syntax.
> 
> There's quite a peculiar split over that.  I find the @ gimmick most
> readable of all, and especially because it forces one decoration per
> physical line.  list-after-def looks like a train wreck piling up
> after the arglist half the time.  But @ looks like a friendly little
> womb, with a happy little birth canal, out of which is born a single
> pure expression.  What's more beautiful than the miracle of birth?
> Certainly not a train wreck.

  Doesn't the fact that the @stuff is at the same level and has no
connection to the def besides 'above' bother you in our universe of
indentation-love?
  And the fact that the pie *is* "at", and why that it just doesn't
"read"? '"define methy1 at staticmethod" is wrong, but "at
staticmethod define methy2" is right.
  I like Python to be able to be read very easily. If we have to have
it at the same level as def and above, we soo need a keyword..pweeze?

class MyLittleToy:
   def methy1(self):
       print 'whee'
   @staticmethod
   def methy2():
        print 'woo'
   @staticmethod
   def methy3(arg):
        for i in arg:
           print 'woohoo'

? That bothers me a looot... and is the primary reason I hate the pie.
That and I really don't want anymore punctuation.

--Stephen
From martin at v.loewis.de  Fri Aug  6 09:02:02 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 09:02:11 2004
Subject: [Python-Dev] trunk is no longer frozen
In-Reply-To: <20040806040410.57C1E3C221@coffee.object-craft.com.au>
References: <41112923.8060505@interlink.com.au>	<4112CCDF.7030609@interlink.com.au>	<4112E9AB.9060003@object-craft.com.au>
	<20040806040410.57C1E3C221@coffee.object-craft.com.au>
Message-ID: <41132CEA.9000605@v.loewis.de>

Andrew McNamara wrote:
> In the name of bringing this thread up out of the noise: we need a
> "go/no go" on applying this patch to the tree prior to a3 (the patch
> adds "socketpair" to the socket module. Can I ask if the gods can
> spend a moment reviewing it (it's well localised and includes test and
> documentation updates)?

Let me repeat my earlier response to this kind of request: If you want
to "push" a patch into Python, the best way to get my attention is to
review 10 patches, comment on them, and post the summary of the review
to python-dev. If you do this often enough, you will become one of the
gods yourself :-)

That said, I think adding socketpair is a harmless extension.

Regards,
Martin
From martin at v.loewis.de  Fri Aug  6 09:26:34 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 09:26:43 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <4112DD1C.3060902@interlink.com.au>
References: <4112DD1C.3060902@interlink.com.au>
Message-ID: <411332AA.8000606@v.loewis.de>

Anthony Baxter wrote:
> I'm extremely averse to making the Python development
> process any heavier than it has to be

I think the original process (PEP1) goes like this:
- PEP champion writes the PEP, starts discussion
- PEP champion collects opinions, to avoid repeated
   discussion
- BDFL pronounces
- code gets written
- code gets checked in

So, in principle, the BDFL can only pronounce on what
is in the PEP, however, that is not really important.

What *is* really important is that there must be a PEP
champion all the time. Once the PEP champion loses
interest, or goes away, the process stops here and now -
irrespective of any code that has been written, or
pronouncements the BDFL has made.

> Having said that, I don't think the lack of completed
> PEP is a reason to back out the @ syntax from CVS.

I do firmly think this is sufficient reason. It means
there is no champion for the PEP, so there can't be
a new feature. In essence, the champion is the one
who "controls" the feature, and who is in charge of
making it work well. He is the one to complain to,
and he will make sure issues get resolved (not necessarily
resolving them himself).

I don't think the code needs to backed-out *now*;
waiting for the next alpha would probably be enough.

> There's also the issue that there's a bunch
> of other existing PEPs describing features that aren't
> up to date at all.

This is the same problem. Nobody will complain if it works
out all nicely, but we can forget about the PEP process
if we are not willing to bow to the rules. If we don't
like the rules, we should losen them, or perhaps drop
the PEP process altogether. However, just ignoring it is
not acceptable.

> I don't
> think we want to make the Wiki the primary copy of
> the document, but I think having the PEPs annotatable
> would be a win.

The easiest way to achieve this is to embed a Wiki link
in each PEP.

Regards,
Martin
From martin at v.loewis.de  Fri Aug  6 09:32:46 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 09:32:51 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <4112E602.4050009@pythonapocrypha.com>
References: <4112DD1C.3060902@interlink.com.au>
	<4112E602.4050009@pythonapocrypha.com>
Message-ID: <4113341E.4070502@v.loewis.de>

Dave Brueck wrote:
> (FWIW, it seems that a lot of the uproar about the syntax change is that 
> it is being interpreted as a permanent change to the language rather 
> than just experimental functionality. I _think_ I read here on 
> python-dev that it was semi-experimental, but if that's the case I'm not 
> sure how well that message got out - e.g. the "What's New in Python 
> 2.4a2" section of NEWS.html doesn't reflect this)

Anthony's understanding is also mine: the feature is in the alpha so
users can experiment with it, and accomodate with it. If the feature
turns out to be completely unacceptable, it could be taken out. If
users contribute something better, it can be replaced. However, users
should not be under the impression that it magically changes or goes
away without any action.

Regards,
Martin
From gmccaughan at synaptics-uk.com  Fri Aug  6 09:33:43 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Fri Aug  6 09:34:16 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408060035.i760Z5PE000754@cosc353.cosc.canterbury.ac.nz>
References: <200408060035.i760Z5PE000754@cosc353.cosc.canterbury.ac.nz>
Message-ID: <200408060833.43982.gmccaughan@synaptics-uk.com>

On Friday 2004-08-06 01:35, Greg Ewing wrote:
> > >> class C(random.choice([dict, list])):
> > >>     pass
> > >>
> > >> is my favourite example of this :-)
> > >
> > > Where is THAT monstrocity used!?
> > 
> > Well, it's not, I hope :-)
> 
> Hmmm... maybe if you had some algorithm that was more efficient
> using either a list or a dict, depending on the input data, but
> in a way that makes it difficult to tell ahead of time which
> to use, and you want to improve the average behaviour...

... then you'd do better to find which out of (dict,list)
does better on average, and always use that one. Sorry. :-)

-- 
g

From lists at hlabs.spb.ru  Fri Aug  6 13:42:39 2004
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Fri Aug  6 09:37:00 2004
Subject: [Python-Dev] OK to lower timeout for
	test_timeout's	testConnectTimeout test?
In-Reply-To: <4112A365.1000203@ocf.berkeley.edu>
References: <4112A365.1000203@ocf.berkeley.edu>
Message-ID: <41136EAF.7000908@hlabs.spb.ru>

Brett Cannon wrote:
> I am actually getting failures from test_timeout's testConnectTimeout 
> since the Net connection I have at school lets me connect to Google in 
> under the .001 connection timeout value.  If I push it to .00000001 (a 
> hundred-millionth of a second) it then fails consistently.
> 
> Now the problem is that the second part of the test uses this and a fuzz 
> value (currently at 2) to see if the test timed out within the proper 
> amount of time.  The comparison is basically if the amount of time it 
> took to do the timed out failure is less than timeout + fuzz.  So 
> lowering this number could possibly affect the test, although at .001 
> seconds, I am doubting that will occur.  But since these types of timing 
> tests can be touchy I thought I would double-check.
> 
> So if anyone thinks it is bad to lower the value to .00000001 then 
> please let me know.  Otherwise I will lower the value before the next 
> alpha goes out.
> 
> -Brett
> 
> P.S.: the only other failures I have on OS X right now is test_curses 
> (because I used -uall) and test__locale .  Should we disable 
> test__locale on OS X to shut it up since Martin seems to think the test 
> isn't all that useful and won't work for OS X ever?

Some time ago I submit patch for test_timeout.py:

http://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=728815

In my patch testConnectTimeout try connect to www.google.com:21 which 
seems like black hole. :)

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru
From martin at v.loewis.de  Fri Aug  6 09:39:40 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 09:39:46 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <16658.63064.942060.30357@montanaro.dyndns.org>
References: <4112DD1C.3060902@interlink.com.au>	<4112E602.4050009@pythonapocrypha.com>
	<16658.63064.942060.30357@montanaro.dyndns.org>
Message-ID: <411335BC.7010504@v.loewis.de>

Skip Montanaro wrote:
> I still think that finding a way to let users play with a couple syntax
> variants would be a reasonable way to sort things out.

I also disagree. People don't get the urgency of the need for feedback
if they are left to guess which syntax will finally make it. If 
confronted with only a single choice, they focus on that syntax. If 
significant factual problems can be detected, the syntax can be 
discarded, and the reason for failure be recorded. In addition, a single 
approach gives focus for well-designed counter-proposals. If there are 
no flaws found and no "better" counter-proposals made, there is no 
reason to change the feature.

Regards,
Martin
From pyth at devel.trillke.net  Fri Aug  6 09:45:09 2004
From: pyth at devel.trillke.net (Holger Krekel)
Date: Fri Aug  6 09:45:13 2004
Subject: Exceptional inheritance patterns (was Re: [Python-Dev] Python in
	Unicode context)
In-Reply-To: <200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz>
References: <20040805153443.GB5208@solar.trillke>
	<200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040806074509.GD5208@solar.trillke>

Greg Ewing wrote:
> > My point is that I like to regard try/except as a mechanism for
> > "out-of-band" objects.  Guidos "should be shot" seems to indicate he
> > sees try/except only useful/applicable to exception-handling.
> 
> If the root exception class were called something else,
> such as 'Raisable', would that make you feel better?

Yes, I certainly wouldn't object. I guess this would mean 'Exception'
would derive from Raisable because Exception itself should probably not
go away.  Hey, strings could inherit from Raisable, too! Just kidding :-) 

Then again, i think Python has a tradition of not requiring inheritance 
but just behaviour.  And doesn't this whole issue only exist because 
"raise X" with X being a class and autoinstantiated is allowed? 

Well, anyway, let's not add too much to the current python-dev traffic 
with this issue.  I think it has been brought up a couple of times
already. 

Hey, i have an idea: why not create a python-syntax mailing list 
(or python-hell :-) ? 

cheers, 

    Holger

From mak at trisoft.com.pl  Fri Aug  6 10:09:04 2004
From: mak at trisoft.com.pl (Grzegorz Makarewicz)
Date: Fri Aug  6 10:09:06 2004
Subject: [Python-Dev] signed - unsigned - big troubles 2.4a2
Message-ID: <41133CA0.3060009@trisoft.com.pl>

Many python programmas are relaying on computer int size - in prior 
versions bitnees is 32 long - now smoething happens and signed/unsigned 
value is not 32 bit long int but unsigned long int or even better 
unsigned uint64

consider that:
	a = ord(data[i]) .. data[i+4] - value is int or int64, signed or unsigned ?

0xf.. in prior versions up to 2.4.a2 int was used as signed value, now 
it is unsigned int64, so my question how to create signed int32 ?

mak

From anthony at interlink.com.au  Fri Aug  6 10:11:12 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 10:11:26 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <411332AA.8000606@v.loewis.de>
References: <4112DD1C.3060902@interlink.com.au> <411332AA.8000606@v.loewis.de>
Message-ID: <41133D20.8040307@interlink.com.au>

Martin v. L?wis wrote:
> What *is* really important is that there must be a PEP
> champion all the time. Once the PEP champion loses
> interest, or goes away, the process stops here and now -
> irrespective of any code that has been written, or
> pronouncements the BDFL has made.

This is a nice way to look at it. Unfortunately, we've
not honoured this much. I'm happy to say that we should
try and do this, but we have a significant issue with
un-updated PEPs.

>> Having said that, I don't think the lack of completed
>> PEP is a reason to back out the @ syntax from CVS.
> 
> 
> I do firmly think this is sufficient reason. It means
> there is no champion for the PEP, so there can't be
> a new feature. In essence, the champion is the one
> who "controls" the feature, and who is in charge of
> making it work well. He is the one to complain to,
> and he will make sure issues get resolved (not necessarily
> resolving them himself).

A couple of people have commented that the int/long PEP,
for instance, is not up-to-date. I don't think this means
that the changes should be backed out.

> I don't think the code needs to backed-out *now*;
> waiting for the next alpha would probably be enough.

Assuming the PEP can't be updated by then, sure. That
then suggests (to me) that the process should be that the
PEP should be updated to the state of the art before the
next release that includes the new features or behavior
(which was the first option in my original list).

In many cases, the process goes more like:

  - PEP is written
  - code is checked in
  - issues occur, code is updated, but PEP is not updated
    to match it.

In an extreme case, that's what's happened here - the basic
goal of the PEP is still there, but the syntax is completely
different, and a large chunk (class decorators) has been
removed. I suspect part of the problem is the sheer amount of
discussions on the topic, combined with decorator-fatigue on
the part of the people who could update it. (In my case, it's
mostly the former problem, combined with a lack of time).

>> There's also the issue that there's a bunch
>> of other existing PEPs describing features that aren't
>> up to date at all.
> 
> This is the same problem. Nobody will complain if it works
> out all nicely, but we can forget about the PEP process
> if we are not willing to bow to the rules. If we don't
> like the rules, we should losen them, or perhaps drop
> the PEP process altogether. However, just ignoring it is
> not acceptable.

Perhaps we need to collect a list of PEPs that are out-of-date,
and push really hard to get them brought up to date. I'd like
to see more use of the PEP status field in this way.


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From mak at trisoft.com.pl  Fri Aug  6 10:33:49 2004
From: mak at trisoft.com.pl (Grzegorz Makarewicz)
Date: Fri Aug  6 10:33:34 2004
Subject: [Python-Dev] signed - unsigned - big troubles 2.4a2
In-Reply-To: <41133CA0.3060009@trisoft.com.pl>
References: <41133CA0.3060009@trisoft.com.pl>
Message-ID: <4113426D.2030904@trisoft.com.pl>

t = -2208988800.0
import time
print time.localtime(t)

another nasty situation

 > timestamp out of range for platform time_t

mak

From Paul.Moore at atosorigin.com  Fri Aug  6 11:03:43 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Fri Aug  6 11:03:44 2004
Subject: [Python-Dev] Questions about '@' in pep 318
Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060F4C@UKDCX001.uk.int.atosorigin.com>

From: Edward K. Ream

> The "mandate" for this post is the following from GvR:
>
>> If good arguments against @ are presented it may be removed in
>> 2.4b1 (that was always the plan but apparently not publicized).
>
> So I would like seriously to consider whether removing '@' might
> be a good idea.
>
> Note: my major concern with pep 318 has always been with syntax.
> I am not interested or qualified to discuss whether annotation
> in general is a good idea...

One point here. To get the "@" syntax removed, you will (should)
need to *either* convince everyone who dislikes the "@" syntax to
back a single alternative syntax (experience shows that this is
unlikely in the extreme) *or* argue that the whole decorator feature
be removed. You can't do the latter without discussing semantics and
"whether annotation in general is a good idea".

Please do not mislead people into thinking that a discussion of syntax
alone is of any value, *unless* there is a willingness on the part
of those against the "@" syntax to compromise on a single alternative.

Semantics is a different matter - I've not heard any strong arguments
against the semantics of decorators.

And process is a different matter again - but that's for a different
posting (which I may well not make...)

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________
From marktrussell at btopenworld.com  Fri Aug  6 11:22:15 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Fri Aug  6 11:22:21 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <59e9fd3a04080519462bf57899@mail.gmail.com>
References: <59e9fd3a04080519462bf57899@mail.gmail.com>
Message-ID: <1091784135.1770.4.camel@localhost>

On Fri, 2004-08-06 at 03:46, Andrew Durdin wrote:
> 8. When using multiple decorators, each appears on a separate line.

No, you can put multiple decorators on a single line -- you just need a
newline before the def.  From test_decorators.py:

	class C(object):
            @funcattrs(abc=1) @staticmethod
            def foo(): return 42

Probably not good style (but neither is "if cond: stmt" I guess).

Mark

From marktrussell at btopenworld.com  Fri Aug  6 11:22:17 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Fri Aug  6 11:22:23 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <59e9fd3a04080519462bf57899@mail.gmail.com>
References: <59e9fd3a04080519462bf57899@mail.gmail.com>
Message-ID: <1091784135.1770.6.camel@localhost>

On Fri, 2004-08-06 at 03:46, Andrew Durdin wrote:
> 8. When using multiple decorators, each appears on a separate line.

No, you can put multiple decorators on a single line -- you just need a
newline before the def.  From test_decorators.py:

	class C(object):
            @funcattrs(abc=1) @staticmethod
            def foo(): return 42

Probably not good style (but neither is "if cond: stmt" I guess).

Mark

From martin at v.loewis.de  Fri Aug  6 11:31:52 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 11:32:02 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <41133D20.8040307@interlink.com.au>
References: <4112DD1C.3060902@interlink.com.au> <411332AA.8000606@v.loewis.de>
	<41133D20.8040307@interlink.com.au>
Message-ID: <41135008.3070906@v.loewis.de>

Anthony Baxter wrote:
> This is a nice way to look at it. Unfortunately, we've
> not honoured this much. I'm happy to say that we should
> try and do this, but we have a significant issue with
> un-updated PEPs.

Yes. However, we can learn, and correct earlier errors.
This doesn't all need to happen before 2.4, but we could
start.

For example, we could try listing issues with existing
PEPs, and then ask the previous champions to deal with
these issues. Responsiveness to such requests is an
indication how responsible a contributor is.

> A couple of people have commented that the int/long PEP,
> for instance, is not up-to-date. I don't think this means
> that the changes should be backed out.

With Moshe Zadka and Guido van Rossum as champions.

Looking at PEP 237, I fail to see why people say the
PEP is not up-to-date. According to the PEP, we should be
in stage B1, and AFAICT, we actually are in stage B1.
The last change to the PEP is from December 2003.

> In many cases, the process goes more like:
> 
>  - PEP is written
>  - code is checked in
>  - issues occur, code is updated, but PEP is not updated
>    to match it.

This is IMO ok if the changes are "minor", where a minor
change is one that doesn't deserve a PEP. Adding new
functions or changing the precise conditions under which
exceptions occur falls into this class.

> In an extreme case, that's what's happened here - the basic
> goal of the PEP is still there, but the syntax is completely
> different, and a large chunk (class decorators) has been
> removed. 

This is still not the process you described: no code had been
checked in before. If the feature as described had been
implemented before, then this change would have been a major
change, and should not have been committed without a PEP
of its own.

> I suspect part of the problem is the sheer amount of
> discussions on the topic, combined with decorator-fatigue on
> the part of the people who could update it. (In my case, it's
> mostly the former problem, combined with a lack of time).

I'm not blaming anybody, and this is all understandable. We
just have to find ways how to deal with the problem that nobody
has time to really work on these things.

My view is: it is sometimes better nothing gets done, rather than
something being done poorly. Of course, this view only applies
to new features - for bug fixes, it is better to do something
that fixes the bug (like ripping the feature out) rather than
doing nothing.

> Perhaps we need to collect a list of PEPs that are out-of-date,
> and push really hard to get them brought up to date. I'd like
> to see more use of the PEP status field in this way.

I don't think it is really that bad wrt. the current PEPs.
Some of the issues I found are
- PEP 263 states that Python 2.4 should give an error
   if no encoding declaration is specified. I propose to
   move this to 2.5.
- PEP 322: we need to evaluate whether reversed is useless.
- PEP 237: CSV is implemented, why is it still Draft, and open?
- PEP 307: Likewise
- PEP 331: Likewise (although I probably still owe a review)

Regards,
Martin


From martin at v.loewis.de  Fri Aug  6 11:37:26 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 11:37:31 2004
Subject: [Python-Dev] signed - unsigned - big troubles 2.4a2
In-Reply-To: <41133CA0.3060009@trisoft.com.pl>
References: <41133CA0.3060009@trisoft.com.pl>
Message-ID: <41135156.9050502@v.loewis.de>

Grzegorz Makarewicz wrote:
> Many python programmas are relaying on computer int size - in prior 
> versions bitnees is 32 long - now smoething happens and signed/unsigned 
> value is not 32 bit long int but unsigned long int or even better 
> unsigned uint64
> 
> consider that:
>     a = ord(data[i]) .. data[i+4] - value is int or int64, signed or 
> unsigned ?
> 
> 0xf.. in prior versions up to 2.4.a2 int was used as signed value, now 
> it is unsigned int64, so my question how to create signed int32 ?

No, it is not unsigned int64:

Python 2.4a2 (#86, Aug  6 2004, 09:08:27)
[GCC 3.3.4 (Debian 1:3.3.4-7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> 0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
28269553036454149273332760011886696253239742350009903329945699220681916415L
 >>>

This number does not fit in 64 bits. Instead, Python integers are 
without bounds. See

http://www.python.org/peps/pep-0237.html

Regards,
Martin
From mal at egenix.com  Fri Aug  6 11:37:48 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Aug  6 11:37:53 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <41135008.3070906@v.loewis.de>
References: <4112DD1C.3060902@interlink.com.au>
	<411332AA.8000606@v.loewis.de>	<41133D20.8040307@interlink.com.au>
	<41135008.3070906@v.loewis.de>
Message-ID: <4113516C.8050900@egenix.com>

Martin v. L?wis wrote:
> I don't think it is really that bad wrt. the current PEPs.
> Some of the issues I found are
> - PEP 263 states that Python 2.4 should give an error
>   if no encoding declaration is specified. I propose to
>   move this to 2.5.

+1

People are just learning to use the source code encoding. Banging
them on the head now with the SyntaxError would put them off,
which is not really what the PEP intends to do.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 06 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mwh at python.net  Fri Aug  6 12:16:10 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 12:16:16 2004
Subject: [Python-Dev] Re: Exceptional inheritance patterns
In-Reply-To: <20040806074509.GD5208@solar.trillke> (Holger Krekel's message
	of "Fri, 6 Aug 2004 09:45:09 +0200")
References: <20040805153443.GB5208@solar.trillke>
	<200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz>
	<20040806074509.GD5208@solar.trillke>
Message-ID: <2macx87es5.fsf@starship.python.net>

Holger Krekel <pyth@devel.trillke.net> writes:

> Greg Ewing wrote:
>> > My point is that I like to regard try/except as a mechanism for
>> > "out-of-band" objects.  Guidos "should be shot" seems to indicate he
>> > sees try/except only useful/applicable to exception-handling.
>> 
>> If the root exception class were called something else,
>> such as 'Raisable', would that make you feel better?
>
> Yes, I certainly wouldn't object. I guess this would mean 'Exception'
> would derive from Raisable because Exception itself should probably not
> go away.  Hey, strings could inherit from Raisable, too! Just kidding :-) 

I would like an exception class that almost exceptions except
KeyboardInterrupt, SystemExit and -- maybe -- RuntimeError and
MemoryError inherited from.

except ExceptionsButThoseNastyOnesIDontWantToCatch:
    pass

? <wink>

> Then again, i think Python has a tradition of not requiring inheritance 
> but just behaviour.  And doesn't this whole issue only exist because 
> "raise X" with X being a class and autoinstantiated is allowed? 

I would say that it's more because it's useful to organize exceptional
conditions in a tree like hierarchy and inheritance is a usable way to
do this.  The fact that arranging things into tree like hierarchies
*isn't* what inheritance is usually used for in Python (unlike many
other languages) is what creates the dissonance, IMHO.

> Well, anyway, let's not add too much to the current python-dev traffic 
> with this issue.  I think it has been brought up a couple of times
> already. 

Indeed, I think someone said what I just said above in this thread
already :-)

> Hey, i have an idea: why not create a python-syntax mailing list 
> (or python-hell :-) ? 

I think I've suggested that before :-)

Today is the first day I remember where there are more new messages
waiting for me in python-dev than comp.lang.python!

Cheers,
mwh

-- 
  "The future" has arrived but they forgot to update the docs.
                                        -- R. David Murray, 9 May 2000
From mcherm at mcherm.com  Fri Aug  6 13:04:18 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri Aug  6 13:04:11 2004
Subject: [Python-Dev] Call for defense of @decorators
Message-ID: <1091790258.411365b20c38a@mcherm.com>

Gustavo Niemeyer writes:
> Is there something to be done that could change the current
> decision? Voting? Collecting pros/cons and classifying them?

Yes, there is. And it's not voting <wink>.

Write a simple "white paper" summarizing the major arguments
and showing how they apply to the 3 or 4 top syntax proposals.
Solicit input from c.l.py or python-dev after you have a first
draft. After the 3rd draft is done, post it here and send it
to Guido. Then sit back and quietly put up with the resulting
decision whether you agree or not.

At this point, I long ago said most everything I had to say
and so I'm working on the "quietly put up with the results"
part myself.

-- Michael Chermside
From mal at egenix.com  Fri Aug  6 13:07:35 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Aug  6 13:07:38 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <200408052324.i75NOxA05160@guido.python.org>
References: <004a01c47b2b$65d19260$6700a8c0@computer>	<4112A4B9.4010907@v.loewis.de>	<000f01c47b39$de3bcd70$6700a8c0@computer>	<4112B6DC.4080605@v.loewis.de>	<000f01c47b40$cde05110$6700a8c0@computer>
	<4112BF9E.40003@v.loewis.de>
	<200408052324.i75NOxA05160@guido.python.org>
Message-ID: <41136677.9090103@egenix.com>

Guido van Rossum wrote:
>>In essence, you are proposing to reserve a new keyword. Solutions 
>>creating new keywords have been ruled out because they might break
>>existing code.
> 
> 
> Perhaps this could be addressed by requiring "from __future__ import
> decorators", for one release, just like was done for "yield".  I
> expect that this would be acceptable to the ObjC folks, too.  It
> wouldn't be my favorite, but I won't rule it out just because of the
> new keyword (and yes, this is a softening of my position on new
> keywords).

At EuroPython 2004 I had the impression that folks were not
really ready for a finalization on the feature. Your conclusion
at the conference was to wait for another release before making
a final decision which was well accepted by the audience.

With all the fuzz around this issue, I think that was a wise
decision and wonder why "something" now sneaked into 2.4a2.

Not being a big fan of decorative art, I'm a firm -0 on
whatever the final syntax may be, but just to throw in
an idea from someone who has not participated in all the
talk about decorator, here's some food for thought:

"""

1. Instead of trying to overgeneralize, we only allow exactly
    *one* decorator per function definition.

2. Since decorators change the definition of a function, that
    one decorator call is placed directly after the "def" and
    before the function name.

Examples:

def staticcounted(func):
     return staticmethod(countcalls(func))

class MyClass:

	def classmethod test1(selfclass, x):

	    """ Test method.
	    """
	    print x

	def staticcounted test2(self, x):

	    """ Test method.
	    """
	    print x

If someone needs more than one decorator or wants to do
funky stuff, it's easy enough to define a combining generator
for that particular need (see e.g. staticcounted).

The only argument I heard at the conference against placing
the decorator between the "def" and the function name was that
of losing the "grep'ability" in this case. I don't consider that
much of a loss, since it only applies to new code and
cases where you actually make use of decorators. The simple
nature of the syntax also guarantees that you can easily
modify your grep expression to accomodate for possible
decorators in front of the name (you'll only ever have one).

"""

If you don't like the idea, that's fine, but perhaps it
spurs some new thoughts of how to make this whole decorator
thing fit better the elegance of the language.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 06 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From edreamleo at charter.net  Fri Aug  6 13:08:10 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 13:08:21 2004
Subject: [Python-Dev] Re: The impact of '@' on Leo
References: <005e01c47b2d$9c0dee80$6700a8c0@computer>
	<eppstein-977D3E.15244105082004@sea.gmane.org>
Message-ID: <003901c47ba5$a937e750$6700a8c0@computer>

> I don't know what Leo is, but the usual solution to this sort of problem
> is to introduce some kind of quoting convention, e.g. @@ gets passed
> through as a single @ to Python, any other @something gets treated as a
> Leo directive.  Is there some reason that wouldn't work?

Yes, in Leo's case there is a strong reason for not using an "escape" or
"quoting" mechanism.  As I said in my original post, not using an escape
mechanism turns out to be the correct choice.

The basic problem is that there are two (or is it three?) views of the data
in the Leo world: the view presented to the user in the outline, the data in
the .leo file, and the data in the derived files.  Leo has a mechanism for
automatically converting data in derived files into the one or two other
forms.  This is call "untangling".  It turns out that it is extremely
difficult to handle escapes properly when untangling.  I know this sounds
weird, and it's true nevertheless.

If the @ syntax goes through, Leo will have to allow users to define their
own lead-in string, and the format of derived files will have to change a
bit to specify what the lead-in string is.  It's not a huge deal, I think.
It would surely be much easier than changing how Leo tangles or untangles
code.

BTW, I find the arguments for list-after-def to be persuasive, viz., that
annotations should clearly be part of a function.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From richie at entrian.com  Fri Aug  6 13:12:31 2004
From: richie at entrian.com (Richie Hindle)
Date: Fri Aug  6 13:08:32 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: 20040805232522.A1B8E1E400A@bag.python.org
Message-ID: <43p6h010jedicol36tbvf2t87c1pniqpbc@4ax.com>


[Shalabh]
> An alternative character (as others have suggested)
> might be good too. If so, I found '|' particularly nice :)
> 
>     |paramnames(username='Username', password='Password')
>     |webmethod(username=str, password=str)
>     def login(self, username, password, return_to=None):
>         request = current.request
>         ...

I much prefer this to @decorator.  In addition to the advantages that
Shalabh gives:

 + It doesn't break Leo, IPython, or any other tool that uses @ as a
   special character.

 + The association with pipes makes some sense: "take this thing and pass
   it through that thing to get a modified thing".

Perhaps someone with write access to the Wiki page at
http://www.python.org/moin/PythonDecorators could add this idea there?
It would be a shame if it got lost in the high-volume traffic here on
python-dev.  Unless there's some overwhelming technical reason why reusing
the '|' character isn't possible, of course.

-- 
Richie Hindle
richie@entrian.com

From edreamleo at charter.net  Fri Aug  6 13:29:35 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 13:29:43 2004
Subject: [Python-Dev] The impact of '@' on Leo
References: <200408060241.i762frUH000947@cosc353.cosc.canterbury.ac.nz>
	<200408052356.29328.fdrake@acm.org>
Message-ID: <006b01c47ba8$a6bf8cf0$6700a8c0@computer>

> I've really only managed to look at Leo much once, and Edward was sitting
next
> to me at the time.

Hi Fred.  Sitting next to you at the sprint was one of the most enlightening
things I have done in a long time.  I hope the next time we sprint together
you'll be able to use Leo much more easily with cvs.  I've spent that last
six months on @thin, which should help a lot...

>  Leo doesn't store Python code in a Leo document as any
> sort of embedded object that's labeled as Python code as far as I can tell
> (though it may in it's data model); it just more text, and can have
> @directives like any other text.  I think this is inherently a risky
approach
> to the data; there needs to be at least a way to say that some portion of
the
> data is some other text format embedded into the Leo document.

.leo files are XML files.  The only thing that matters in that context is
that the xml escapes are handled properly.  What matters for the integrity
of data in derived files is that uses don't mess with the so-called sentinel
lines.  These lines "carry" the outline structure.  Destroy those lines,
destroy something essential.

Leo has a number of ways of recovering partially from such destruction, but
the fact remains that uses insert, delete, fold or mutilate sentinel lines
at their own risk.  It's as simple as that.  Sentinel lines start with #@ at
present, and it's conceivable that they will always do so, even if @ gets
used for something in Python.

> I'm sure Edward can find a way to make things work for Leo users, but I
hope
> the .leo files have some sort of format version indicator to make
upgrading
> relatively easy for Leo users.

Actually, both .leo files and derived files have various format indicators.
However, the format of .leo files hasn't changed significantly enough to
need a new number in years.  Allowing different lead-in strings for Leo
directives will have absolutely no effect on the format of .leo files.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From amk at amk.ca  Fri Aug  6 13:34:26 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Aug  6 13:34:41 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112B5B4.9@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<4112A18D.5090801@v.loewis.de>
	<002301c47b3a$469bc5f0$6700a8c0@computer> <4112B5B4.9@v.loewis.de>
Message-ID: <20040806113426.GA30824@rogue.amk.ca>

On Fri, Aug 06, 2004 at 12:33:24AM +0200, "Martin v. L?wis" wrote:
> it's just the PEP that needs updating (a section in whatsnew also needs
> to be drafted). 

The current version of whatsnew already includes a section on
decorators.  That section, at
http://www.python.org/dev/doc/devel/whatsnew/node5.html, may be useful
as an overview of what's currently implemented in 2.4a2, given that
the PEP isn't up-to-date.

--amk

From mwh at python.net  Fri Aug  6 13:36:55 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 13:36:56 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112A746.6010501@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Thu,
	05 Aug 2004 23:31:50 +0200")
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de> <1091740792.8546.133.camel@localhost>
	<4112A746.6010501@v.loewis.de>
Message-ID: <2m657w7b1k.fsf@starship.python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> Barry Warsaw wrote:
>> Martin makes a good point.  Guido could threaten to remove the feature
>> by beta 1 (and thus for 2.4 final) if the PEP is not brought up to date.
>> not-that-i'm-volunteering-'cause-i'd-rather-see-it-done-ly y'rs,
>
> I'm atleast volunteering to take the code out if no proper
> implementation (including documentation and test cases) is delivered by
> some deadline.

Uhh, there *are* test cases (test_decorators).  I'm fairly sure
there's at least ref manual documentation (in ref7.tex).  Possibly
there should be something elsewhere -- in the description of
classmethod?

Or are you talking in the abstract here?

Cheers,
mwh

-- 
  "The future" has arrived but they forgot to update the docs.
                                        -- R. David Murray, 9 May 2000
From mwh at python.net  Fri Aug  6 13:40:23 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 13:40:25 2004
Subject: [Python-Dev] OK to lower timeout for test_timeout's
	testConnectTimeout test?
In-Reply-To: <4112A365.1000203@ocf.berkeley.edu> (Brett Cannon's message of
	"Thu, 05 Aug 2004 14:15:17 -0700")
References: <4112A365.1000203@ocf.berkeley.edu>
Message-ID: <2m1xik7avs.fsf@starship.python.net>

Brett Cannon <bac@OCF.Berkeley.EDU> writes:

> P.S.: the only other failures I have on OS X right now is test_curses
> (because I used -uall) and test__locale .  Should we disable
> test__locale on OS X to shut it up since Martin seems to think the
> test isn't all that useful and won't work for OS X ever?

ONE of these days, I'll get around to doing something about
test_curses... the test is being silly (or at least, that's what I
remember from the last time I dug into this).

I'm still seeing test_socket fail on OS X, though (Jaguar,
testGetServByName).

Cheers,
mwh

-- 
  Ignoring the rules in the FAQ: 1" slice in spleen and prevention 
    of immediate medical care.
                                              -- Mark C. Langston, asr
From mwh at python.net  Fri Aug  6 13:42:46 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 13:42:48 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <200408060833.43982.gmccaughan@synaptics-uk.com> (Gareth
	McCaughan's message of "Fri, 6 Aug 2004 08:33:43 +0100")
References: <200408060035.i760Z5PE000754@cosc353.cosc.canterbury.ac.nz>
	<200408060833.43982.gmccaughan@synaptics-uk.com>
Message-ID: <2mwu0c5w7d.fsf@starship.python.net>

Gareth McCaughan <gmccaughan@synaptics-uk.com> writes:

> On Friday 2004-08-06 01:35, Greg Ewing wrote:
>> > >> class C(random.choice([dict, list])):
>> > >>     pass
>> > >>
>> > >> is my favourite example of this :-)
>> > >
>> > > Where is THAT monstrocity used!?
>> > 
>> > Well, it's not, I hope :-)
>> 
>> Hmmm... maybe if you had some algorithm that was more efficient
>> using either a list or a dict, depending on the input data, but
>> in a way that makes it difficult to tell ahead of time which
>> to use, and you want to improve the average behaviour...
>
> ... then you'd do better to find which out of (dict,list)
> does better on average, and always use that one. Sorry. :-)

By running the above code on a wide range of inputs? :-)

Cheers,
mwh

-- 
  To summarise the summary of the summary:- people are a problem.
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 12
From marktrussell at btopenworld.com  Fri Aug  6 13:43:25 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Fri Aug  6 13:43:27 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <2m657w7b1k.fsf@starship.python.net>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<1091740792.8546.133.camel@localhost> <4112A746.6010501@v.loewis.de>
	<2m657w7b1k.fsf@starship.python.net>
Message-ID: <1091792604.1770.8.camel@localhost>

On Fri, 2004-08-06 at 12:36, Michael Hudson wrote:
> Possibly
> there should be something elsewhere -- in the description of
> classmethod?

That's already done - I updated the classmethod and staticmethod
descriptions to use the new syntax, with a pointer to the ref manual
desctiption.  I think it is just the PEP that needs updating.

Mark Russell

From rnd at onego.ru  Fri Aug  6 13:44:54 2004
From: rnd at onego.ru (Roman Suzi)
Date: Fri Aug  6 13:45:06 2004
Subject: [Python-Dev] Questions about '@' in pep 318
Message-ID: <Pine.LNX.4.58.0408061536370.3743@suzi.com.onego.ru>


Hello,

I think @ before def are unacceptable because they aren't too Pythonic
and thus break style and aestetics of Python. If we absolutely need
those decorators, I propose to use existing construct which has some
resemblance with other places of Python:

def func(x, y, z) % decorator:
  ...


and

def func(x, y, z) % (decorator1, decorator2, ...):
  ...


this has a meaning of "applying" decorators to the function (method).
And it has minimal impact on readability.


Sincerely yours, Roman A.Suzi
-- 
 - Petrozavodsk - Karelia - Russia - mailto:rnd@onego.ru -

From martin at v.loewis.de  Fri Aug  6 13:53:22 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 13:53:26 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <2m657w7b1k.fsf@starship.python.net>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>
	<1091740792.8546.133.camel@localhost>	<4112A746.6010501@v.loewis.de>
	<2m657w7b1k.fsf@starship.python.net>
Message-ID: <41137132.8060506@v.loewis.de>

Michael Hudson wrote:
> Or are you talking in the abstract here?

I was talking in a concrete, uninformed way. The tests should have
been easy to find, although I had to grep to find the documentation
in the language reference.

Regards,
Martin
From martin at v.loewis.de  Fri Aug  6 13:56:28 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 13:56:32 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <Pine.LNX.4.58.0408061536370.3743@suzi.com.onego.ru>
References: <Pine.LNX.4.58.0408061536370.3743@suzi.com.onego.ru>
Message-ID: <411371EC.10403@v.loewis.de>

Roman Suzi wrote:
> I think @ before def are unacceptable because they aren't too Pythonic
> and thus break style and aestetics of Python. 

This is not a convincing statement. Who is going to define what is
Pythonic and what is not? My understanding of that term was that it
can only refer to programs (e.g. a function could do something in
a Pythonic way - namely, if that is how other Python practitioners
would have written it). The language itself is by nature Pythonic -
even if it would add curly braces. Of course, Pythonic version 2.4
differs from Pythonic version 1.4.

Regards,
Martin
From gmccaughan at synaptics-uk.com  Fri Aug  6 14:00:44 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Fri Aug  6 14:01:15 2004
Subject: [Python-Dev] 2.4a2, and @decorators
In-Reply-To: <2mwu0c5w7d.fsf@starship.python.net>
References: <200408060035.i760Z5PE000754@cosc353.cosc.canterbury.ac.nz>
	<200408060833.43982.gmccaughan@synaptics-uk.com>
	<2mwu0c5w7d.fsf@starship.python.net>
Message-ID: <200408061300.44475.gmccaughan@synaptics-uk.com>

On Friday 2004-08-06 12:42, Michael Hudson wrote:

>>>>>> class C(random.choice([dict, list])):
>>>>>>     pass
>>>>>>
>>>>>> is my favourite example of this :-)
>>>>>
>>>>> Where is THAT monstrocity used!?
>>>>
>>>> Well, it's not, I hope :-)
>>>
>>> Hmmm... maybe if you had some algorithm that was more efficient
>>> using either a list or a dict, depending on the input data, but
>>> in a way that makes it difficult to tell ahead of time which
>>> to use, and you want to improve the average behaviour...
>>
>> ... then you'd do better to find which out of (dict,list)
>> does better on average, and always use that one. Sorry. :-)
>
> By running the above code on a wide range of inputs? :-)

<boring>No, you'd get more reliable results for a given expenditure
of effort by running half as many input datasets with class C(dict)
and class C(list).</boring>

-- 
g


From amk at amk.ca  Fri Aug  6 14:04:05 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Aug  6 14:04:24 2004
Subject: [Python-Dev] OK to lower timeout for test_timeout's
	testConnectTimeout test?
In-Reply-To: <2m1xik7avs.fsf@starship.python.net>
References: <4112A365.1000203@ocf.berkeley.edu>
	<2m1xik7avs.fsf@starship.python.net>
Message-ID: <20040806120405.GB30824@rogue.amk.ca>

On Fri, Aug 06, 2004 at 12:40:23PM +0100, Michael Hudson wrote:
> ONE of these days, I'll get around to doing something about
> test_curses... the test is being silly (or at least, that's what I
> remember from the last time I dug into this).

What's test_curses doing wrong?

--amk
From rnd at onego.ru  Fri Aug  6 14:13:30 2004
From: rnd at onego.ru (Roman Suzi)
Date: Fri Aug  6 14:13:44 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <411371EC.10403@v.loewis.de>
References: <Pine.LNX.4.58.0408061536370.3743@suzi.com.onego.ru>
	<411371EC.10403@v.loewis.de>
Message-ID: <Pine.LNX.4.58.0408061610020.3743@suzi.com.onego.ru>


By "pythonic" I mean "in accordance with other Python syntax".
And I am not against '@', but rather against placing decorators
before the def. IMHO, it is confusing, error-prone and out of style.

Sincerely yours, Roman A.Suzi
-- 
 - Petrozavodsk - Karelia - Russia - mailto:rnd@onego.ru -

On Fri, 6 Aug 2004, [ISO-8859-1] "Martin v. L?wis" wrote:

> Roman Suzi wrote:
> > I think @ before def are unacceptable because they aren't too Pythonic
> > and thus break style and aestetics of Python.
>
> This is not a convincing statement. Who is going to define what is
> Pythonic and what is not? My understanding of that term was that it
> can only refer to programs (e.g. a function could do something in
> a Pythonic way - namely, if that is how other Python practitioners
> would have written it). The language itself is by nature Pythonic -
> even if it would add curly braces. Of course, Pythonic version 2.4
> differs from Pythonic version 1.4.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/rnd%40onego.ru
>
From della at linux.it  Fri Aug  6 14:32:13 2004
From: della at linux.it (Matteo Dell'Amico)
Date: Fri Aug  6 14:33:17 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <1091746327.1419.18.camel@localhost>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<002301c47b3a$469bc5f0$6700a8c0@computer>  <4112B5B4.9@v.loewis.de>
	<1091746327.1419.18.camel@localhost>
Message-ID: <1091795533.4501.6.camel@marvin.linux.it>

Il ven, 2004-08-06 alle 00:52, Mark Russell ha scritto:
> Good thought.  Or even something like:
> 
> 	|staticmethod|
> 	def foo(): pass
> 
> I just tried converting test_decorators.py to this syntax, and it looks
> quite nice to my eye.

I like it.

Probably, I'd like even more:

<staticmethod, foomethod>
def foo(): pass

I think these kind of proposal is extetically pleasing to my eye, has
more or less all the pros of the list-before-def syntax, and wouldn't
have its main drawback: the "magic" behaviour of evaluating a string,
since <a, b, c> is no python object.

A drawback would be that no mainstream language uses this syntax, but I
think the C# one is sufficiently similar.

(and excuse me for proposing Just Another Syntax :-))

-- 
Ciao,
Matteo

From skip at pobox.com  Fri Aug  6 14:45:42 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 14:46:04 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <E1BsvdS-0001ZJ-00@swing.co.at>
References: <20040805202245.GH29796@burma.localdomain>
	<E1BsvdS-0001ZJ-00@swing.co.at>
Message-ID: <16659.32118.414524.314310@montanaro.dyndns.org>


    Christian> To all proponents of list markup: that would be very hard to
    Christian> look up in the documentation. `@` should be easy to find in
    Christian> the index.

It would be easy to find if you search for "def" or "function"...  This idea
has been raised before and is a red herring in my opinion.

Skip
From della at linux.it  Fri Aug  6 14:48:26 2004
From: della at linux.it (Matteo Dell'Amico)
Date: Fri Aug  6 14:49:29 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <1091795533.4501.6.camel@marvin.linux.it>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<002301c47b3a$469bc5f0$6700a8c0@computer>  <4112B5B4.9@v.loewis.de>
	<1091746327.1419.18.camel@localhost>
	<1091795533.4501.6.camel@marvin.linux.it>
Message-ID: <1091796505.4501.8.camel@marvin.linux.it>

Il ven, 2004-08-06 alle 14:32, Matteo Dell'Amico ha scritto:
> Probably, I'd like even more:
> 
> <staticmethod, foomethod>
> def foo(): pass

Please ignore me. I read the PEP, but I managed to forget it while
reading python-dev. :-(

-- 
Ciao,
Matteo

From john at neggie.net  Fri Aug  6 14:54:45 2004
From: john at neggie.net (John Belmonte)
Date: Fri Aug  6 14:54:54 2004
Subject: [Python-Dev] Would like to add Edward Loper as a Python developer
In-Reply-To: <20040721211609.8FA333B8038@smtp.zope.com>
References: <20040721211609.8FA333B8038@smtp.zope.com>
Message-ID: <41137F95.4040606@neggie.net>

Tim Peters wrote:
> better support for writing "whole file" doctests (tutorial narratives
> liberally illustrated with Python examples -- Zope3 is finding these very
> helpful); and documenting all the additions that snuck into Python 2.3.

I noticed that the "whole file doctests" item didn't make it to the 
DoctestIdeas page.  I was looking for more details about that.

I've found that for tutorial narratives, I really don't want to bother 
with expected output.  For complex software, the output may not always 
be the same anyway.  Also, it's not fun to manipulate input code that 
has prompts and is within a multi-line string.  Syntax highlighting is 
not available, for one thing.

I've made my own tutorial generator.  Given this input script:

     ## Double hash comments treated as meta comments and not output to
     ## the tutorial.

     """
     Triple-quote comments that start in the first column are treated as
     tutorial text.

     First we'll import some modules:
     """
     import os

     """
     Some code follows.  In addition to hash comments, and triple-quote
     comments not starting in the first column are preserved.  Note that
     code blocks such as functions must end with a blank line, just as
     is done in the interactive interpreter.
     """

     def foo():
         """foo description here"""
         print 'hello'  # be friendly

     #my_path = 'foo/bar'
     my_path = os.path.join('foo', 'bar')
     my_path
     foo()

the output is:

     Triple-quote comments that start in the first column are treated as
     tutorial text.

     First we'll import some modules:

         >>> import os

     Some code follows.  In addition to hash comments, and triple-quote
     comments not starting in the first column are preserved.  Note that
     code blocks such as functions must end with a blank line, just as
     is done in the interactive interpreter.

         >>> def foo():
         ...     """foo description here"""
         ...     print 'hello'  # be friendly
         ...
         >>> #my_path = 'foo/bar'
         >>> my_path = os.path.join('foo', 'bar')
         >>> my_path
         'foo/bar'
         >>> foo()
         hello


-John


-- 
http://giftfile.org/  ::  giftfile project
From skip at pobox.com  Fri Aug  6 14:55:52 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 14:56:30 2004
Subject: [Python-Dev] Re: Exceptional inheritance patterns
In-Reply-To: <2macx87es5.fsf@starship.python.net>
References: <20040805153443.GB5208@solar.trillke>
	<200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz>
	<20040806074509.GD5208@solar.trillke>
	<2macx87es5.fsf@starship.python.net>
Message-ID: <16659.32728.411134.439644@montanaro.dyndns.org>


    Michael> I would like an exception class that almost exceptions except
    Michael> KeyboardInterrupt, SystemExit and -- maybe -- RuntimeError and
    Michael> MemoryError inherited from.

    Michael> except ExceptionsButThoseNastyOnesIDontWantToCatch:
    Michael>     pass

    Michael> ? <wink>

I proposed a change to the exceptions hierarchy a few years ago that would
allow this.  It was obviously never implemented, but I no longer remember
why.

Skip
From skip at pobox.com  Fri Aug  6 15:00:58 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 15:01:16 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <43p6h010jedicol36tbvf2t87c1pniqpbc@4ax.com>
References: <43p6h010jedicol36tbvf2t87c1pniqpbc@4ax.com>
Message-ID: <16659.33034.4995.438267@montanaro.dyndns.org>

    Richie> Perhaps someone with write access to the Wiki page at
    Richie> http://www.python.org/moin/PythonDecorators could add this idea
    Richie> there?

Anyone can scribble on the Wiki.  You just have to be logged in.  Poke the
UserPreferences link to sign up.  If you had previously been registered it
appears all login info was lost with the latest Moin upgrade on
www.python.org.

Skip
From skip at pobox.com  Fri Aug  6 15:18:33 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 15:18:49 2004
Subject: [Python-Dev] PEP 318 - writing to func_name attribute
Message-ID: <16659.34089.645670.39263@montanaro.dyndns.org>


One thing I added awhile back to PEP 318 as an open issue is that for ease
of writing wrappers the func_name attribute of a function object should be
writable.  For example, in situations where the decorator returns a new
function object (often, I would think) it would really be nice if that new
function object had the same name as the undecorated function.  Consider:

    def a(func):
        def _inner(*args, **kwds):
            return func(*args, **kwds)
        return _inner

    @a
    def func(*args, **kwds):
        print args
        print kwds

    print "func's name:", func.func_name

I realize you can use new.function() to create a new function object, but
that seems like a fair amount of extra complexity just to change the
function's name.  I'd prefer it if the decorator could be written as:

    def a(func):
        def _inner(*args, **kwds):
            return func(*args, **kwds)
        _inner.func_name = func.func_name
        return _inner

That fails because func_name is readonly.  Any chance this restriction can
be loosened up?

Skip
From skip at pobox.com  Fri Aug  6 15:21:02 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 15:21:20 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <1091792604.1770.8.camel@localhost>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<1091740792.8546.133.camel@localhost>
	<4112A746.6010501@v.loewis.de> <2m657w7b1k.fsf@starship.python.net>
	<1091792604.1770.8.camel@localhost>
Message-ID: <16659.34238.443364.229226@montanaro.dyndns.org>


    Mark> I think it is just the PEP that needs updating.

Not to pick on Mark, but everyone seems to think the PEP needs updating but
nobody's stepping up to offer patches.  Anthony and I are both about out of
round tuits...

Skip

From mwh at python.net  Fri Aug  6 15:39:25 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 15:39:26 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com>
	(Phillip J. Eby's message of "Thu, 05 Aug 2004 15:45:35 -0400")
References: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com>
Message-ID: <2msmb05qsy.fsf@starship.python.net>

"Phillip J. Eby" <pje@telecommunity.com> writes:

> At 12:24 PM 8/5/04 -0700, IxokaI wrote:
>
>>I added "with", although I havn't seen it.
>
> Guido's reserving "with" for this purpose in some future Python:
>
>      with x.y:
>          .z = spam    # set x.y.z = spam
>          print .q.r   # print x.y.q.r

Except that the only extant PEP involving with actually uses it for
something else :-)

I think talking about what Guido is or isn't doing is a bit ... wrong?

Cheers,
mwh

-- 
41. Some programming languages manage to absorb change, but
    withstand progress.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html
From barbidule at melshake.com  Fri Aug  6 15:29:27 2004
From: barbidule at melshake.com (maxime B)
Date: Fri Aug  6 15:40:10 2004
Subject: [Python-Dev] Re: Tuple/list assignment question
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8C000@au3010avexu1.global.avaya.com>
Message-ID: <loom.20040806T152506-752@post.gmane.org>

Delaney, Timothy C (Timothy <tdelaney <at> avaya.com> writes:
> I think it's about time this proposal got a PEP of its own ... something
> like "Enhanced Tuple Unpacking" (to match "Enhanced Argument Tuples" ...

Even the PEP would be rejected, I think it's a good idea to
have the definitive answer of the BDFL.

Maxime

From tanzer at swing.co.at  Fri Aug  6 15:42:50 2004
From: tanzer at swing.co.at (Christian Tanzer)
Date: Fri Aug  6 15:43:45 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: Your message of "Fri, 06 Aug 2004 07:45:42 CDT."
	<16659.32118.414524.314310@montanaro.dyndns.org>
Message-ID: <E1Bt4zi-0003Sr-00@swing.co.at>


> Christian> To all proponents of list markup: that would be very hard to
> Christian> look up in the documentation. `@` should be easy to find in
> Christian> the index.
>
> It would be easy to find if you search for "def" or "function"...  This idea
> has been raised before and is a red herring in my opinion.

I know about looking at the documentation of `def`. But you are
assuming here that the puzzled reader associates the (leading) list
markup with the function definition.

The trailing list markup (that I personally favored) hasn't been in
contention since May? June?

Anyway, the pie syntax offers more index entries than the list syntax.

Tuning-out-for-good-now-ly yrs,

-- 
Christian Tanzer                                    http://www.c-tanzer.at/

From mwh at python.net  Fri Aug  6 15:44:07 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 15:44:09 2004
Subject: [Python-Dev] OK to lower timeout for test_timeout's
	testConnectTimeout test?
In-Reply-To: <20040806120405.GB30824@rogue.amk.ca> (A. M. Kuchling's message
	of "Fri, 6 Aug 2004 08:04:05 -0400")
References: <4112A365.1000203@ocf.berkeley.edu>
	<2m1xik7avs.fsf@starship.python.net>
	<20040806120405.GB30824@rogue.amk.ca>
Message-ID: <2moelo5ql4.fsf@starship.python.net>

"A.M. Kuchling" <amk@amk.ca> writes:

> On Fri, Aug 06, 2004 at 12:40:23PM +0100, Michael Hudson wrote:
>> ONE of these days, I'll get around to doing something about
>> test_curses... the test is being silly (or at least, that's what I
>> remember from the last time I dug into this).
>
> What's test_curses doing wrong?

I think the problem is that it sometimes calls "curses.curs_set(1)"
which doesn't work (returns ERR) when certain definitions (cuvis, I
guess) aren't in the terminfo database, which gets translated into a
Python exception.

Cheers,
mwh

-- 
 All parts should go together without forcing. You must remember that
 the parts you are  reassembling were disassembled by you.  Therefore,
 if you can't get them together again, there must be a reason. By all
 means, do not use a hammer.           -- IBM maintenance manual, 1925
From richie at entrian.com  Fri Aug  6 15:52:37 2004
From: richie at entrian.com (Richie Hindle)
Date: Fri Aug  6 15:48:37 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
Message-ID: <au27h0dmpkdcv4hh228m484pib1349531s@4ax.com>


[Skip]
> Anyone can scribble on the Wiki. You just have to be logged in. Poke the
> UserPreferences link to sign up

Thanks, Skip (and amk, who emailed me the same info).  The page said it was
immutable, and I didn't realise I could change that by logging in.

I've added Shalabh's "|decorator" syntax to the wiki page at
http://www.python.org/moin/PythonDecorators

-- 
Richie Hindle
richie@entrian.com

From edreamleo at charter.net  Fri Aug  6 15:50:08 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 15:50:17 2004
Subject: [Python-Dev] Questions about '@' in pep 318
References: <Pine.LNX.4.58.0408061536370.3743@suzi.com.onego.ru>
	<411371EC.10403@v.loewis.de>
Message-ID: <007d01c47bbc$49a2a340$6700a8c0@computer>

> > I think @ before def are unacceptable because they aren't too Pythonic
> > and thus break style and aesthetics of Python.
>
> This is not a convincing statement.

I agree.  The argument that @ lines aren't clearly "attached" to functions
seems much more convincing to me.

Surely I am repeating somebody else's points, but I'm going to say them
anyway:

1.  There doesn't seem much debate that dropping @ from the list of symbols
that can never appear in Python programs (outside of comments and strings)
is, in fact, a major change to the language.

2.  Maybe the @ syntax is somehow the best alternative, considered in
isolation.  Fair enough.  But how much better?  Enough to justify a major
change to the language?  What is so unreadable about def (...)[...]: ?

3. I agree with Phillip Eby that an @ line looks like some external document
markup.  This is, if fact, a common usage for the @ sign: Leo uses the @
sign for this reason.  But to be fair, this is a very minor point.

4. No doubt Python programmers will grow used to any syntax whatever it is,
including @ or def (...)[...].  Whatever slight discomfort people will feel
with a new syntax will quickly fade.

In short, a major change to Python (@) would seem to demand a truly
convincing justification.  IMO, the argument that @ is more readable than
def (...)[...]: doesn't even come _close_ to such a justification.  YMMV.
And only GvR's mileage counts :-)

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From mwh at python.net  Fri Aug  6 16:15:05 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 16:15:08 2004
Subject: [Python-Dev] OK to lower timeout for test_timeout's
	testConnectTimeout test?
In-Reply-To: <2moelo5ql4.fsf@starship.python.net> (Michael Hudson's message
	of "Fri, 06 Aug 2004 14:44:07 +0100")
References: <4112A365.1000203@ocf.berkeley.edu>
	<2m1xik7avs.fsf@starship.python.net>
	<20040806120405.GB30824@rogue.amk.ca>
	<2moelo5ql4.fsf@starship.python.net>
Message-ID: <2mk6wc5p5i.fsf@starship.python.net>

Michael Hudson <mwh@python.net> writes:

> "A.M. Kuchling" <amk@amk.ca> writes:
>
>> On Fri, Aug 06, 2004 at 12:40:23PM +0100, Michael Hudson wrote:
>>> ONE of these days, I'll get around to doing something about
>>> test_curses... the test is being silly (or at least, that's what I
>>> remember from the last time I dug into this).
>>
>> What's test_curses doing wrong?
>
> I think the problem is that it sometimes calls "curses.curs_set(1)"
> which doesn't work

Argh; it *always* calls curses.curs_set(1) which *sometimes* doesn't
work...

> (returns ERR) when certain definitions (cuvis, I guess) aren't in
> the terminfo database, which gets translated into a Python
> exception.
>

-- 
  About the use of language: it is impossible to sharpen a
  pencil with a blunt axe.  It is equally vain to try to do
  it with ten blunt axes instead.
      -- E.W.Dijkstra, 18th June 1975. Perl did not exist at the time.
From mwh at python.net  Fri Aug  6 16:20:52 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 16:20:55 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <003601c47b34$0b4bf660$6700a8c0@computer> (Edward K. Ream's
	message of "Thu, 5 Aug 2004 16:34:52 -0500")
References: <004a01c47b2b$65d19260$6700a8c0@computer>
	<20040805205559.GA31288@burma.localdomain>
	<003601c47b34$0b4bf660$6700a8c0@computer>
Message-ID: <2mfz705ovv.fsf@starship.python.net>

"Edward K. Ream" <edreamleo@charter.net> writes:

>> > annotate.accepts(int, (int,float))
>>
>> I don't understand. How are you changing/annotating the function?
>
> I'm glad you asked.  My suggestion was that the compiler turn
>
> annotate.accepts(int, (int,float))
>
> into exactly what it would have turned
>
> @accepts(int, (int,float))
>
> into.

I don't like this sort of idea: something that's done by the compiler
should *look* like it is being done by the compiler, i.e. be new
syntax.

Cheers,
mwh

-- 
  GAG: I think this is perfectly normal behaviour for a Vogon. ...
VOGON: That is exactly what you always say.
  GAG: Well, I think that is probably perfectly normal behaviour for a
      psychiatrist. -- The Hitch-Hikers Guide to the Galaxy, Episode 9
From guido at python.org  Fri Aug  6 16:22:55 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 16:23:01 2004
Subject: [Python-Dev] PEP 237
In-Reply-To: Your message of "Fri, 06 Aug 2004 18:11:12 +1000."
	<41133D20.8040307@interlink.com.au> 
References: <4112DD1C.3060902@interlink.com.au> <411332AA.8000606@v.loewis.de>
	<41133D20.8040307@interlink.com.au> 
Message-ID: <200408061422.i76EMu406927@guido.python.org>

> A couple of people have commented that the int/long PEP,
> for instance, is not up-to-date. I don't think this means
> that the changes should be backed out.

I've seen this particular PEP brought up a couple of times, but I
don't see any significant inaccuracies in that PEP (except for some
uses of "currently" referring to Python versions before 2.2).

I've tried my best to keep the implementation sections up to date as
our timeline and semantics changed.

So please find another example.  (And if the *only* reason to back out
@decorators ever becomes the lack of a PEP champion, I'll champion it
myself. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From edreamleo at charter.net  Fri Aug  6 16:16:52 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 16:28:12 2004
Subject: [Python-Dev] PEP 318, and the PEP process
References: <4112DD1C.3060902@interlink.com.au> <411332AA.8000606@v.loewis.de>
Message-ID: <008b01c47bc0$0550dfa0$6700a8c0@computer>

> I do firmly think this is sufficient reason. It means
> there is no champion for the PEP, so there can't be
> a new feature. In essence, the champion is the one
> who "controls" the feature, and who is in charge of
> making it work well. He is the one to complain to,
> and he will make sure issues get resolved (not necessarily
> resolving them himself).

Python has matured into a language in which many people and companies have
significant investments.  Changes to Python matter.

In this environment, making haste slowly makes more and more sense.  In
particular, not having pep's reflect reality is likely to waste a lot of
time.  It also just doesn't seem like proper procedure.

This is a highly sensitive topic.  I have the highest regard for Python's
volunteers, and I have no intention of even suggesting what they should or
shouldn't do.  It is for GvR to pronounce, if he pleases.

The present discussion of pep 318 shows the pitfalls in not treating Python
as the supremely important language that it clearly is.  I served, very
briefly, on the ANSI C Standards Committee about 10 years ago.  Clearly,
Python needn't follow the extremely picky rules for public disclosure and
public comment that are de rigueur in the Standards world.  However, it
might help to start shading just a bit in that direction, if only to head
off the kind of public outcry we have had with 318.

As I said on c.l.py, it does not seem fair to claim to be discussing a
proposal publicly if it's pep does not reflect reality.  The reason is
simple:  people like me will ignore discussion on python-dev if the pep does
not seem to concern them.  It's like announcing a meeting to discuss sewers,
then using that (public) meeting to give all the county commissioners a
raise.  The open meeting law has been violated.

Edward

P.S.  To respond to the original quote :-)  My own opinion is the lack of a
champion is more than sufficient reason to delay or kill a feature (or a
pep).  To repeat: changes to Python matter.  If there isn't sufficient will
to keep peps up-to-date, how can we be assured of the quality of the design,
or of the code?  Again, I have no right to demand that this criterion go
into effect.

EKR
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From mwh at python.net  Fri Aug  6 16:36:11 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 16:36:13 2004
Subject: [Python-Dev] pre-PEP: Complete, Structured Regular Expression
	Group Matching
In-Reply-To: <lkdekmlvv8k.fsf@panix2.panix.com> (Mike Coleman's message of
	"05 Aug 2004 22:46:51 -0400")
References: <lkdekmlvv8k.fsf@panix2.panix.com>
Message-ID: <2mbrho5o6c.fsf@starship.python.net>

Mike Coleman <mkc@mathdogs.com> writes:

> If you've ever been frustrated that there wasn't an easy way to parse a string
> or file, even though you could easily match it with re.match, then this PEP
> may be for you.  If what's being proposed seems a little ghastly, well, I'll
> just say that I've been wanting something like this for a long time and this
> is the best I could come up with.  Your comments invited.

Generally, it strikes me as mildly useful.  An implementation would
surely help :-)

Presumably 'structmatch' would also become a metho on pattern objects?

> Additional Features
> -------------------
>
> In many cases in which ``'re.structmatch'`` fails to match, the cause will
> be due to an unexpected error in the format of the string being
> matched.  In order to assist the calling program in generating a more
> meaningful possible error message, ``'re.structmatch'`` will return the
> endpoint of the largest match against the searched string.  So, for
> example ::
>
>         re.structmatch('abcd', 'abxxx') --> 2
>         re.structmatch('abcde|z', 'abxxx') --> 2
>         re.structmatch('x*?y', 'xxx') --> 3
>
> (This return value should be considered advisory rather than exact, as
> future improvements in the match algorithm may make it difficult to
> calculate the exact value.)

I don't really like this idea; it seems to be that it would be more
helpful to raise an exception and attach this data to the exception.
But that's a bit inconsisent with how match and search work.

Cheers,
mwh

-- 
  I have a cat, so I know that when she digs her very sharp claws into
  my chest or stomach it's really a sign of affection, but I don't see
  any reason for programming languages to show affection with pain.
                                        -- Erik Naggum, comp.lang.lisp
From nhodgson at bigpond.net.au  Fri Aug  6 16:44:58 2004
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Fri Aug  6 16:45:06 2004
Subject: [Python-Dev] Questions about '@' in pep 318
References: <Pine.LNX.4.58.0408061536370.3743@suzi.com.onego.ru>
	<411371EC.10403@v.loewis.de>
	<007d01c47bbc$49a2a340$6700a8c0@computer>
Message-ID: <079701c47bc3$f2146ac0$034b8890@neil>

Edward K. Ream:

> 1.  There doesn't seem much debate that dropping @ from the list of
symbols
> that can never appear in Python programs (outside of comments and strings)
> is, in fact, a major change to the language.

   I do not believe using these symbols is a major change to the language.
To me '@', '$' and '?' are currently unused symbols that may be used when
there is a good reason to use them. In the past, there has been discussion
about using '?' for inline if, '$' for string interpolation, and '@' for
extended operators in IIRC matrix operations.

   Neil

From edreamleo at charter.net  Fri Aug  6 16:44:47 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 16:45:22 2004
Subject: [Python-Dev] Questions about '@' in pep 318
Message-ID: <016c01c47bc3$ec1f7420$6700a8c0@computer>

> Please do not mislead people into thinking that a discussion of syntax
alone is of any value, *unless* there is a willingness on the part
of those against the "@" syntax to compromise on a single alternative.

This statement seems a bit too strong, for the following reasons:

1. Adding @ impacts major external tools, negatively.
2. Adding @ is, in fact, a major change to the language.
3. The "willingness to compromise" is not really the driving force here: GvR
is the only vote that matters.

I have stated that Leo can probably live with the at sign.  I have also
stated elsewhere my reasons for not being particularly happy with it.

For the record, I am +0 on the semantics.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From guido at python.org  Fri Aug  6 16:45:51 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 16:45:57 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: Your message of "Fri, 06 Aug 2004 20:58:27 +1000."
	<41136453.1040908@interlink.com.au> 
References: <41136453.1040908@interlink.com.au> 
Message-ID: <200408061445.i76EjqT07026@guido.python.org>

[Anthony]
> I'm having trouble finding a single piece from you where you
> outline the problems you have with the [decorators] after args,
> before :. Could you possibly post a message to python-dev
> stating your problems with it? It's the major one I've been
> unable to find your objections to.

And nobody else has expressed their objections to it either? :-)

Just look at it when either the list of decorators or the argument
list doesn't fit on one line:

class C(object):

    def longMethodNameForEffect(longArgumentOne=None,
                                longArgumentTwo=42) [
        staticmethod,
        funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
                  status="experimental", author="BDFL")
        ]:
        """This method blah, blah.

        It supports the following arguments:
	- longArgumentOne -- a string giving ...
        - longArgumentTwo -- a number giving ...

	blah, blah.

	"""
        raise NotYetImplemented

That's a total jumble of stuff ending with a smiley.  (True story: I
left out the colon when typing up this example and only noticed in
proofing.)

Problems with this form:

- it hides crucial information (e.g. that it is a static method)
  after the signature, where it is easily missed

- it's easy to miss the transition between a long argument list and a
  long decorator list

- it's cumbersome to cut and paste a decorator list for reuse, because
  it starts and ends in the middle of a line

Given that the whole point of adding decorator syntax is to move the
decorator from the end ("foo = staticmethod(foo)" after a 100-line
body) to the front, where it is more in-your-face, it should IMO be
moved all the way to the front.

(If I could design a language from scratch, I might want to move
docstrings to before the 'def' as well; I've come to appreciate this
in Java code.  Fredrik Lundh's PythonDoc also uses comments prefixing
the method definition; he also uses @ so he should be happy with this
syntax, as he seemed to be when I mentioned it to him at EuroPython.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jhylton at gmail.com  Fri Aug  6 16:52:15 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Fri Aug  6 16:52:17 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <200408061445.i76EjqT07026@guido.python.org>
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org>
Message-ID: <e8bf7a5304080607524990942d@mail.gmail.com>

On Fri, 06 Aug 2004 07:45:51 -0700, Guido van Rossum <guido@python.org> wrote:
> [Anthony]
> > I'm having trouble finding a single piece from you where you
> > outline the problems you have with the [decorators] after args,
> > before :. Could you possibly post a message to python-dev
> > stating your problems with it? It's the major one I've been
> > unable to find your objections to.
> 
> And nobody else has expressed their objections to it either? :-)

I thought it was obvious that this option was not acceptable.
 
> (If I could design a language from scratch, I might want to move
> docstrings to before the 'def' as well; I've come to appreciate this
> in Java code.  Fredrik Lundh's PythonDoc also uses comments prefixing
> the method definition; he also uses @ so he should be happy with this
> syntax, as he seemed to be when I mentioned it to him at EuroPython.)

It's not at all clear that Fredrik likes it, but he doesn't
participate in Python development any more so we may never know :-(. 
>From his blog:

> ...so someone I've never heard of added a major syntactic feature to Python 2.4,
> using a syntax that Guido doesn't really like. sorry, guys, but I liked Python better
> when the core language was controlled by a single designer with a strong intuition.
> the current design-by-arguing-until-nobody-cares-anymore-committee just don't
> understand what made Python's design so damn good (as the "[new syntax] is no
> worse than [existing operator] meaning [something when used with a given type]"
> comment clearly shows). sigh. good thing nobody can force me to use decorators (or
> update to 2.4, for that matter). time to finish my own interpreter, perhaps. or at least
> start using a fresh one.

Jeremy
From guido at python.org  Fri Aug  6 16:52:39 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 16:52:45 2004
Subject: [Python-Dev] Re: Exceptional inheritance patterns
In-Reply-To: Your message of "Fri, 06 Aug 2004 07:55:52 CDT."
	<16659.32728.411134.439644@montanaro.dyndns.org> 
References: <20040805153443.GB5208@solar.trillke>
	<200408060039.i760dht9000761@cosc353.cosc.canterbury.ac.nz>
	<20040806074509.GD5208@solar.trillke>
	<2macx87es5.fsf@starship.python.net> 
	<16659.32728.411134.439644@montanaro.dyndns.org> 
Message-ID: <200408061452.i76Eqd207077@guido.python.org>

>     Michael> I would like an exception class that almost exceptions except
>     Michael> KeyboardInterrupt, SystemExit and -- maybe -- RuntimeError and
>     Michael> MemoryError inherited from.
> 
>     Michael> except ExceptionsButThoseNastyOnesIDontWantToCatch:
>     Michael>     pass
> 
>     Michael> ? <wink>
> 
> I proposed a change to the exceptions hierarchy a few years ago that would
> allow this.  It was obviously never implemented, but I no longer remember
> why.
> 
> Skip

Probably inertia and compatibility issues (2.2 must've been brand new
when you proposed that).  I've become swayed by Paul Prescod's recent
argument that exceptions use subclassing for classification reasons
and that there's nothing wrong with *requiring* them to subclass some
common base class.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Fri Aug  6 16:53:51 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 16:53:55 2004
Subject: [Python-Dev] PEP 318 - writing to func_name attribute
In-Reply-To: Your message of "Fri, 06 Aug 2004 08:18:33 CDT."
	<16659.34089.645670.39263@montanaro.dyndns.org> 
References: <16659.34089.645670.39263@montanaro.dyndns.org> 
Message-ID: <200408061453.i76Erpx07092@guido.python.org>

> One thing I added awhile back to PEP 318 as an open issue is that for ease
> of writing wrappers the func_name attribute of a function object should be
> writable.  For example, in situations where the decorator returns a new
> function object (often, I would think) it would really be nice if that new
> function object had the same name as the undecorated function.  Consider:
> 
>     def a(func):
>         def _inner(*args, **kwds):
>             return func(*args, **kwds)
>         return _inner
> 
>     @a
>     def func(*args, **kwds):
>         print args
>         print kwds
> 
>     print "func's name:", func.func_name
> 
> I realize you can use new.function() to create a new function object, but
> that seems like a fair amount of extra complexity just to change the
> function's name.  I'd prefer it if the decorator could be written as:
> 
>     def a(func):
>         def _inner(*args, **kwds):
>             return func(*args, **kwds)
>         _inner.func_name = func.func_name
>         return _inner
> 
> That fails because func_name is readonly.  Any chance this restriction can
> be loosened up?

Good idea, independent from decorators; please add it to the PEP!

--Guido van Rossum (home page: http://www.python.org/~guido/)
From krunkle at gmail.com  Fri Aug  6 16:54:50 2004
From: krunkle at gmail.com (Kendall Clark)
Date: Fri Aug  6 16:56:25 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz>
References: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz>
Message-ID: <afaf79d5040806075424ec6e07@mail.gmail.com>

On Fri, 06 Aug 2004 14:35:08 +1200, Greg Ewing
<greg@cosc.canterbury.ac.nz> wrote:
> "Phillip J. Eby" <pje@telecommunity.com>:
> 
> > Now, I don't know if my thought process here is at all typical or
> > representative of anything, other than my own idiosyncratic self.
> 
> You're not alone. I agree with *everything* you said there.

+1

Philip's is the best argument I've heard so far. I favor
list-after-def over @ for decorator syntax -- FWIW. List after def
seems very unambiguously some kind of modifier of a function or
method.

Kendall Grant Clark
Managing Editor, XML.com
From krunkle at gmail.com  Fri Aug  6 16:56:23 2004
From: krunkle at gmail.com (Kendall Clark)
Date: Fri Aug  6 16:56:27 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz>
References: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz>
Message-ID: <afaf79d504080607563058e283@mail.gmail.com>

On Fri, 06 Aug 2004 14:35:08 +1200, Greg Ewing
<greg@cosc.canterbury.ac.nz> wrote:
> "Phillip J. Eby" <pje@telecommunity.com>:
> 
> > Now, I don't know if my thought process here is at all typical or
> > representative of anything, other than my own idiosyncratic self.
> 
> You're not alone. I agree with *everything* you said there.

+1

Eby's is the best argument I've heard so far. I favor list-after-def
over @ for decorator syntax -- FWIW. List after def seems very
unambiguously some kind of modifier of a function or method.

Kendall Grant Clark
Managing Editor, XML.com
From guido at python.org  Fri Aug  6 16:56:53 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 16:56:59 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: Your message of "Fri, 06 Aug 2004 14:39:25 BST."
	<2msmb05qsy.fsf@starship.python.net> 
References: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com>  
	<2msmb05qsy.fsf@starship.python.net> 
Message-ID: <200408061456.i76EurX07119@guido.python.org>

> >>I added "with", although I havn't seen it.
> >
> > Guido's reserving "with" for this purpose in some future Python:
> >
> >      with x.y:
> >          .z = spam    # set x.y.z = spam
> >          print .q.r   # print x.y.q.r
> 
> Except that the only extant PEP involving with actually uses it for
> something else :-)

And I wish that PEP would propose a different name.  (In fact, the
fact that 'with' is slated for a different use should be added to it.)

> I think talking about what Guido is or isn't doing is a bit
> ... wrong?

Yes if it's speculation (like what I would consider "pythonic").  In
this case, I have repeatedly stated exactly what is quoted above as my
preferred use for 'with' in Python 3.0.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From krunkle at gmail.com  Fri Aug  6 16:54:50 2004
From: krunkle at gmail.com (Kendall Clark)
Date: Fri Aug  6 16:57:34 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz>
References: <200408060235.i762Z8cF000939@cosc353.cosc.canterbury.ac.nz>
Message-ID: <afaf79d5040806075424ec6e07@mail.gmail.com>

On Fri, 06 Aug 2004 14:35:08 +1200, Greg Ewing
<greg@cosc.canterbury.ac.nz> wrote:
> "Phillip J. Eby" <pje@telecommunity.com>:
> 
> > Now, I don't know if my thought process here is at all typical or
> > representative of anything, other than my own idiosyncratic self.
> 
> You're not alone. I agree with *everything* you said there.

+1

Philip's is the best argument I've heard so far. I favor
list-after-def over @ for decorator syntax -- FWIW. List after def
seems very unambiguously some kind of modifier of a function or
method.

Kendall Grant Clark
Managing Editor, XML.com
From skip at pobox.com  Fri Aug  6 16:56:08 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 16:58:37 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <E1Bt4zi-0003Sr-00@swing.co.at>
References: <16659.32118.414524.314310@montanaro.dyndns.org>
	<E1Bt4zi-0003Sr-00@swing.co.at>
Message-ID: <16659.39944.597362.102214@montanaro.dyndns.org>


    >> It would be easy to find if you search for "def" or "function"...
    >> This idea has been raised before and is a red herring in my opinion.

    Christian> I know about looking at the documentation of `def`. But you
    Christian> are assuming here that the puzzled reader associates the
    Christian> (leading) list markup with the function definition.

Ah, my apologies.  I had list-after-def decorators in mind when I replied.

    Christian> The trailing list markup (that I personally favored) hasn't
    Christian> been in contention since May? June?

Until a couple days ago I don't think anything regarding PEP 318 had been
discussed since June on c.l.py or python-dev.

Skip
From pje at telecommunity.com  Fri Aug  6 17:06:20 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug  6 17:02:12 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <43p6h010jedicol36tbvf2t87c1pniqpbc@4ax.com>
References: <20040805232522.A1B8E1E400A@bag.python.org>
Message-ID: <5.1.1.6.0.20040806110536.03583e40@mail.telecommunity.com>

At 12:12 PM 8/6/04 +0100, Richie Hindle wrote:

>Perhaps someone with write access to the Wiki page at
>http://www.python.org/moin/PythonDecorators could add this idea there?

FYI, all you need to do to get write access is "log in" by creating an 
account - Click on UserPreferences.

From mwh at python.net  Fri Aug  6 17:04:19 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 17:04:20 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <200408061456.i76EurX07119@guido.python.org> (Guido van
	Rossum's message of "Fri, 06 Aug 2004 07:56:53 -0700")
References: <5.1.1.6.0.20040805154144.02aa96c0@mail.telecommunity.com>
	<2msmb05qsy.fsf@starship.python.net>
	<200408061456.i76EurX07119@guido.python.org>
Message-ID: <2m7jsc5mvg.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> >>I added "with", although I havn't seen it.
>> >
>> > Guido's reserving "with" for this purpose in some future Python:
>> >
>> >      with x.y:
>> >          .z = spam    # set x.y.z = spam
>> >          print .q.r   # print x.y.q.r
>> 
>> Except that the only extant PEP involving with actually uses it for
>> something else :-)
>
> And I wish that PEP would propose a different name.  (In fact, the
> fact that 'with' is slated for a different use should be added to it.)

Noted.  I'll do something about it eventually...

>> I think talking about what Guido is or isn't doing is a bit
>> ... wrong?
>
> Yes if it's speculation (like what I would consider "pythonic").  In
> this case, I have repeatedly stated exactly what is quoted above as my
> preferred use for 'with' in Python 3.0.

Somehow I'd missed that.

Cheers,
mwh

-- 
  Its unmanageable complexity has spawned more fear-preventing tools
  than any other language, but the solution _should_ have been to 
  create and use a language that does not overload the whole goddamn 
  human brain with irrelevant details.  -- Erik Naggum, comp.lang.lisp
From pje at telecommunity.com  Fri Aug  6 17:11:36 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug  6 17:07:29 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <41136677.9090103@egenix.com>
References: <200408052324.i75NOxA05160@guido.python.org>
	<004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de>
	<000f01c47b39$de3bcd70$6700a8c0@computer>
	<4112B6DC.4080605@v.loewis.de>
	<000f01c47b40$cde05110$6700a8c0@computer>
	<4112BF9E.40003@v.loewis.de>
	<200408052324.i75NOxA05160@guido.python.org>
Message-ID: <5.1.1.6.0.20040806110706.03586ec0@mail.telecommunity.com>

At 01:07 PM 8/6/04 +0200, M.-A. Lemburg wrote:

>1. Instead of trying to overgeneralize, we only allow exactly
>    *one* decorator per function definition.

Not practical.  Since many use cases for multiple decorators exist, 
restricting the syntax to one simply pushes the issue into functions to 
combine decorators, further decreasing readability.


>2. Since decorators change the definition of a function, that
>    one decorator call is placed directly after the "def" and
>    before the function name.

This makes it difficult to e.g. grep for 'def functionname', and will 
confuse the heck out of simplistic code scanners, not to mention Tim 
Peters.  :)


From guido at python.org  Fri Aug  6 17:07:48 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 17:07:52 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: Your message of "Fri, 06 Aug 2004 10:52:15 EDT."
	<e8bf7a5304080607524990942d@mail.gmail.com> 
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org> 
	<e8bf7a5304080607524990942d@mail.gmail.com> 
Message-ID: <200408061507.i76F7mI07201@guido.python.org>

> It's not at all clear that Fredrik likes it, but he doesn't
> participate in Python development any more so we may never know :-(. 
> From his blog:
> 
> > ...so someone I've never heard of added a major syntactic feature
> > to Python 2.4, using a syntax that Guido doesn't really
> > like. sorry, guys, but I liked Python better when the core
> > language was controlled by a single designer with a strong
> > intuition.  the current
> > design-by-arguing-until-nobody-cares-anymore-committee just don't
> > understand what made Python's design so damn good (as the "[new
> > syntax] is no worse than [existing operator] meaning [something
> > when used with a given type]" comment clearly shows). sigh. good
> > thing nobody can force me to use decorators (or update to 2.4, for
> > that matter). time to finish my own interpreter, perhaps. or at
> > least start using a fresh one.
> 
> Jeremy

I saw that blog entry, and was surprised, because he really did seem
to like it back then.  It's clear that Fredrik (a) forgot that I
showed it to him, and (b) misunderstood (or worse, deliberately
misconstrued) how it came checked in.  I'm disappointed.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From edreamleo at charter.net  Fri Aug  6 17:08:04 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 17:08:14 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org>
Message-ID: <017601c47bc7$2c84b720$6700a8c0@computer>

[quote]
Problems with this form:

- it hides crucial information (e.g. that it is a static method)
  after the signature, where it is easily missed

- it's easy to miss the transition between a long argument list and a
  long decorator list

- it's cumbersome to cut and paste a decorator list for reuse, because
  it starts and ends in the middle of a line

Given that the whole point of adding decorator syntax is to move the
decorator from the end ("foo = staticmethod(foo)" after a 100-line
body) to the front, where it is more in-your-face, it should IMO be
moved all the way to the front.
[end quote]

Ah Jeeze :-)  Are you really, truly saying that you want to introduce @ to
help people format their declarations?  This just seems like complete
nonsense to me.

How about:

class C(object):

     def longMethodNameForEffect(
        longArgumentOne=None,
        longArgumentTwo=42)\
     [
         staticmethod,
         funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
              status="experimental", author="BDFL")
     ]:
    SNIP

or even:

class C(object):

     def longMethodNameForEffect(
        longArgumentOne=None,
        longArgumentTwo=42)\
     \
     # ***** DECORATION *****
     [
         staticmethod,
         funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
              status="experimental", author="BDFL")
     ]:
    SNIP

Isn't "a total jumble of stuff ending with a smiley." a complete red
herring?  Look, there is a lot of information that has to be presented,
whatever the means.  How is _any_ syntax going to simplify:

funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
    status="experimental", author="BDFL")

??

And are you seriously saying that problems cutting and pasting should drive
this discussion?

real-python-programmers-don't-need-help-formatting-ly yours,

Edward

P.S. In Leo one would render this:

class C(object):

     def longMethodNameForEffect(
        longArgumentOne=None,
        longArgumentTwo=42) [
        << decoration for longMethodNameForEffect >>
     ]:
     SNIP

but I suppose that's not relevant :-)  BTW, << decoration for
longMethodNameForEffect >> would be syntax colored in red by default (with
blue << and >>) so it would stand out.

EKR
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From bkc at murkworks.com  Fri Aug  6 17:22:33 2004
From: bkc at murkworks.com (Brad Clements)
Date: Fri Aug  6 17:16:04 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <200408061445.i76EjqT07026@guido.python.org>
References: Your message of "Fri,
	06 Aug 2004 20:58:27 +1000."	<41136453.1040908@interlink.com.au> 
Message-ID: <411369F9.4580.DDF2B6F1@coal.murkworks.com>

On 6 Aug 2004 at 7:45, Guido van Rossum wrote:

> Just look at it when either the list of decorators or the argument
> list doesn't fit on one line:

Could the decorator list be treated specially, like a docstring?

class C(object):

    def longMethodNameForEffect(longArgumentOne=None,
                                longArgumentTwo=42):
        [
            staticmethod,
            funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
                      status="experimental", author="BDFL")
        ]
        """This method blah, blah.

        It supports the following arguments:
        - longArgumentOne -- a string giving ...
        - longArgumentTwo -- a number giving ...
        """


compare to:

class C(object):

    @staticmethod
    @funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
                      status="experimental", author="BDFL")
    def longMethodNameForEffect(longArgumentOne=None,
                                longArgumentTwo=42):
        """This method blah, blah.

        It supports the following arguments:
        - longArgumentOne -- a string giving ...
        - longArgumentTwo -- a number giving ...
        """



-- 
Brad Clements,                bkc@murkworks.com   (315)268-1000
http://www.murkworks.com                          (315)268-9812 Fax
http://www.wecanstopspam.org/                   AOL-IM: BKClements

From pje at telecommunity.com  Fri Aug  6 17:21:45 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug  6 17:17:37 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <1091784135.1770.6.camel@localhost>
References: <59e9fd3a04080519462bf57899@mail.gmail.com>
	<59e9fd3a04080519462bf57899@mail.gmail.com>
Message-ID: <5.1.1.6.0.20040806112037.035812e0@mail.telecommunity.com>

At 10:22 AM 8/6/04 +0100, Mark Russell wrote:
>On Fri, 2004-08-06 at 03:46, Andrew Durdin wrote:
> > 8. When using multiple decorators, each appears on a separate line.
>
>No, you can put multiple decorators on a single line -- you just need a
>newline before the def.  From test_decorators.py:
>
>         class C(object):
>             @funcattrs(abc=1) @staticmethod
>             def foo(): return 42
>
>Probably not good style (but neither is "if cond: stmt" I guess).

*shudder*

Can we change this?  The positive benefits that the @ syntax *does* have, 
seem completely  lost by doing this.

From mwh at python.net  Fri Aug  6 17:23:12 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 17:23:13 2004
Subject: [Python-Dev] PEP 318 - writing to func_name attribute
In-Reply-To: <200408061453.i76Erpx07092@guido.python.org> (Guido van
	Rossum's message of "Fri, 06 Aug 2004 07:53:51 -0700")
References: <16659.34089.645670.39263@montanaro.dyndns.org>
	<200408061453.i76Erpx07092@guido.python.org>
Message-ID: <2m1xik5lzz.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> That fails because func_name is readonly.  Any chance this restriction can
>> be loosened up?
>
> Good idea, independent from decorators; please add it to the PEP!

Quick question: would it be better to allow any object to be assigned
to func_name and fix the (two, for core C code anway) places this
matters or only allow strings to be assigned to func_name?

I'm thinking probably the latter.

Cheers,
mwh

-- 
  Not only does the English Language borrow words from other
  languages, it sometimes chases them down dark alleys, hits
  them over the head, and goes through their pockets.   -- Eddy Peters
From mwh at python.net  Fri Aug  6 17:28:26 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 17:28:28 2004
Subject: [Python-Dev] status of statistics.py?
Message-ID: <2mwu0c476t.fsf@starship.python.net>

I see statistics.py is still in nondist/sandbox/.  Is there any deep
reason for this?  I see the file ends with

XXX = """
    Other possible functions include data groupers for
    binning, counting, and splitting into equivalence classes.
"""

but that hardly seems a blocker.  I'd like to see the module in 2.4.

Cheers,
mwh

-- 
  Important data should not be entrusted to Pinstripe, as it may
  eat it and make loud belching noises.
   -- from the announcement of the beta of "Pinstripe" aka. Redhat 7.0
From mal at egenix.com  Fri Aug  6 17:35:02 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Aug  6 17:35:09 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <5.1.1.6.0.20040806110706.03586ec0@mail.telecommunity.com>
References: <200408052324.i75NOxA05160@guido.python.org>
	<004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de>
	<000f01c47b39$de3bcd70$6700a8c0@computer>
	<4112B6DC.4080605@v.loewis.de>
	<000f01c47b40$cde05110$6700a8c0@computer>
	<4112BF9E.40003@v.loewis.de>
	<200408052324.i75NOxA05160@guido.python.org>
	<5.1.1.6.0.20040806110706.03586ec0@mail.telecommunity.com>
Message-ID: <4113A526.3090100@egenix.com>

Phillip J. Eby wrote:
> At 01:07 PM 8/6/04 +0200, M.-A. Lemburg wrote:
> 
>> 1. Instead of trying to overgeneralize, we only allow exactly
>>    *one* decorator per function definition.
> 
> 
> Not practical.  Since many use cases for multiple decorators exist, 
> restricting the syntax to one simply pushes the issue into functions to 
> combine decorators, further decreasing readability.
> 
> 
>> 2. Since decorators change the definition of a function, that
>>    one decorator call is placed directly after the "def" and
>>    before the function name.
> 
> 
> This makes it difficult to e.g. grep for 'def functionname', and will 
> confuse the heck out of simplistic code scanners, not to mention Tim 
> Peters.  :)

Please read my entire post. Here's the grep pattern to
use: "def( \w+)? funcname".

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 06 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From guido at python.org  Fri Aug  6 17:37:57 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 17:38:02 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: Your message of "Fri, 06 Aug 2004 10:08:04 CDT."
	<017601c47bc7$2c84b720$6700a8c0@computer> 
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org> 
	<017601c47bc7$2c84b720$6700a8c0@computer> 
Message-ID: <200408061537.i76Fbv207317@guido.python.org>

> Ah Jeeze :-)  Are you really, truly saying that you want to introduce @ to
> help people format their declarations?  This just seems like complete
> nonsense to me.

Insulting words don't suit you, Edward.

Yes, this whole thing is deeply about readability.  It *started* with
the observation that the post-function-body decorator "syntax" had
usability / readability issues.  A lot of Python is about that.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From s.percivall at chello.se  Fri Aug  6 17:38:21 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Fri Aug  6 17:38:23 2004
Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and Classes
Message-ID: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>

On pep 318 (like everything else here).

Is it actually supposed to work for classes? like the
pep title says.

It doesn't seem to work, instead it issues a SyntaxError
on the class keyword. Also, the discussion is completely
focused on decorating functions and methods.

//Simon

From barry at python.org  Fri Aug  6 17:39:34 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug  6 17:39:25 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <ceuk0e$rut$1@sea.gmane.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain> <ceuk0e$rut$1@sea.gmane.org>
Message-ID: <1091806774.8414.63.camel@localhost>

On Thu, 2004-08-05 at 20:39, Fernando Perez wrote:

> If @ ends up being accepted, ipython can obviously adapt.  I'll replace it with
> alternative syntax, be it @@, %, or some other kind of special-casing trick.
> 
> It would not make me happy, and it worries me that ipython explicitly uses (for
> good reasons, I like to think), the three characters explicitly avoided by
> python:

So I spent a little time futzing with python-mode and some of my
decorator code, to try some alternative leading characters.  Of the ones
that cannot be used in valid Python code today (i.e. no backward
compatibility issues), I tried ':', '/', '=', '*', and '|'.

: wasn't bad, and it had some nice symmetry with the function definition
that follows.  But those two little dots can be hard to read in some
fonts and I'm worried that because colons are so special in Python, it
might give existing tools some problems (python-mode had no problem with
it though).

/ and | also weren't bad, although bar is probably preferred. 
Interestingly enough, in my font-lock settings, I italicize the
decorator line, so these two characters looked pretty similar.  Someone
else pointing out that the bar kind of looks like it's anchoring the
decorator to the function, and I got a similar sense of that.

* was interesting because it evokes a feeling of a bulleted list, but
ultimately it looked too jarring.

= wasn't bad at all.  It also had a bullet list feel to it, but without
the discomfort of using stars.  This was my favorite of the
alternatives.

In summary, I can definitely sympathize with the concern of tool authors
who have settled on @ as a meta-character that Python will not use.  I
wonder how the Leo or ipython folks would feel if the only change to the
current implementation of PEP 318 was to change the decorator
introducing character.  If we can increase the community's consensus by
making this little change, I wouldn't oppose it.

My preference then would be to use = with | and : as alternatives, in
that order.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/014f8527/attachment.pgp
From guido at python.org  Fri Aug  6 17:39:55 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 17:40:00 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: Your message of "Fri, 06 Aug 2004 11:21:45 EDT."
	<5.1.1.6.0.20040806112037.035812e0@mail.telecommunity.com> 
References: <59e9fd3a04080519462bf57899@mail.gmail.com>
	<59e9fd3a04080519462bf57899@mail.gmail.com> 
	<5.1.1.6.0.20040806112037.035812e0@mail.telecommunity.com> 
Message-ID: <200408061539.i76FdtG07333@guido.python.org>

> >No, you can put multiple decorators on a single line -- you just need a
> >newline before the def.  From test_decorators.py:
> >
> >         class C(object):
> >             @funcattrs(abc=1) @staticmethod
> >             def foo(): return 42
> >
> >Probably not good style (but neither is "if cond: stmt" I guess).
> 
> *shudder*
> 
> Can we change this?  The positive benefits that the @ syntax *does*
> have, seem completely lost by doing this.

That's a bit strong, but yes, it would be trivial to change.  If
that's all that's needed to save @deco, it's a lot easier than ripping
the whole thing out... :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Fri Aug  6 17:40:31 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 17:40:36 2004
Subject: [Python-Dev] PEP 318 - writing to func_name attribute
In-Reply-To: Your message of "Fri, 06 Aug 2004 16:23:12 BST."
	<2m1xik5lzz.fsf@starship.python.net> 
References: <16659.34089.645670.39263@montanaro.dyndns.org>
	<200408061453.i76Erpx07092@guido.python.org> 
	<2m1xik5lzz.fsf@starship.python.net> 
Message-ID: <200408061540.i76FeVd07348@guido.python.org>

> Quick question: would it be better to allow any object to be assigned
> to func_name and fix the (two, for core C code anway) places this
> matters or only allow strings to be assigned to func_name?
> 
> I'm thinking probably the latter.

Let's stick to strings.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From barry at python.org  Fri Aug  6 17:43:16 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug  6 17:43:07 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <4112DD32.5080302@interlink.com.au>
References: <200408051636.i75Gaak03654@guido.python.org>
	<4112DD32.5080302@interlink.com.au>
Message-ID: <1091806996.8413.66.camel@localhost>

On Thu, 2004-08-05 at 21:21, Anthony Baxter wrote:
> My take on the @decorator syntax is that after initially
> hating it, I've found it growing on me more and more.
> It's extremely clear and very readable. Most importantly,
> in my eyes, it's obviously grouped with the def that
> follows it. From being a -0 (when Guido first mentioned
> it) to a +0 (at the time of checkin) I'm now +1 on this
> form of decorator.

I'm also +1 on using the format/position of the @decorators.  It's
vastly superior to list-before-colon or list-before-def.  See my other
post for alternative possible solutions to Leo/ipython problem.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/0c5906b4/attachment-0001.pgp
From s.percivall at chello.se  Fri Aug  6 17:43:13 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Fri Aug  6 17:43:16 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
Message-ID: <53CF6F70-E7BF-11D8-8E6E-0003934AD54A@chello.se>

Sorry. I actually read some of the pep this time ... saw the Open 
Issues part.

Still, is there any thinking about this? if I may ask.

On 2004-08-06, at 17.38, Simon Percivall wrote:

> On pep 318 (like everything else here).
>
> Is it actually supposed to work for classes? like the
> pep title says.
>
> It doesn't seem to work, instead it issues a SyntaxError
> on the class keyword. Also, the discussion is completely
> focused on decorating functions and methods.
>
> //Simon

From skip at pobox.com  Fri Aug  6 17:48:16 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 17:48:46 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
Message-ID: <16659.43072.703540.287723@montanaro.dyndns.org>


    Simon> On pep 318 (like everything else here).
    Simon> Is it actually supposed to work for classes? 

That was the original proposal, way back when.  The motivation for
decorating classes seems much weaker than for functions.  The current
implementation doesn't decorate classes.  The PEP title should probably be
changed.

Skip
From barry at python.org  Fri Aug  6 17:50:40 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug  6 17:50:35 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <4112DD1C.3060902@interlink.com.au>
References: <4112DD1C.3060902@interlink.com.au>
Message-ID: <1091807440.8406.69.camel@localhost>

On Thu, 2004-08-05 at 21:21, Anthony Baxter wrote:
> There's also the issue that there's a bunch
> of other existing PEPs describing features that aren't
> up to date at all.

Heh, yeah, let's rip out rich comparisons, since although the feature's
been in Python since 2.1, the PEP was definitely out of date.  'Course,
I just fixed it so go ahead and put them back in <wink>.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/4b9a79ee/attachment.pgp
From guido at python.org  Fri Aug  6 17:50:44 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 17:50:49 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: Your message of "Fri, 06 Aug 2004 17:35:02 +0200."
	<4113A526.3090100@egenix.com> 
References: <200408052324.i75NOxA05160@guido.python.org>
	<004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de>
	<000f01c47b39$de3bcd70$6700a8c0@computer>
	<4112B6DC.4080605@v.loewis.de>
	<000f01c47b40$cde05110$6700a8c0@computer>
	<4112BF9E.40003@v.loewis.de>
	<200408052324.i75NOxA05160@guido.python.org>
	<5.1.1.6.0.20040806110706.03586ec0@mail.telecommunity.com> 
	<4113A526.3090100@egenix.com> 
Message-ID: <200408061550.i76Foig07409@guido.python.org>

> Please read my entire post. Here's the grep pattern to
> use: "def( \w+)? funcname".

That doesn't work with grep on my box.

'nuff said.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From jim at zope.com  Fri Aug  6 17:54:53 2004
From: jim at zope.com (Jim Fulton)
Date: Fri Aug  6 17:54:57 2004
Subject: [Python-Dev] I'd like to add an __iter__ method to
	unittest.TestSuite
Message-ID: <4113A9CD.6030904@zope.com>


TestSuite objects provide no public mechanism for iteration.
We have a need to be able to iterate over them.  Up to now,
we've been using the _tests attribute, which is bad.

If no one objects, I'll add an __iter__ method.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From guido at python.org  Fri Aug  6 17:55:31 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 17:55:37 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: Your message of "Fri, 06 Aug 2004 17:38:21 +0200."
	<A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se> 
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se> 
Message-ID: <200408061555.i76FtV707448@guido.python.org>

> On pep 318 (like everything else here).
> 
> Is it actually supposed to work for classes? like the
> pep title says.

Like so many things in the PEP, it's out of date.

> It doesn't seem to work, instead it issues a SyntaxError
> on the class keyword. Also, the discussion is completely
> focused on decorating functions and methods.

Some people have argued that it should work for classes (Java and C#
decorators work for any declaration).  I've never been convinced,
although I'm not dead set against it either.

It would seem that the functionality isn't quite as useful there; you
can get most of the same effects with metaclasses.  I'm not aware of
any idiom that takes a class and passes it through some kind of
wrapper or transformation *after* the class declaration; if somebody
is using this, it surely must be a very rare use indeed.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Fri Aug  6 17:57:16 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 17:57:23 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: Your message of "Fri, 06 Aug 2004 11:39:34 EDT."
	<1091806774.8414.63.camel@localhost> 
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost> 
Message-ID: <200408061557.i76FvGo07481@guido.python.org>

> So I spent a little time futzing with python-mode and some of my
> decorator code, to try some alternative leading characters.  Of the ones
> that cannot be used in valid Python code today (i.e. no backward
> compatibility issues), I tried ':', '/', '=', '*', and '|'.

If the community can rally behind one of these, I think that would be
acceptable.  They all seem arbitrary, but so is the choice of '@'. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From mal at egenix.com  Fri Aug  6 17:58:19 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Aug  6 17:58:21 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <200408061550.i76Foig07409@guido.python.org>
References: <200408052324.i75NOxA05160@guido.python.org>	<004a01c47b2b$65d19260$6700a8c0@computer>
	<4112A4B9.4010907@v.loewis.de>	<000f01c47b39$de3bcd70$6700a8c0@computer>
	<4112B6DC.4080605@v.loewis.de>	<000f01c47b40$cde05110$6700a8c0@computer>
	<4112BF9E.40003@v.loewis.de>	<200408052324.i75NOxA05160@guido.python.org>	<5.1.1.6.0.20040806110706.03586ec0@mail.telecommunity.com>
	<4113A526.3090100@egenix.com>
	<200408061550.i76Foig07409@guido.python.org>
Message-ID: <4113AA9B.5020608@egenix.com>

Guido van Rossum wrote:
>>Please read my entire post. Here's the grep pattern to
>>use: "def( \w+)? funcname".
> 
> That doesn't work with grep on my box.

Dev-Python/Lib> egrep "def( \w+)? __init__" *.py
Bastion.py:    def __init__(self, get, name):
Bastion.py:        def __init__(self):
ConfigParser.py:    def __init__(self, msg=''):
ConfigParser.py:    def __init__(self, section):
...
test-decorate.py:def staticmethod test(self):

> 'nuff said.

Indeed.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 06 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From edreamleo at charter.net  Fri Aug  6 18:02:26 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Fri Aug  6 18:02:33 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org>
	<017601c47bc7$2c84b720$6700a8c0@computer>
	<200408061537.i76Fbv207317@guido.python.org>
Message-ID: <002901c47bce$c51edc70$6700a8c0@computer>

> Insulting words don't suit you, Edward.

I apologize for the insult.  None was intended.

> Yes, this whole thing is deeply about readability.

OK.  I've said everything I can say on this topic, except this:

I would like to commend you personally Guido, and everyone else on this list
for their willingness to reopen what is no doubt was a painful discussion.
Leo and I can live with whatever you decide.  I hope you can convince most
Python users that the choice is a good one.  Bye for now.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From anthony at interlink.com.au  Fri Aug  6 18:03:32 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 18:04:09 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: <16659.43072.703540.287723@montanaro.dyndns.org>
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<16659.43072.703540.287723@montanaro.dyndns.org>
Message-ID: <4113ABD4.9010809@interlink.com.au>

Skip Montanaro wrote:
>     Simon> On pep 318 (like everything else here).
>     Simon> Is it actually supposed to work for classes? 
> 
> That was the original proposal, way back when.  The motivation for
> decorating classes seems much weaker than for functions.  The current
> implementation doesn't decorate classes.  The PEP title should probably be
> changed.

And this is now done. There are probably still references to classes in
inappropriate places in the doco. I will spend more time on it this
weekend.

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From tim.peters at gmail.com  Fri Aug  6 18:06:49 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug  6 18:06:53 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <1091806774.8414.63.camel@localhost>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain> <ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>
Message-ID: <1f7befae040806090644be9c04@mail.gmail.com>

[Barry Warsaw]
> So I spent a little time futzing with python-mode and some of my
> decorator code, to try some alternative leading characters.  Of the ones
> that cannot be used in valid Python code today (i.e. no backward
> compatibility issues), I tried ':', '/', '=', '*', and '|'.
> 
> : wasn't bad, and it had some nice symmetry with the function definition
> that follows.  But those two little dots can be hard to read in some
> fonts ...

Indeed!  I read that paragraph four times before I realized it started
with a colon -- a fleck of dirt on the screen obscured it completely.

> / and | also weren't bad, although bar is probably preferred.

Ya, but then we preclude adding the unary-pipe operator.  This is very
important for numeric types that need to implement half the absolute
value:

>>> |3
1.5
>>> |-3
1.5

...
> Someone else pointing out that the bar kind of looks like it's anchoring the
> decorator to the function, and I got a similar sense of that.

OK, seriously, me too.

> ...
> In summary, I can definitely sympathize with the concern of tool authors
> who have settled on @ as a meta-character that Python will not use.

Yup.

> I wonder how the Leo or ipython folks would feel if the only change to the
> current implementation of PEP 318 was to change the decorator
> introducing character.  If we can increase the community's consensus by
> making this little change, I wouldn't oppose it.

Me neither.

> My preference then would be to use = with | and : as alternatives, in
> that order.

Same here, except -1 on the colon.  Although = would preclude
introducing the unary-equals operator, which is very important for
numeric types that need to implement the absolute value rotated by 90
degrees.
From mwh at python.net  Fri Aug  6 18:07:14 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug  6 18:07:17 2004
Subject: [Python-Dev] PEP 318 - writing to func_name attribute
In-Reply-To: <200408061540.i76FeVd07348@guido.python.org> (Guido van
	Rossum's message of "Fri, 06 Aug 2004 08:40:31 -0700")
References: <16659.34089.645670.39263@montanaro.dyndns.org>
	<200408061453.i76Erpx07092@guido.python.org>
	<2m1xik5lzz.fsf@starship.python.net>
	<200408061540.i76FeVd07348@guido.python.org>
Message-ID: <2mpt6445e5.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> Quick question: would it be better to allow any object to be assigned
>> to func_name and fix the (two, for core C code anway) places this
>> matters or only allow strings to be assigned to func_name?
>> 
>> I'm thinking probably the latter.
>
> Let's stick to strings.

Cool.  http://www.python.org/sf/1004703 awaits review.

Cheers,
mwh

-- 
  ... Windows proponents tell you that it will solve things that
  your Unix system people keep telling you are hard.  The Unix 
  people are right: they are hard, and Windows does not solve 
  them, ...                            -- Tim Bradshaw, comp.lang.lisp
From anthony at interlink.com.au  Fri Aug  6 18:09:08 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 18:11:15 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <16659.34238.443364.229226@montanaro.dyndns.org>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<1091740792.8546.133.camel@localhost>	<4112A746.6010501@v.loewis.de>
	<2m657w7b1k.fsf@starship.python.net>	<1091792604.1770.8.camel@localhost>
	<16659.34238.443364.229226@montanaro.dyndns.org>
Message-ID: <4113AD24.9050205@interlink.com.au>

Skip Montanaro wrote:
>     Mark> I think it is just the PEP that needs updating.
> 
> Not to pick on Mark, but everyone seems to think the PEP needs updating but
> nobody's stepping up to offer patches.  Anthony and I are both about out of
> round tuits...

What Skip said. For a topic generating this much heat, volunteers have
been thin on the ground.

In case it's not clear - there is an _enormous_ amount of work that
could be put into the PEP. Collecting posts from the 2+ years that
this topic was active on python-dev is a mammoth task. It's also
something that parallelises well. Please, if you care about this (and
it's obvious from the volume of posts that lots of people do), spend
a bit of time adding entries to the wiki. References to python-dev
posts from the archives, in particular, are _very_ nice.

I have a crapload of other things I'd much prefer to be working on,
and help would be _much_ appreciated.

Anthony
From anthony at interlink.com.au  Fri Aug  6 18:11:07 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 18:11:20 2004
Subject: [Python-Dev] PEP 318, and the PEP process
In-Reply-To: <1091807440.8406.69.camel@localhost>
References: <4112DD1C.3060902@interlink.com.au>
	<1091807440.8406.69.camel@localhost>
Message-ID: <4113AD9B.3090003@interlink.com.au>

Barry Warsaw wrote:
> 
> Heh, yeah, let's rip out rich comparisons, since although the feature's
> been in Python since 2.1, the PEP was definitely out of date.  'Course,
> I just fixed it so go ahead and put them back in <wink>.

Maybe rich comparisions should be prefixed with the $ sign, or the
unicode glyph for euros, or whatever the local currency symbol is.

i18n-ising-the-grammar-is-the-next-big-thing,
Anthony


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From anthony at interlink.com.au  Fri Aug  6 18:13:57 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 18:14:08 2004
Subject: [Python-Dev] pie-thon redux
Message-ID: <4113AE45.5080004@interlink.com.au>

As a break from the decorator thread, NTK (www.ntk.net)
had this writeup of the Pie-Thon. It amused me.

                                >> HARD NEWS <<
                                Perl hullabaloos


          Last year, DAN SUGALSKI, lead developer of the forthcoming
          Perl6 virtual machine, Parrot, bet the Python developers
          that run Python faster on the fledgling VM - and the creator
          of Python could throw a pie at him at the next OSCON if he
          didn't. He didn't; the pie-ing was duly arranged. As
          everyone knows, while Perlites are chaotic/good trickster
          archetypes who love such events, Pythonistas are peaceful,
          have-their-glasses-on-a-little-string types, like hobbits or
          the Dutch. In the end, Guido van Rossum refused to throw the
          pie, and instead offered to share it as food with the Perl
          developers. Nothing, of course, could have been more
          guaranteed to throw Perlsters into violent rage. An extended
          period of acrimonious bargaining followed, in which the Perl
          crew grew more and more insistent that their own chief
          developer be humiliated, with many walking out of the
          session, muttering about "all foreplay and no sex". Later,
          the Perl faction took it upon themselves to pie Sugalski -
          much, we are sure, to the shock of the pacifistic Pythonese,
          who may well have planned that using their psychomathematics
          and indented whitespace necromancy all along. (Mind
          you, that didn't stop Guido finally joining in. Feel the
          punctuation rising in you, Guido!)
          http://www.sidhe.org/~dan/blog/archives/000372.html
                                   - Dan gets last laugh later in August



-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From tim.peters at gmail.com  Fri Aug  6 18:16:17 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug  6 18:16:20 2004
Subject: [Python-Dev] I'd like to add an __iter__ method to
	unittest.TestSuite
In-Reply-To: <4113A9CD.6030904@zope.com>
References: <4113A9CD.6030904@zope.com>
Message-ID: <1f7befae04080609165c13c748@mail.gmail.com>

[Jim Fulton]
> TestSuite objects provide no public mechanism for iteration.
> We have a need to be able to iterate over them.  Up to now,
> we've been using the _tests attribute, which is bad.
>
> If no one objects, I'll add an __iter__ method.

+1.  The iteration protocol is endlessly useful, and this a natural use for it.
From anthony at interlink.com.au  Fri Aug  6 18:18:15 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 18:18:29 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <1f7befae040806090644be9c04@mail.gmail.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
Message-ID: <4113AF47.1010405@interlink.com.au>

Tim Peters wrote:
> [Barry Warsaw]
>>...
>>In summary, I can definitely sympathize with the concern of tool authors
>>who have settled on @ as a meta-character that Python will not use.
> 
> Yup.

Having said that, it's quite possible that in the future some other
feature will use the @ symbol - I think arrays have been mentioned
in the same sense, although presumably not in the sense of @variable
meaning "it's an array".

So, we either:
  - explicitly list a symbol that Shall Not Be Used (I favour the
unicode glyph for biohazard), or
  - change the language ref that talks about this to state that although
Python currently doesn't use these symbols, this is not stopping a
future version of Python from using them for some new feature.

Anthony


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From josh at janrain.com  Fri Aug  6 18:20:20 2004
From: josh at janrain.com (Josh Hoyt)
Date: Fri Aug  6 18:20:22 2004
Subject: [Python-Dev] Density of pie-decorator syntax
Message-ID: <4113AFC4.5010004@janrain.com>

One problem I have with the pie-decorator syntax is that it is very 
typographically dense. The @ character runs in with the following 
function name. If there are multiple lines of decorators, the problem is 
worsened. For clarity, the decorators will have no whitespace lines 
between them, creating a dense block.

After experimenting, I have found that readability is increased if there 
is one space required after the @. Don't take my word for it:

Current Python 2.4a2 syntax:



@framework_stuff(lots, of, args)
class Quux(object):

    @check_args(int, str)
    @counted
    @staticmethod
    def frobnicate(foo, bar):
        pass



Proposed change requiring a space:



@ framework_stuff(lots of args)
class Quux(object):

    @ check_args(int, str)
    @ counted
    @ staticmethod
    def frobnicate(foo, bar):
        pass

From jimjjewett at yahoo.com  Fri Aug  6 18:31:47 2004
From: jimjjewett at yahoo.com (Jim J Jewett)
Date: Fri Aug  6 18:31:50 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
Message-ID: <20040806163147.82284.qmail@web50710.mail.yahoo.com>

Nicolas Fleury asked:

> Is there ambiguities with the following? 

> def decorator1(decoratorN(foo(arg1, argN))): 
>     pass

Remember that there is already an obscure
possibility for nested argument lists.

def func (arg1, (arg2, arg3)):
    pass



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
From guido at python.org  Fri Aug  6 18:34:14 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 18:34:25 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: Your message of "Sat, 07 Aug 2004 02:18:15 +1000."
	<4113AF47.1010405@interlink.com.au> 
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com> 
	<4113AF47.1010405@interlink.com.au> 
Message-ID: <200408061634.i76GYEZ07623@guido.python.org>

> - change the language ref that talks about this to state that
> although Python currently doesn't use these symbols, this is not
> stopping a future version of Python from using them for some new
> feature.

Does that really have to be stated?  Isn't this just giving in to the
silly lawyers?  A reference manual describes the status quo.  How can
it possibly begin to describe future directions?  (Except when
reserving certain classes of identifiers like C does.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From anthony at interlink.com.au  Fri Aug  6 18:37:13 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 18:37:24 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408061634.i76GYEZ07623@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
	<4113AF47.1010405@interlink.com.au>
	<200408061634.i76GYEZ07623@guido.python.org>
Message-ID: <4113B3B9.6080801@interlink.com.au>

Guido van Rossum wrote:
>>- change the language ref that talks about this to state that
>>although Python currently doesn't use these symbols, this is not
>>stopping a future version of Python from using them for some new
>>feature.
> 
> 
> Does that really have to be stated?  Isn't this just giving in to the
> silly lawyers?  A reference manual describes the status quo.  How can
> it possibly begin to describe future directions?  (Except when
> reserving certain classes of identifiers like C does.)

Beats me. I'm just going from the comments that Edward(?) made that
he thought the current wording meant that they'd never be used.

Anthony


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From michael.walter at gmail.com  Fri Aug  6 18:40:52 2004
From: michael.walter at gmail.com (Michael Walter)
Date: Fri Aug  6 18:40:56 2004
Subject: [Python-Dev] Density of pie-decorator syntax
In-Reply-To: <4113AFC4.5010004@janrain.com>
References: <4113AFC4.5010004@janrain.com>
Message-ID: <877e9a1704080609407d680e45@mail.gmail.com>

Please, no! :)

Cheers,
Michael

On Fri, 06 Aug 2004 09:20:20 -0700, Josh Hoyt <josh@janrain.com> wrote:
> One problem I have with the pie-decorator syntax is that it is very
> typographically dense. The @ character runs in with the following
> function name. If there are multiple lines of decorators, the problem is
> worsened. For clarity, the decorators will have no whitespace lines
> between them, creating a dense block.
> 
> After experimenting, I have found that readability is increased if there
> is one space required after the @. Don't take my word for it:
> 
> Current Python 2.4a2 syntax:
> 
> @framework_stuff(lots, of, args)
> class Quux(object):
> 
>     @check_args(int, str)
>     @counted
>     @staticmethod
>     def frobnicate(foo, bar):
>         pass
> 
> Proposed change requiring a space:
> 
> @ framework_stuff(lots of args)
> class Quux(object):
> 
>     @ check_args(int, str)
>     @ counted
>     @ staticmethod
>     def frobnicate(foo, bar):
>         pass
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com
>
From nidoizo at yahoo.com  Fri Aug  6 18:50:27 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Fri Aug  6 18:50:34 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <200408061537.i76Fbv207317@guido.python.org>
References: <41136453.1040908@interlink.com.au>	<200408061445.i76EjqT07026@guido.python.org>
	<017601c47bc7$2c84b720$6700a8c0@computer>
	<200408061537.i76Fbv207317@guido.python.org>
Message-ID: <cf0csj$htn$1@sea.gmane.org>

Guido van Rossum wrote:
> Yes, this whole thing is deeply about readability.  It *started* with
> the observation that the post-function-body decorator "syntax" had
> usability / readability issues.  A lot of Python is about that.

I agree.  In fact, I've been convinced by both sides that both 
@decorators and list-after-def solutions have major readability 
drawbacks.  For example, this is more readable than both:

decorate classmethod:
     def foo(arg1,arg2):
         ...

But it is less usable when the decorator applies to a single function, 
unless a syntax is allowed to do it without indentation.  And there's a 
new keyword.  But adding a special character to avoid adding a new 
keyword?  Wouldn't a "from __future__ import decorate" until Python 2.X 
or 3 be better?  It seems like a good compromise to me...

Regards,
Nicolas

From guido at python.org  Fri Aug  6 18:55:12 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 18:55:29 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: Your message of "Sat, 07 Aug 2004 02:37:13 +1000."
	<4113B3B9.6080801@interlink.com.au> 
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
	<4113AF47.1010405@interlink.com.au>
	<200408061634.i76GYEZ07623@guido.python.org> 
	<4113B3B9.6080801@interlink.com.au> 
Message-ID: <200408061655.i76GtCJ07709@guido.python.org>

> Beats me. I'm just going from the comments that Edward(?) made that
> he thought the current wording meant that they'd never be used.

He'd have a hard time proving that in court.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Fri Aug  6 18:58:59 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 18:59:05 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: Your message of "Fri, 06 Aug 2004 12:50:27 EDT."
	<cf0csj$htn$1@sea.gmane.org> 
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org>
	<017601c47bc7$2c84b720$6700a8c0@computer>
	<200408061537.i76Fbv207317@guido.python.org> 
	<cf0csj$htn$1@sea.gmane.org> 
Message-ID: <200408061658.i76Gwxf07727@guido.python.org>

> decorate classmethod:
>      def foo(arg1,arg2):
>          ...

Somebody should add to the PEP that solutions requiring an extra
indent level for the function have a drawback.  Having your minimal
indent level be three deep is painful for those using limited-width
windows; also, the inconsistent indentation between methods with and
without decorators would be a readability problem (plus,
adding/removing decorators would require reindenting the entire method
body).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From jlg at python.org  Fri Aug  6 18:52:28 2004
From: jlg at python.org (Johannes Gijsbers)
Date: Fri Aug  6 19:00:37 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
Message-ID: <pan.2004.08.06.16.52.28.690827@python.org>

On Fri, 06 Aug 2004 11:39:34 -0400, Barry Warsaw wrote:

> = wasn't bad at all.  It also had a bullet list feel to it, but without
> the discomfort of using stars.  This was my favorite of the
> alternatives.

'=' has too srong an association with assignment for me, The first thing I
wondered was where the '=' was assigning to. Then I realized that, in the
context of decorators, it has to be the function/method that follows. So
in the following example:

=classmethod
def foo(arg1,arg2):
    ...

'foo' '=' a classmethod. This does broaden the meaning of '=' from just
equality of identity to belonging to a group, but that's mostly okay.
However, the next example really bugs me:

=accepts(int,int)
=returns(float)
def bar(low,high):
    ...

'bar' '=' in no sense an 'accepts' or a 'returns'. It doesn't seem right
to use an assignment operator for decorating.

Keeping Tim's comments about ':' in mind, my personal favorite is '|'.

Johannes

From niemeyer at conectiva.com  Fri Aug  6 19:10:22 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Fri Aug  6 19:08:04 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <200408061658.i76Gwxf07727@guido.python.org>
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org>
	<017601c47bc7$2c84b720$6700a8c0@computer>
	<200408061537.i76Fbv207317@guido.python.org>
	<cf0csj$htn$1@sea.gmane.org>
	<200408061658.i76Gwxf07727@guido.python.org>
Message-ID: <20040806171022.GA3205@burma.localdomain>

> > decorate classmethod:
> >      def foo(arg1,arg2):
> >          ...
> 
> Somebody should add to the PEP that solutions requiring an extra
> indent level for the function have a drawback.  Having your minimal
> indent level be three deep is painful for those using limited-width
> windows; also, the inconsistent indentation between methods with and
> without decorators would be a readability problem (plus,

I still prefer this method over the current implementation, given
the other pros/cons. Anyway, I'm adding these to the Wiki as
negative points.

> adding/removing decorators would require reindenting the entire method
> body).

I belive all of us are used to quickly reidenting Python
code by now. :-)

-- 
Gustavo Niemeyer
http://niemeyer.net
From shane.holloway at ieee.org  Fri Aug  6 19:07:22 2004
From: shane.holloway at ieee.org (Shane Holloway (IEEE))
Date: Fri Aug  6 19:08:19 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: <200408061555.i76FtV707448@guido.python.org>
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
Message-ID: <4113BACA.6040801@ieee.org>

Guido van Rossum wrote:
 >>It doesn't seem to work, instead it issues a SyntaxError
 >>on the class keyword. Also, the discussion is completely
 >>focused on decorating functions and methods.
 >
 >
 > Some people have argued that it should work for classes (Java and C#
 > decorators work for any declaration).  I've never been convinced,
 > although I'm not dead set against it either.
 >
 > It would seem that the functionality isn't quite as useful there; you
 > can get most of the same effects with metaclasses.  I'm not aware of
 > any idiom that takes a class and passes it through some kind of
 > wrapper or transformation *after* the class declaration; if somebody
 > is using this, it surely must be a very rare use indeed.
 >
 > --Guido van Rossum (home page: http://www.python.org/~guido/)

I have done a few things in the realm of Aspect Oriented programming, 
replacing __metaclass__ with a callable that delegates the "hard work" 
to type() and then plays with the resulting class rather like 
classmethod does.  I remember trying the replace semantics after the end 
of the class, but preferred the in-the-face-"pay-attention" of using the 
metaclass hook.  I think that class pie-decorators would serve the same 
purpose.

I could see the protocols "implements" functionality expressed through 
class decorators, although what they have now works fine.  On the 
positive side, decorators for classes will probably be significantly 
easier to understand and implement than metaclasses, though less powerful.

So, my thoughts on class decorators are:
1) They have their purposes, and direct support will probably bring out 
more uses of them
2) Orthogonality is good -- the same syntax for methods and classes 
would be preferred, and understanding would transfer between the two forms
3) They are not needed right now -- lets just make sure they can work in 
the same way as method decorators if we want them later


And for the record, I prefer the @-syntax.  Readable, short, easily 
commentable, multi-line, and before the function declaration.  I think 
it will be a nice addition.  Though other characters would be fine, too. 
  I'm really tired of typing my method names three times with the 2.3 
syntax. ;)

Thanks for your time,
-Shane Holloway
From martin at v.loewis.de  Fri Aug  6 19:08:24 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 19:08:25 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <1091806774.8414.63.camel@localhost>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
Message-ID: <4113BB08.1080407@v.loewis.de>

Barry Warsaw wrote:
> In summary, I can definitely sympathize with the concern of tool authors
> who have settled on @ as a meta-character that Python will not use.  I
> wonder how the Leo or ipython folks would feel if the only change to the
> current implementation of PEP 318 was to change the decorator
> introducing character.  If we can increase the community's consensus by
> making this little change, I wouldn't oppose it.

My exact feelings. That it breaks existing tools *is* a relevant and
objective objection, and if all it takes to fix it is to use a different
character, this gets +1 from me.

Regards,
Martin
From niemeyer at conectiva.com  Fri Aug  6 19:11:54 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Fri Aug  6 19:09:35 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <411369F9.4580.DDF2B6F1@coal.murkworks.com>
References: <41136453.1040908@interlink.com.au>
	<411369F9.4580.DDF2B6F1@coal.murkworks.com>
Message-ID: <20040806171154.GB3205@burma.localdomain>

> Could the decorator list be treated specially, like a docstring?
> 
> class C(object):
> 
>     def longMethodNameForEffect(longArgumentOne=None,
>                                 longArgumentTwo=42):
>         [
>             staticmethod,
>             funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
>                       status="experimental", author="BDFL")
>         ]
>         """This method blah, blah.

There are compatibility problems, since this is valid code
right now (same issues which left dec-before-def as a no-no).

-- 
Gustavo Niemeyer
http://niemeyer.net
From tismer at stackless.com  Fri Aug  6 19:15:04 2004
From: tismer at stackless.com (Christian Tismer)
Date: Fri Aug  6 19:15:03 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408061557.i76FvGo07481@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
Message-ID: <4113BC98.3020407@stackless.com>

Guido van Rossum wrote:

>>So I spent a little time futzing with python-mode and some of my
>>decorator code, to try some alternative leading characters.  Of the ones
>>that cannot be used in valid Python code today (i.e. no backward
>>compatibility issues), I tried ':', '/', '=', '*', and '|'.
> 
> 
> If the community can rally behind one of these, I think that would be
> acceptable.  They all seem arbitrary, but so is the choice of '@'. :-)

Not that I want to put more oil into this never ending
story, but I think a few messages ago you mentioned that
the primary purpose of all the decorator syntax effort
is to get the

foo = classmethod(foo)

into the right place, not after but before the definition.

Dunno if this was mentioned before, but why not simply do that?
Move the statement right before the def and provide a way
that this gets recognized correctly.

No syntax change, just the convenience people asked for.

Well, just an idea -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From barry at python.org  Fri Aug  6 19:20:59 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug  6 19:20:49 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <f8ef9ae40408052303384b55c@mail.gmail.com>
References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz>
	<e443ad0e04080522023e4029e6@mail.gmail.com>
	<1f7befae04080522314d818d8f@mail.gmail.com>
	<f8ef9ae40408052303384b55c@mail.gmail.com>
Message-ID: <1091812859.8413.82.camel@localhost>

On Fri, 2004-08-06 at 02:03, IxokaI wrote:

>   Doesn't the fact that the @stuff is at the same level and has no
> connection to the def besides 'above' bother you in our universe of
> indentation-love?

It did for me...originally.  In fact, early in the discussions I favored
list-before-colon syntax.  Now that I've had a chance to play with pie
syntax and rewrite my code (many several of times) for all the various
syntax options, I am definitely favoring decorator-before-def (e.g. pie
decorators).  My primary concerns with that were in using list notation
which is legal code today (that's bad because decorators would subtly
not do what's intended in older Pythons), and the visual attachment of
decorators to the function they modify.  The latter turns out not to be
much of a problem in practice, IMO.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/4f6eb503/attachment-0001.pgp
From tim.peters at gmail.com  Fri Aug  6 19:21:42 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug  6 19:21:44 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408061634.i76GYEZ07623@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com> 
	<4113AF47.1010405@interlink.com.au>
	<200408061634.i76GYEZ07623@guido.python.org>
Message-ID: <1f7befae0408061021600409fb@mail.gmail.com>

[Guido]
> A reference manual describes the status quo.  How can
> it possibly begin to describe future directions?  (Except when
> reserving certain classes of identifiers like C does.)

If it's a designer's intent that, say, the '#' character will never be
used by the language, then the reference manual can simply say that. 
In the absence of such explicit promises, it is indeed only describing
the current state of the language,

It's unreasonable to read the current Python ref as promising that @
(and ? and $) will never be used in any version of Python.  It's
understandable that people would bet on it, though, since that little
set of unused characters hasn't changed in Python's history.

If you want to say that Python will *never* use one (or more) of those
characters, it's easy to add such a promise.  If you don't want to
make such a promise, then it may be clearer (for non-lawyers) to say
that Python reserves @$?.
From ixokai at gmail.com  Fri Aug  6 19:23:19 2004
From: ixokai at gmail.com (IxokaI)
Date: Fri Aug  6 19:23:24 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <200408061658.i76Gwxf07727@guido.python.org>
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org>
	<017601c47bc7$2c84b720$6700a8c0@computer>
	<200408061537.i76Fbv207317@guido.python.org> 
	<cf0csj$htn$1@sea.gmane.org>
	<200408061658.i76Gwxf07727@guido.python.org>
Message-ID: <f8ef9ae4040806102365c736b2@mail.gmail.com>

On Fri, 06 Aug 2004 09:58:59 -0700, Guido van Rossum <guido@python.org> wrote:
> > decorate classmethod:
> >      def foo(arg1,arg2):
> >          ...
> Somebody should add to the PEP that solutions requiring an extra
> indent level for the function have a drawback.  Having your minimal
> indent level be three deep is painful for those using limited-width
> windows; also, the inconsistent indentation between methods with and
> without decorators would be a readability problem (plus,
> adding/removing decorators would require reindenting the entire method
> body).

Doh. That's true... What about:

class foo:
    def bar(arg1, arg2):
        | staticmethod
        | interface(int, string)
        """ This is my doc string """
        for i in range(arg1):
            print arg2

I've been convinced that the list-after-def is bad, but still loathe @
as painful, and really, really, really hate how the @decorators do not
fit into our indented world of grouping stuff together well. I think a
keyword like, say, "using" instead of | would be even better, but...
:)

class foo:
    def blah(self): 
        return None
    | staticmethod
    | interface(int, string)
    def bar(arg1, arg2):
        """ This is my doc string """
        for i in range(arg1):
            print arg2

In that example, only knowing "the bar goes down" clearly connects the
decorators to the right method. And if 'blah' was one of those
bad-styled one-line-functions...

--Stephen
From niemeyer at conectiva.com  Fri Aug  6 19:28:48 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Fri Aug  6 19:26:29 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <1091790258.411365b20c38a@mcherm.com>
References: <1091790258.411365b20c38a@mcherm.com>
Message-ID: <20040806172848.GC3205@burma.localdomain>

> > Is there something to be done that could change the current
> > decision? Voting? Collecting pros/cons and classifying them?
> 
> Yes, there is. And it's not voting <wink>.
> 
> Write a simple "white paper" summarizing the major arguments
> and showing how they apply to the 3 or 4 top syntax proposals.
> Solicit input from c.l.py or python-dev after you have a first
> draft. After the 3rd draft is done, post it here and send it
> to Guido. Then sit back and quietly put up with the resulting
> decision whether you agree or not.

I just did that. I've gone through all the recent discussions
updating the Wiki page with mentioned arguments and new
syntax proposals.

> At this point, I long ago said most everything I had to say
> and so I'm working on the "quietly put up with the results"
> part myself.

I don't think there's something else I could
collaborate either.

-- 
Gustavo Niemeyer
http://niemeyer.net
From niemeyer at conectiva.com  Fri Aug  6 19:34:04 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Fri Aug  6 19:31:48 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <16659.34238.443364.229226@montanaro.dyndns.org>
References: <20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<1091740792.8546.133.camel@localhost>
	<4112A746.6010501@v.loewis.de> <2m657w7b1k.fsf@starship.python.net>
	<1091792604.1770.8.camel@localhost>
	<16659.34238.443364.229226@montanaro.dyndns.org>
Message-ID: <20040806173403.GD3205@burma.localdomain>

>     Mark> I think it is just the PEP that needs updating.
> 
> Not to pick on Mark, but everyone seems to think the PEP needs
> updating but nobody's stepping up to offer patches.  Anthony and I are
> both about out of round tuits...

I've just updated the Wiki with all arguments and syntax proposals
I could find in the recent discussions. Not sure if staying in the
Wiki is ok or if it should be migrated to the PEP now. Adding
information to the Wiki is a lot easier, so perhaps the PEP should
be updated with information from there gradually, or after
stabilization.

-- 
Gustavo Niemeyer
http://niemeyer.net
From fperez528 at yahoo.com  Fri Aug  6 19:40:47 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Fri Aug  6 19:40:55 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
	<4113AF47.1010405@interlink.com.au>
	<200408061634.i76GYEZ07623@guido.python.org>
Message-ID: <cf0fqv$pu9$1@sea.gmane.org>

Guido van Rossum wrote:

>> - change the language ref that talks about this to state that
>> although Python currently doesn't use these symbols, this is not
>> stopping a future version of Python from using them for some new
>> feature.
> 
> Does that really have to be stated?  Isn't this just giving in to the
> silly lawyers?  A reference manual describes the status quo.  How can
> it possibly begin to describe future directions?  (Except when
> reserving certain classes of identifiers like C does.)

As I stated before, it was always perfectly clear to me that in using @$? for
ipython, I was taking a risk.  So I don't feel in any way 'lied to' by the
documentation, since nowhere did it make any kind of promise for the future. 
It only stated the current status quo, and that's how I read it.  I decided to
make use of that status quo for my purposes, but being fully aware that I
might have to adapt in the future.

Best,

f

From psheer at icon.co.za  Fri Aug  6 19:39:16 2004
From: psheer at icon.co.za (Paul Sheer)
Date: Fri Aug  6 19:42:55 2004
Subject: [Python-Dev] Merging Python and Gtk...
Message-ID: <20040806173916.GA2212@divinian>


Now that Gtk is supported on Windows, X, and framebuffers,
would it not be worthwhile to consider packaging Python and
Gtk together?

Python+Gtk is a killer combination, and I see the only
argument against Python's mainstream adoption to be its lack
of a standardized widget library as portable and eligant as
Python itself.

If Python and Gtk installed as one package, this requirement
would be met. At the moment, getting Python to work with
Gtk on an arbitrary Unix system requires compiling no less
than 14 tar balls from source. There are also huge overlaps in
functionality, like Python and Gtk image support (jpeg etc.)

comments?

-paul

Paul Sheer . . . . . . . . . . . . . . . . .  Tel  . . +27 (0)21 6869634
Email . . . http://2038bug.com/email.gif . . . . . . . . . . . . . . . .
http://www.icon.co.za/~psheer . . . . . . . . .  http://rute.2038bug.com
L I N U X . . . . . . . . . . . . . . . . The Choice of a GNU Generation

From bac at OCF.Berkeley.EDU  Fri Aug  6 19:50:01 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug  6 19:50:02 2004
Subject: [Python-Dev] Merging Python and Gtk...
In-Reply-To: <20040806173916.GA2212@divinian>
References: <20040806173916.GA2212@divinian>
Message-ID: <4113C4C9.2090904@ocf.berkeley.edu>

Paul Sheer wrote:

> Now that Gtk is supported on Windows, X, and framebuffers,
> would it not be worthwhile to consider packaging Python and
> Gtk together?
> 
> Python+Gtk is a killer combination, and I see the only
> argument against Python's mainstream adoption to be its lack
> of a standardized widget library as portable and eligant as
> Python itself.
> 
> If Python and Gtk installed as one package, this requirement
> would be met. At the moment, getting Python to work with
> Gtk on an arbitrary Unix system requires compiling no less
> than 14 tar balls from source. There are also huge overlaps in
> functionality, like Python and Gtk image support (jpeg etc.)
> 
> comments?
> 

Isn't Gtk under the LGPL?  That pretty much kills it right there if it is.

Besides Guido has made the comment publicly (last PyCON) that if he were 
to choose a GUI toolkit to bundle with Python it would be wxPython.  So 
there is already one vote going with another toolkit.

And the 14 tarball thing ain't our fault.  =)

-Brett
From fperez528 at yahoo.com  Fri Aug  6 19:50:25 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Fri Aug  6 19:50:32 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
Message-ID: <cf0gd1$smv$1@sea.gmane.org>

Tim Peters wrote:

> [Barry Warsaw]

>> In summary, I can definitely sympathize with the concern of tool authors
>> who have settled on @ as a meta-character that Python will not use.
> 
> Yup.
> 
>> I wonder how the Leo or ipython folks would feel if the only change to the
>> current implementation of PEP 318 was to change the decorator
>> introducing character.  If we can increase the community's consensus by
>> making this little change, I wouldn't oppose it.
> 
> Me neither.
> 
>> My preference then would be to use = with | and : as alternatives, in
>> that order.

Well, here's  from the ipython author...

As I've said, I can adapt to @, but it will mean a bunch of silly work (lots of
documentation changes to track), and significant disruption to a large
existing user base.  Note that ipython has made it into SuSe Professional, and
it's available for OSX via Fink, Debian, Gentoo, and several other smaller
Linux distributions, as well as WinXP/2k.  I mention this only to point out
the extent of the community which will feel this disruption, not in trying to
force anyone's hand by a 'mob argument'.

If the 'decorated line before def' approach wins in Guido's mind, and the
choice of decorator character is the only remaining question, I'd have to say
that using | (or one of the others) would definitely make my life easier.  And
that of all my users as well.

The visual effect of | when initially proposed looked surprisingly nice to me,
given its 'diagrammatic' appearance of drawing a graph between the decoration
lines and the function definition.  So FWIW, I'd say that I personally kind of
like it.  Though I tend to prefer the []-after-args-before-colon one even
better, but I don't want to get into the whole debate.  I'm trying to confine
my comments to the effects of this change on ipython, which seems to be (with
Leo) one of the worst-hit third-party tools by the '@' proposal.

I'd like to say that I greatly appreciate the fact that the python core team is
willing to keep my concerns in mind, even when it was my own choice to use @
without any promises from them.  I think in the middle of a discussion which
has sometimes strayed from the technical to the emotional, it's worth
recognizing the quality of the people we have the privilege of working with.

Best to all,

f

From mmclay at comcast.net  Fri Aug  6 20:06:31 2004
From: mmclay at comcast.net (Michael McLay)
Date: Fri Aug  6 20:05:15 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408061557.i76FvGo07481@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
Message-ID: <200408061406.31244.mmclay@comcast.net>

On Friday 06 August 2004 11:57 am, Guido van Rossum wrote:
> > So I spent a little time futzing with python-mode and some of my
> > decorator code, to try some alternative leading characters.  Of the ones
> > that cannot be used in valid Python code today (i.e. no backward
> > compatibility issues), I tried ':', '/', '=', '*', and '|'.
>
> If the community can rally behind one of these, I think that would be
> acceptable.  They all seem arbitrary, but so is the choice of '@'. :-)

Why was  ';' not on Barry's list? It looks like it could be safely used as a 
leading character.

>> def f():
...   ; bar = 3
  File "<stdin>", line 2
    ; bar = 3
    ^
SyntaxError: invalid syntax
>>> ; descriptor=42
  File "<stdin>", line 1
    ; descriptor=42
    ^
SyntaxError: invalid syntax

Using ';' instead of '@' would also make creating a list of decorators on one 
line consistent with other uses of ';' since ';' is already used to indicate 
multiple statement on one line. 

;framework_stuff(lots, of, args)
class Quux(object):

    ;check_args(int, str)
    ;counted ;staticmethod
    def frobnicate(foo, bar):
        pass

Admittedly the ';' does suffer from not being as visible as '@', but at least 
the tail on the bottom dot makes it more readable than the ':' character. 

The other downside is the potential confusion it might cause to geriatric 
elisp programmers.


From anthony at interlink.com.au  Fri Aug  6 20:08:00 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 20:08:14 2004
Subject: [Python-Dev] PEP-0006, Bug Fix Releases
Message-ID: <4113C900.6060902@interlink.com.au>

This is an updated version of PEP-0006, the Bugfix Release PEP.
Unless I hear some screams, I'll be checking this into CVS. It
more accurately reflects the current state of the bugfix process.
(This has been sitting in my todo list for a while)

-------------- next part --------------
PEP: 6
Title: Bug Fix Releases
Version: $Revision: 1.10 $
Author: aahz@pobox.com (Aahz), anthony@interlink.com.au (Anthony Baxter)
Status: Active
Type: Informational
Created: 15-Mar-2001
Post-History: 15-Mar-2001 18-Apr-2001


Abstract

    Python has historically had only a single fork of development,
    with releases having the combined purpose of adding new features
    and delivering bug fixes (these kinds of releases will be referred
    to as "feature releases").  This PEP describes how to fork off
    maintenance, or bug fix, releases of old versions for the primary 
    purpose of fixing bugs.

    This PEP is not, repeat NOT, a guarantee of the existence of bug fix
    releases; it only specifies a procedure to be followed if bug fix
    releases are desired by enough of the Python community willing to
    do the work.


Motivation

    With the move to SourceForge, Python development has accelerated.
    There is a sentiment among part of the community that there was
    too much acceleration, and many people are uncomfortable with
    upgrading to new versions to get bug fixes when so many features
    have been added, sometimes late in the development cycle.

    One solution for this issue is to maintain the previous feature
    release, providing bug fixes until the next feature release.  This
    should make Python more attractive for enterprise development,
    where Python may need to be installed on hundreds or thousands of
    machines.


Prohibitions

    Bug fix releases are required to adhere to the following restrictions:

    1. There must be zero syntax changes.  All .pyc and .pyo files
       must work (no regeneration needed) with all patch releases
       forked off from a feature release.

    2. There must be zero pickle changes.

    3. There must be no incompatible C API changes.  All extensions
       must continue to work without recompiling in all patch releases
       in the same fork as a feature release.

    Breaking any of these prohibitions requires a BDFL proclamation
    (and a prominent warning in the release notes).

Not-Quite-Prohibitions

    Where possible, bug fix releases should also:

    1. Have no new features. The purpose of a bug fix release is to 
       fix bugs, not add the latest and greatest whizzo feature from
       the HEAD of the CVS root.

    2. Be a painless upgrade. Users should feel confident that an
       upgrade from 2.x.y to 2.x.(y+1) will not break their running
       systems. This means that, unless it is necessary to fix a bug,
       the standard library should not change behaviour, or worse yet,
       APIs.

Helping the Bug Fix Releases Happen

    Here's a few pointers on helping the bug fix release process along.

    1. Backport bug fixes. If you fix a bug, and it seems appropriate,
       port it to the CVS branch for the current bug fix release. If
       you're unwilling or unable to backport it yourself, make a note
       in the commit message.

    2. If you're not sure, ask. Ask the person managing the current bug
       fix releases if they think a particular fix is appropriate.

    3. If there's a particular bug you'd particularly like fixed in a
       bug fix release, jump up and down and try to get it done. Do not
       wait until 48 hours before a bug fix release is due, and then
       start asking for bug fixes to be included.

Version Numbers

    Starting with Python 2.0, all feature releases are required to have
    a version number of the form X.Y; patch releases will always be of
    the form X.Y.Z.

    The current feature release under development is referred to as
    release N; the just-released feature version is referred to as N-1.

    In CVS, the bug fix releases happen on a branch. For release 2.x,
    the branch is named 'release2x-maint'. For example, the branch for
    the 2.3 maintenance releases is release23-maint


Procedure

    The process for managing patch releases is modeled in part on the cl
    Tsystem [1].

    The Patch Czar is the counterpart to the BDFL for patch releases.
    However, the BDFL and designated appointees retain veto power over
    individual patches.

    As individual patches get contributed to the feature release fork,
    each patch contributor is requested to consider whether the patch is
    a bug fix suitable for inclusion in a patch release. If the patch is
    considered suitable, the patch contributor will mail the SourceForge
    patch (bug fix?) number to the maintainers' mailing list.

    In addition, anyone from the Python community is free to suggest
    patches for inclusion. Patches may be submitted specifically for
    patch releases; they should follow the guidelines in PEP 3 [2].
    In general, though, it's probably better that a bug in a specific
    release also be fixed on the HEAD as well as the branch.

    The Patch Czar decides when there are a sufficient number of patches
    to warrant a release. The release gets packaged up, including a
    Windows installer, and made public. If any new bugs are found, they
    must be fixed immediately and a new patch release publicized (with
    an incremented version number).

    Bug fix releases are expected to occur at an interval of roughly
    six months. This is only a guideline, however - obviously, if a
    major bug is found, a bugfix release may be appropriate sooner. In
    general, only the N-1 release will be under active maintenance at
    any time. That is, during Python 2.4's development, Python 2.3 gets
    bugfix releases.


Patch Czar History

    Anthony Baxter is the Patch Czar for 2.3.1 through 2.3.4.

    Barry Warsaw is the Patch Czar for 2.2.3.

    Guido van Rossum is the Patch Czar for 2.2.2.

    Michael Hudson is the Patch Czar for 2.2.1.

    Anthony Baxter is the Patch Czar for 2.1.2 and 2.1.3.

    Thomas Wouters is the Patch Czar for 2.1.1.

    Moshe Zadka is the Patch Czar for 2.0.1.


History

    This PEP started life as a proposal on comp.lang.python.  The
    original version suggested a single patch for the N-1 release to
    be released concurrently with the N release.  The original version
    also argued for sticking with a strict bug fix policy.

    Following feedback from the BDFL and others, the draft PEP was
    written containing an expanded patch release cycle that permitted
    any previous feature release to obtain patches and also relaxed
    the strict bug fix requirement (mainly due to the example of PEP
    235 [3], which could be argued as either a bug fix or a feature).

    Discussion then mostly moved to python-dev, where BDFL finally
    issued a proclamation basing the Python patch release process on
    Tcl's, which essentially returned to the original proposal in
    terms of being only the N-1 release and only bug fixes, but
    allowing multiple patch releases until release N is published.

    Anthony Baxter then took this PEP and revised it, based on 
    lessons from the 2.3 release cycle.

References

    [1] http://dev.scriptics.com:8080/cgi-bin/tct/tip/28.html

    [2] PEP 3, Guidelines for Handling Bug Reports, Hylton
        http://www.python.org/peps/pep-0003.html

    [3] PEP 235, Import on Case-Insensitive Platforms, Peters
        http://www.python.org/peps/pep-0235.html


Copyright

    This document has been placed in the public domain.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:
From anthony at interlink.com.au  Fri Aug  6 20:15:31 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 20:15:50 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408061406.31244.mmclay@comcast.net>
References: <200408051636.i75Gaak03654@guido.python.org>	<1091806774.8414.63.camel@localhost>	<200408061557.i76FvGo07481@guido.python.org>
	<200408061406.31244.mmclay@comcast.net>
Message-ID: <4113CAC3.1010603@interlink.com.au>

Michael McLay wrote:
> On Friday 06 August 2004 11:57 am, Guido van Rossum wrote:
> 
>>>So I spent a little time futzing with python-mode and some of my
>>>decorator code, to try some alternative leading characters.  Of the ones
>>>that cannot be used in valid Python code today (i.e. no backward
>>>compatibility issues), I tried ':', '/', '=', '*', and '|'.
>>
>>If the community can rally behind one of these, I think that would be
>>acceptable.  They all seem arbitrary, but so is the choice of '@'. :-)
> 
> 
> Why was  ';' not on Barry's list? It looks like it could be safely used as a 
> leading character.

One objection is that there's a number of contexts where ; is the
comment notation. Another is, to me, the ';' just looks misplaced.
Maybe it was supposed to be on the end of the line before, but there's
a line break in the wrong place?

I also don't think ; is that much better than : - I think it violates
one of the unwritten bits of Python Zen - Syntax Shall Not Look Like
Grit On Tim's Monitor. <wink>


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From pje at telecommunity.com  Fri Aug  6 20:21:07 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug  6 20:17:00 2004
Subject: [Python-Dev] pep 318, Decorators for Functions, Methods
	and Classes
In-Reply-To: <200408061555.i76FtV707448@guido.python.org>
References: <Your message of "Fri, 06 Aug 2004 17:38:21 +0200."
	<A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
Message-ID: <5.1.1.6.0.20040806141835.032fed90@mail.telecommunity.com>

At 08:55 AM 8/6/04 -0700, Guido van Rossum wrote:
>It would seem that the functionality isn't quite as useful there; you
>can get most of the same effects with metaclasses.  I'm not aware of
>any idiom that takes a class and passes it through some kind of
>wrapper or transformation *after* the class declaration; if somebody
>is using this, it surely must be a very rare use indeed.

Zope and PyProtocols use this approach for inline interface declarations, 
though it's implemented by _getframe and a temporary metaclass.  E.g. Zope's:

class Foo:
     implements(IFoo)

works by effectively applying a decorator to the finished 
class.  PyProtocols doesn't change the class in the process, but I think 
Zope might (e.g. to set __implements__).

From pje at telecommunity.com  Fri Aug  6 20:23:39 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug  6 20:19:32 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408061557.i76FvGo07481@guido.python.org>
References: <Your message of "Fri,
	06 Aug 2004 11:39:34 EDT." <1091806774.8414.63.camel@localhost>
	<200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
Message-ID: <5.1.1.6.0.20040806142156.03300620@mail.telecommunity.com>

At 08:57 AM 8/6/04 -0700, Guido van Rossum wrote:
> > So I spent a little time futzing with python-mode and some of my
> > decorator code, to try some alternative leading characters.  Of the ones
> > that cannot be used in valid Python code today (i.e. no backward
> > compatibility issues), I tried ':', '/', '=', '*', and '|'.
>
>If the community can rally behind one of these, I think that would be
>acceptable.  They all seem arbitrary, but so is the choice of '@'. :-)

How about '%'? This:

     %classmethod

looks more like some kind of pragma or compiler directive to me than '@', 
which looks more like some kind of documentation.

From martin at v.loewis.de  Fri Aug  6 20:22:41 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 20:22:47 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408061634.i76GYEZ07623@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>	<1f7befae040806090644be9c04@mail.gmail.com>
	<4113AF47.1010405@interlink.com.au>
	<200408061634.i76GYEZ07623@guido.python.org>
Message-ID: <4113CC71.4070104@v.loewis.de>

Guido van Rossum wrote:
> Does that really have to be stated?  Isn't this just giving in to the
> silly lawyers?  A reference manual describes the status quo.  How can
> it possibly begin to describe future directions? 

However, the Python reference manual became more than a reference 
manual, as it also talks about what a Python implementation can
and cannot do, e.g. for memory management.

Providing future directions in the reference manual is a good
thing to do if possible as it helps competing implementations
and tool vendors. Clearly, people do make assumptions that a
documented aspect of the language is not going to change, unless
the documentation says that it is likely to change.

Regards,
Martin
From pje at telecommunity.com  Fri Aug  6 20:28:30 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug  6 20:24:21 2004
Subject: [Python-Dev] pie-thon redux
In-Reply-To: <4113AE45.5080004@interlink.com.au>
Message-ID: <5.1.1.6.0.20040806142605.03303d90@mail.telecommunity.com>

At 02:13 AM 8/7/04 +1000, Anthony Baxter wrote:
>As a break from the decorator thread, NTK (www.ntk.net)
>had this writeup of the Pie-Thon. It amused me.
>
>[snip]

I'm not usually one for today's IM slang, but...

OMG!!!!  RTFLOL!!!  :)


>Feel the punctuation rising in you, Guido!)

And so, the '@' sign, symbol of the pie, came into being...  ;)

From martin at v.loewis.de  Fri Aug  6 20:25:28 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 20:25:30 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <4113BC98.3020407@stackless.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com>
Message-ID: <4113CD18.8080803@v.loewis.de>

Christian Tismer wrote:
> the primary purpose of all the decorator syntax effort
> is to get the
> 
> foo = classmethod(foo)
> 
> into the right place, not after but before the definition.
> 
> Dunno if this was mentioned before, but why not simply do that?
> Move the statement right before the def and provide a way
> that this gets recognized correctly.
> 
> No syntax change, just the convenience people asked for.

I don't think this can work. How precisely would you implement
it? Python traditionally executes source code from top to bottom,
except for loops.

Regards,
Martin
From John.Marshall at ec.gc.ca  Fri Aug  6 20:25:52 2004
From: John.Marshall at ec.gc.ca (John Marshall)
Date: Fri Aug  6 20:26:04 2004
Subject: [Python-Dev] Re: elements of decorator syntax suggestions
In-Reply-To: <d11dcfba.0408060919.10c7cdd6@posting.google.com>
References: <d11dcfba.0408060919.10c7cdd6@posting.google.com>
Message-ID: <1091816752.3439.33.camel@mango.cmc.ec.gc.ca>

On Fri, 2004-08-06 at 17:19, Steven Bethard wrote:
> I think one of the biggest reasons we're having such problems coming
> to any agreement on decorator syntax is that each proposal makes a
> number of syntax decisions, not just one.  For decorators, I see the
> following decisions that must be made:
> 

> 1) Indicator
> 
> 2) Location
> 
> 3) List notation
> 
> 4) Indentation

I think you have done a good job identifying and
isolating what the issues are regarding decorator
options. I think another thing that should perhaps
be made clearer (maybe for me only?) is to identify,
in order of importance/signfigance, the relationship
between decorators and the function:
1) is the function/method signature more important
   than the decorators (to the programmer)?
2) do the decorators belong or are affected by the function?
3) what other constructs in Python say/imply "look ahead
   for what I am about to affect/define"?

I understand that designing programming languages is
sometimes an art rather than a science (natural languages
a chock full of exceptions to the rule). But if the
priority of functions and decorators are established,
then I think this would help.

In response to my own questions above:
1) From my perspective, it seems that almost everybody
   would say that the function (class also) signature is
   more important than decorators. For example, how many
   people would say "I wonder where the classmethod
   function is?" Instead, people would say, "Where is my
   function xyz()". This seems to argue for the decorators
   following (somewhere) after the signature.

   Of course highlighting editors can help by marking off
   function names and parameters, but this argument only
   serves to confirm that the signature is the KEY thing
   programmers are interested in. Apart from aesthetics,
   which are subjective and could be debated endlessly, I
   cannot see how one could argue otherwise.

2) Perhaps I am not understanding fully. Can someone explain
   how decorators before the function logically belong outside
   of the function definition block if they are intended to
   affect the function? Are the decorators meant to affect
   something other than the function? It seems to me that
   in every other situation in Python (please correct me if I
   am wrong) blocks are the way we are able to identify what
   belongs to what (e.g., local variable to a function).
   The current decorator approach says to me, "I belong to
   the class" (if a class is involved), or "I belong to the
   module" (if no class is involved).

3) In a sense, this point is somewhat related to #2. Does
   Python support an implicit "I am doing something now, on
   something I haven't defined, but which is coming up next"
   anywhere else than the decorator before function
   construction? Even in classes, Python forces the use of
   self.xxx (explicit).
    
Hope this contributes to the discussion.

John

From skip at pobox.com  Fri Aug  6 20:34:58 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 20:35:21 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <200408061658.i76Gwxf07727@guido.python.org>
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org>
	<017601c47bc7$2c84b720$6700a8c0@computer>
	<200408061537.i76Fbv207317@guido.python.org>
	<cf0csj$htn$1@sea.gmane.org>
	<200408061658.i76Gwxf07727@guido.python.org>
Message-ID: <16659.53074.263268.946749@montanaro.dyndns.org>


    Guido> Somebody should add to the PEP that solutions requiring an extra
    Guido> indent level for the function have a drawback.  

Done.

Skip
From martin at v.loewis.de  Fri Aug  6 20:38:16 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 20:38:19 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <007d01c47bbc$49a2a340$6700a8c0@computer>
References: <Pine.LNX.4.58.0408061536370.3743@suzi.com.onego.ru>	<411371EC.10403@v.loewis.de>
	<007d01c47bbc$49a2a340$6700a8c0@computer>
Message-ID: <4113D018.3020809@v.loewis.de>

Edward K. Ream wrote:
> I agree.  The argument that @ lines aren't clearly "attached" to functions
> seems much more convincing to me.

I can't see that. In Python, things at the same indentation level all
belong together. Would you say that in

   try:
      foo()
      more_code()
   except expr:
      bar()

the except is not attached to the try? You probably wouldn't, because
you know they belong together. However, you equally have no problem
seeing how

   while:
      foo()
      more_code()
   if expr:
      bar()

the while and the if don't belong together. People can learn these
things, and learning the @ thing seems very easy.

> 2.  Maybe the @ syntax is somehow the best alternative, considered in
> isolation.  Fair enough.  But how much better?  Enough to justify a major
> change to the language?  What is so unreadable about def (...)[...]: ?

That parameters and decorators are essentially in a single sequence, 
just separated by )[. It is important that you can easily tell how many
parameters a function has, and how it is decorated.

Furthermore, this syntax *will* break more tools than the @decorator
syntax.

> In short, a major change to Python (@) would seem to demand a truly
> convincing justification.  IMO, the argument that @ is more readable than
> def (...)[...]: doesn't even come _close_ to such a justification.

Take into account also breakage of existing tools.

Regards,
Martin
From amk at amk.ca  Fri Aug  6 20:42:47 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Aug  6 20:43:03 2004
Subject: [Python-Dev] Merging Python and Gtk...
In-Reply-To: <20040806173916.GA2212@divinian>
References: <20040806173916.GA2212@divinian>
Message-ID: <20040806184247.GA32440@rogue.amk.ca>

On Fri, Aug 06, 2004 at 07:39:16PM +0200, Paul Sheer wrote:
> If Python and Gtk installed as one package, this requirement
> would be met. At the moment, getting Python to work with

This isn't really a core Python issue; distributing the GTk+ source
with Python isn't going to happen because of the license, because the
version would have to be kept in sync, etc.  However, there's nothing
stopping someone from building their own Windows installer that
includes GTk+ as well as (or in place of) Tkinter.  

PEP 206 was started to describe a richer Python distribution, but its
author no longer has time to work on the task.  You could certainly
pick up the PEP and resume working on it.

--amk
From marktrussell at btopenworld.com  Fri Aug  6 20:50:07 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Fri Aug  6 20:50:11 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <cf0gd1$smv$1@sea.gmane.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain> <ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
	<cf0gd1$smv$1@sea.gmane.org>
Message-ID: <1091818207.7529.33.camel@localhost>

On Fri, 2004-08-06 at 18:50, Fernando Perez wrote:
> The visual effect of | when initially proposed looked surprisingly nice to me 
> given its 'diagrammatic' appearance of drawing a graph between the decoration
> lines and the function definition.  So FWIW, I'd say that I personally kind of
> like it.

I too like |.  It is not quite as visible as @, but I don't think that's
a serious problem, and not breaking ipython and other tools is a big
win.  I just tried implementing this change, and all the tests still
pass (not a big surprise, but worth confirming I guess).

Mark

From marktrussell at btopenworld.com  Fri Aug  6 20:50:09 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Fri Aug  6 20:50:15 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <cf0gd1$smv$1@sea.gmane.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain> <ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
	<cf0gd1$smv$1@sea.gmane.org>
Message-ID: <1091818207.7529.34.camel@localhost>

On Fri, 2004-08-06 at 18:50, Fernando Perez wrote:
> The visual effect of | when initially proposed looked surprisingly nice to me 
> given its 'diagrammatic' appearance of drawing a graph between the decoration
> lines and the function definition.  So FWIW, I'd say that I personally kind of
> like it.

I too like |.  It is not quite as visible as @, but I don't think that's
a serious problem, and not breaking ipython and other tools is a big
win.  I just tried implementing this change, and all the tests still
pass (not a big surprise, but worth confirming I guess).

Mark

From python at rcn.com  Fri Aug  6 08:51:29 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug  6 20:53:18 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408061406.31244.mmclay@comcast.net>
Message-ID: <016d01c47b81$cd859d60$e841fea9@oemcomputer>

> > > that cannot be used in valid Python code today (i.e. no backward
> > > compatibility issues), I tried ':', '/', '=', '*', and '|'.

Is '!' an option?  It has a certain beauty and it looks like it points
at the following line.


Raymond

From skip at pobox.com  Fri Aug  6 20:58:22 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 20:58:45 2004
Subject: [Python-Dev] Questions about '@' in pep 318
In-Reply-To: <4113D018.3020809@v.loewis.de>
References: <Pine.LNX.4.58.0408061536370.3743@suzi.com.onego.ru>
	<411371EC.10403@v.loewis.de>
	<007d01c47bbc$49a2a340$6700a8c0@computer>
	<4113D018.3020809@v.loewis.de>
Message-ID: <16659.54478.967147.267963@montanaro.dyndns.org>


    Martin> In Python, things at the same indentation level all belong
    Martin> together.

I think the point is that the collection of @-decorators and the immediately
following function definition (which can be separated from each other by a
lot of white space or comments, btw) effectively form a single function
definition.  I don't believe there are any other syntactic constructs in
Python where the keyword introducing the construct ("def" in this case)
doesn't occur at the beginning of the construct.

Hmmm...  Now that I think about it, might you "lose" an @ in a comment block
since # and @ have more-or-less the same density?  | might not suffer from
that sort of confusion.

    # description of what decorator1 does
    # description of what decorator1 does
    # description of what decorator1 does
    @ decorator1
    # description of what decorator2 does
    # description of what decorator2 does
    # description of what decorator2 does
    @ decorator2
    # description of what func does
    # description of what func does
    # description of what func does
    def func(...):
        pass

vs. 

    # description of what decorator1 does
    # description of what decorator1 does
    # description of what decorator1 does
    | decorator1
    # description of what decorator2 does
    # description of what decorator2 does
    # description of what decorator2 does
    | decorator2
    # description of what func does
    # description of what func does
    # description of what func does
    def func(...):
        pass

Eh, maybe not.  Small point in any case, I'm sure.

Skip
From martin at v.loewis.de  Fri Aug  6 20:59:06 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 20:59:09 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <016d01c47b81$cd859d60$e841fea9@oemcomputer>
References: <016d01c47b81$cd859d60$e841fea9@oemcomputer>
Message-ID: <4113D4FA.6090805@v.loewis.de>

Raymond Hettinger wrote:
> Is '!' an option?  It has a certain beauty and it looks like it points
> at the following line.

Syntactically, yes. It reminds me of comments, though (e.g. in Xresource
files).

Regards,
Martin
From skip at pobox.com  Fri Aug  6 21:01:14 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 21:01:31 2004
Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it?
In-Reply-To: <E1Bt9kW-0006Lb-NH@sc8-pr-cvs1.sourceforge.net>
References: <E1Bt9kW-0006Lb-NH@sc8-pr-cvs1.sourceforge.net>
Message-ID: <16659.54650.788854.375861@montanaro.dyndns.org>

>From Raymond's checkin:

    Programming Recommendations
  
    - Code should be written in a way that does not disadvantage other
      implementations of Python (PyPy, Jython, IronPython, Pyrex, Psyco, and
      such).  For example, do not rely on CPython's efficient implementation
      of in-place string concatenation for statements in the form a+=b or
      a=a+b.  Those statements run more slowly in Jython.  In performance
      sensitive parts of the library, the ''.join() form should be used
      instead.  This will assure that concatenation occurs in linear time
      across various implementations.

I'm a bit confused.  Why bother to speed up s+=t if we aren't supposed to
use it?

Skip
From anthony at interlink.com.au  Fri Aug  6 21:02:57 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug  6 21:04:39 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <1091818207.7529.34.camel@localhost>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<1f7befae040806090644be9c04@mail.gmail.com>	<cf0gd1$smv$1@sea.gmane.org>
	<1091818207.7529.34.camel@localhost>
Message-ID: <4113D5E1.6040006@interlink.com.au>

Mark Russell wrote:
> I too like |.  It is not quite as visible as @, but I don't think that's
> a serious problem, and not breaking ipython and other tools is a big
> win.  I just tried implementing this change, and all the tests still
> pass (not a big surprise, but worth confirming I guess).

To jump on the band-wagon - I can live with | as well. I was going to
add it to the PEP rewrite, but I'll wait for a decision, to save myself
the writing ;)

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From pje at telecommunity.com  Fri Aug  6 21:09:54 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug  6 21:05:46 2004
Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it?
In-Reply-To: <16659.54650.788854.375861@montanaro.dyndns.org>
References: <E1Bt9kW-0006Lb-NH@sc8-pr-cvs1.sourceforge.net>
	<E1Bt9kW-0006Lb-NH@sc8-pr-cvs1.sourceforge.net>
Message-ID: <5.1.1.6.0.20040806150836.02729ec0@mail.telecommunity.com>

At 02:01 PM 8/6/04 -0500, Skip Montanaro wrote:
>For example, do not rely on CPython's efficient implementation
>       of in-place string concatenation for statements in the form a+=b or
>       a=a+b.  Those statements run more slowly in Jython.

Is that really true?  I thought the last time this came up, somebody said 
that common Java implementations actually had special case code to speed up 
string concatenation.

From barry at python.org  Fri Aug  6 21:09:52 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug  6 21:09:44 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408061539.i76FdtG07333@guido.python.org>
References: <59e9fd3a04080519462bf57899@mail.gmail.com>
	<59e9fd3a04080519462bf57899@mail.gmail.com>
	<5.1.1.6.0.20040806112037.035812e0@mail.telecommunity.com>
	<200408061539.i76FdtG07333@guido.python.org>
Message-ID: <1091819392.10217.3.camel@localhost>

On Fri, 2004-08-06 at 11:39, Guido van Rossum wrote:

> That's a bit strong, but yes, it would be trivial to change.  If
> that's all that's needed to save @deco, it's a lot easier than ripping
> the whole thing out... :-)

I haven't played with many multi-decorator decorators, but the fact that
you can have more than one of them on a line without an intervening
semi-colon doesn't seem right to me.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/58dafc0b/attachment.pgp
From skip at pobox.com  Fri Aug  6 21:11:35 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 21:11:57 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <016d01c47b81$cd859d60$e841fea9@oemcomputer>
References: <200408061406.31244.mmclay@comcast.net>
	<016d01c47b81$cd859d60$e841fea9@oemcomputer>
Message-ID: <16659.55271.674627.269077@montanaro.dyndns.org>


    Raymond> Is '!' an option?  It has a certain beauty and it looks like it
    Raymond> points at the following line.

\N{DOWNWARDS ARROW} anyone?

Skip
From python at rcn.com  Fri Aug  6 09:32:21 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug  6 21:34:09 2004
Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it?
In-Reply-To: <16659.54650.788854.375861@montanaro.dyndns.org>
Message-ID: <018101c47b87$82a884a0$e841fea9@oemcomputer>

> I'm a bit confused.  Why bother to speed up s+=t if we aren't supposed
to
> use it?

In fact, it is does occur all over the place (esp. in newbie code) so it
ought to run as fast as possible or least not be a total disaster.

OTOH, library code gets run on various implementations of Python and it
would be a mistake to change any library code from ''.join() and have it
start performing badly on other implementations.  That was essentially
the only argument against the patch which is otherwise a pure win.

On a side note, the += optimization only works if there is space
(fragmentationwise, not total memory) to resize a string in-place.
While this is usually true, ''.join() always runs in linear time.  Also,
''.join() doesn't run through the eval-loop.  IOW, notwithstanding
cross-implementation performance, ''.join() is usually a better choice
in performance sensitive code.

The goal of the patch is not to change coding styles.  Rather, the goal
is to have it hurt less when you coded += anyways.



Raymond

From bob at redivi.com  Fri Aug  6 21:34:01 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Aug  6 21:34:13 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <4113D5E1.6040006@interlink.com.au>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<1f7befae040806090644be9c04@mail.gmail.com>	<cf0gd1$smv$1@sea.gmane.org>
	<1091818207.7529.34.camel@localhost>
	<4113D5E1.6040006@interlink.com.au>
Message-ID: <91F184DE-E7DF-11D8-ADD9-000A95686CD8@redivi.com>


On Aug 6, 2004, at 3:02 PM, Anthony Baxter wrote:

> Mark Russell wrote:
>> I too like |.  It is not quite as visible as @, but I don't think 
>> that's
>> a serious problem, and not breaking ipython and other tools is a big
>> win.  I just tried implementing this change, and all the tests still
>> pass (not a big surprise, but worth confirming I guess).
>
> To jump on the band-wagon - I can live with | as well. I was going to
> add it to the PEP rewrite, but I'll wait for a decision, to save myself
> the writing ;)

My only objection to | is that it looks near-identical to I in many 
fonts, such as the one I use for email, so it could potentially be 
confused with some sort of magical sys._getframe() using interface 
declaration rather than compile-type syntax.

-bob
From barry at python.org  Fri Aug  6 21:39:53 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug  6 21:39:43 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408061406.31244.mmclay@comcast.net>
References: <200408051636.i75Gaak03654@guido.python.org>
	<1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<200408061406.31244.mmclay@comcast.net>
Message-ID: <1091821193.10211.11.camel@localhost>

On Fri, 2004-08-06 at 14:06, Michael McLay wrote:

> Why was  ';' not on Barry's list? It looks like it could be safely used as a 
> leading character.

It wasn't because he didn't think of it <wink>.

> Admittedly the ';' does suffer from not being as visible as '@', but at least 
> the tail on the bottom dot makes it more readable than the ':' character. 
> 
> The other downside is the potential confusion it might cause to geriatric 
> elisp programmers.

<snort> I resemble that remark. :)

But really, at the very least, semis suffer from readability problems
similar to colons.

bellying-up-to-the-bar-ly y'rs,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/bd0df16a/attachment.pgp
From tim.peters at gmail.com  Fri Aug  6 21:41:19 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug  6 21:41:22 2004
Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it?
In-Reply-To: <5.1.1.6.0.20040806150836.02729ec0@mail.telecommunity.com>
References: <E1Bt9kW-0006Lb-NH@sc8-pr-cvs1.sourceforge.net>
	<E1Bt9kW-0006Lb-NH@sc8-pr-cvs1.sourceforge.net>
	<5.1.1.6.0.20040806150836.02729ec0@mail.telecommunity.com>
Message-ID: <1f7befae040806124128872bc5@mail.gmail.com>

>> For example, do not rely on CPython's efficient implementation
>>       of in-place string concatenation for statements in the form a+=b or
>>       a=a+b.  Those statements run more slowly in Jython.

[Phillip J. Eby]
> Is that really true?  I thought the last time this came up, somebody said
> that common Java implementations actually had special case code to speed up
> string concatenation.

Java != Jython.  Java compilers know when they're dealing with
strings, and the Java spec defines '+' on strings in terms of
mutable-StringBuffer operations.  Even so, the pragmatics of string
concatenation in Java are complex, well illustrated by this
excruciating "tech tip":

    http://java.sun.com/developer/JDCTechTips/2002/tt0305.html

Jython is as blind as Python at compile-time about what the types are
in "a=a+b".
From skip at pobox.com  Fri Aug  6 21:45:43 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 21:46:01 2004
Subject: [Python-Dev] use of #pragma GCC system_header to
	suppress        _XOPEN_SOURCE warnings
In-Reply-To: <4112A968.2030401@v.loewis.de>
References: <16658.42673.542930.749915@montanaro.dyndns.org>
	<4112A968.2030401@v.loewis.de>
Message-ID: <16659.57319.931941.158458@montanaro.dyndns.org>


    >> Unfortunately GCC also defines it, so a "macro redefined" warning is
    >> emitted when compiling with GCC and including Python.h.

    Martin> That is not true. GCC does not define that macro, on any
    Martin> platform.

I was shown today that g++ defines that macro internally (I misspoke when I
referred to gcc yesterday - I tend to lump them together and forget that C++
is a different language than C), at least 3.3.2 on Solaris/Intel does.

Given this small source file:

    #include "Python.h"

    int main() {
            printf("%x\n", PyList_GetItem);
            return 0;
    }

compilation with gcc (we're using 3.3.2, but I also tried with 3.4.0)
produces no warnings, but compilation with g++ it produces this output:

    g++ -I/opt/lang/python/include/python2.3 -c example.c
    In file included from /opt/lang/python/include/python2.3/Python.h:8,
                     from example.c:2:
    /opt/lang/python/include/python2.3/pyconfig.h:862:1: warning: "_XOPEN_SOURCE" redefined
    <built-in>:73:1: warning: this is the location of the previous definition

Sure enough, grepping through the gcc/g++ installation directory doesn't
turn up an actual header file containing "#define _XOPEN_SOURCE".  I did
find a file (.../lib/gcc/i386-pc-solaris2.8/3.4.0/install-tools/
mkheaders.conf) that contains this line:

    FIXPROTO_DEFINES="-D_XOPEN_SOURCE"

    Martin> Some system header may define it, though. I believe this is a
    Martin> bug in the system - the macro *should* be defined in the
    Martin> application. What system are you referring to, and where does
    Martin> that define _XOPEN_SOURCE? Is that definition conditional?

Does the fact that we're talking g++ and a built-in macro instead of gcc and
a macro defined in a .h file suggest a different way to suppress this
warning?

A little more searching located the standards(5) man page on the system.  In
part it says:

  X/Open CAE
     To build or compile an application that conforms to  one  of
     the X/Open CAE specifications, use the following guidelines.
     Applications need not set the POSIX feature test  macros  if
     they require both CAE and POSIX functionality.

     ...

     SUSv2 The application must define _XOPEN_SOURCE=500.

This suggests that Sun expects the application or its build tools to define
_XOPEN_SOURCE, not the compiler or its header files.  That suggests a bug in
g++.

Skip
From skip at pobox.com  Fri Aug  6 21:50:18 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug  6 21:51:00 2004
Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it?
In-Reply-To: <018101c47b87$82a884a0$e841fea9@oemcomputer>
References: <16659.54650.788854.375861@montanaro.dyndns.org>
	<018101c47b87$82a884a0$e841fea9@oemcomputer>
Message-ID: <16659.57594.110222.939272@montanaro.dyndns.org>


    >> I'm a bit confused.  Why bother to speed up s+=t if we aren't
    >> supposed to use it?

    Raymond> In fact, it is does occur all over the place (esp. in newbie
    Raymond> code) so it ought to run as fast as possible or least not be a
    Raymond> total disaster.
    ...

Thanks for the explanation.  Viewing it in a larger context than just the
checkin diff makes it a bit more clear that this doesn't refer to garden
variety application code as well.

Skip
From martin at v.loewis.de  Fri Aug  6 22:04:49 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 22:04:52 2004
Subject: [Python-Dev] use of #pragma GCC system_header to suppress
	_XOPEN_SOURCE warnings
In-Reply-To: <16659.57319.931941.158458@montanaro.dyndns.org>
References: <16658.42673.542930.749915@montanaro.dyndns.org>
	<4112A968.2030401@v.loewis.de>
	<16659.57319.931941.158458@montanaro.dyndns.org>
Message-ID: <4113E461.20600@v.loewis.de>

Skip Montanaro wrote:
> I was shown today that g++ defines that macro internally (I misspoke when I
> referred to gcc yesterday - I tend to lump them together and forget that C++
> is a different language than C), at least 3.3.2 on Solaris/Intel does.

Ah, that one.

I don't think it is a goal that Python generates no warnings if compiled 
with a C++ compiler (I don't think you actually *can* compile all of 
Python with a C++ compiler).

> This suggests that Sun expects the application or its build tools to define
> _XOPEN_SOURCE, not the compiler or its header files.  That suggests a bug in
> g++.

Indeed, and I believe this is a known bug. g++ needs to define
_XOPEN_SOURCE so that some of Sun's system headers declare a number of
prototypes which are then referred-to in inline functions of libstdc++
header files; I think this is about wchar_t functions, thread functions,
or C99 floating-point functions. It's not clear whether this is Sun's
or g++' fault - g++ should probably fixinclude the system headers
appropriately, but doesn't.

This gives us a number of options:
1. On Solaris, define _XOPEN_SOURCE to the same value that g++ uses.
    This should then cause no redefinition, but might cause some
    functions go away. We would need an autoconf test to find out
    what the value of _XOPEN_SOURCE is that g++ uses.
2. As g++ is only used to compile main.cc, we could #undef
    _XOPEN_SOURCE there - either the one of g++, or our own one
    (appropriately commented, of course).
3. We could advise users to explicitly compile --without-cxx
    on Solaris when using g++. I believe it is not actually needed,
    but our autoconf test is not strong enough to find out, and
    errs on the side of building main.cc with a C++ compiler if
    one is available and the user hasn't requested that it won't.

Regards,
Martin
From martin at v.loewis.de  Fri Aug  6 22:07:46 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 22:07:50 2004
Subject: [Python-Dev] PEP 0008 confusion - here it is, but don't use it?
In-Reply-To: <16659.54650.788854.375861@montanaro.dyndns.org>
References: <E1Bt9kW-0006Lb-NH@sc8-pr-cvs1.sourceforge.net>
	<16659.54650.788854.375861@montanaro.dyndns.org>
Message-ID: <4113E512.8050407@v.loewis.de>

Skip Montanaro wrote:
>>From Raymond's checkin:
> 
>     Programming Recommendations
>   
>     - Code should be written in a way that does not disadvantage other
>       implementations of Python (PyPy, Jython, IronPython, Pyrex, Psyco, and
>       such).  For example, do not rely on CPython's efficient implementation
>       of in-place string concatenation for statements in the form a+=b or
>       a=a+b.  Those statements run more slowly in Jython.  In performance
>       sensitive parts of the library, the ''.join() form should be used
>       instead.  This will assure that concatenation occurs in linear time
>       across various implementations.
> 
> I'm a bit confused.  Why bother to speed up s+=t if we aren't supposed to
> use it?

Because it helps users who aren't aware of the problem, and use the 
obvious approach. Developers of the standard libraries are supposed to
know, and use the "better, but non-obvious" technique.

Regards,
Martin
From barry at python.org  Fri Aug  6 22:15:47 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug  6 22:15:38 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <4113D5E1.6040006@interlink.com.au>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain> <ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>	<cf0gd1$smv$1@sea.gmane.org>
	<1091818207.7529.34.camel@localhost>
	<4113D5E1.6040006@interlink.com.au>
Message-ID: <1091823347.10212.17.camel@localhost>

On Fri, 2004-08-06 at 15:02, Anthony Baxter wrote:

> To jump on the band-wagon - I can live with | as well. I was going to
> add it to the PEP rewrite, but I'll wait for a decision, to save myself
> the writing ;)

Sure, me too (obviously).  In a perfect world, I'd still prefer @, but
if using | puts this issue to bed, +1 from me.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/762607dd/attachment.pgp
From tim at zope.com  Fri Aug  6 22:30:49 2004
From: tim at zope.com (Tim Peters)
Date: Fri Aug  6 22:31:25 2004
Subject: [Python-Dev] Would like to add Edward Loper as a Python developer
In-Reply-To: <41137F95.4040606@neggie.net>
Message-ID: <20040806203049.AD9843B8038@smtp.zope.com>

[John Belmonte]
> I noticed that the "whole file doctests" item didn't make it to the
> DoctestIdeas page.

There's more on the wiki now, but no real details.  Code for it exists in
the doctest.py now on tim-doctest-branch.

> I was looking for more details about that.

Here's the class documentation.  In context (but not in isolation as here),
it's clear that it creates a unittest.TestSuite:

def DocFileSuite(*paths, **kw):
    """Creates a suite of doctest files.

    One or more text file paths are given as strings.  These should
    use "/" characters to separate path segments.  Paths are relative
    to the directory of the calling module, or relative to the package
    passed as a keyword argument.

    A number of options may be provided as keyword arguments:

    package
      The name of a Python package.  Text-file paths will be
      interpreted relative to the directory containing this package.
      The package may be supplied as a package object or as a dotted
      package name.

    setUp
      The name of a set-up function.  This is called before running the
      tests in each file.

    tearDown
      The name of a tear-down function.  This is called after running the
      tests in each file.

    globs
      A dictionary containing initial global variables for the tests.
    """

For concrete examples, put the Zope3 source code on a wall, and throw darts
at it.

> I've found that for tutorial narratives, I really don't want to bother
> with expected output.

Do you show any output at all?  If not, this isn't for you.

> For complex software, the output may not always be the same anyway.

We want to focus on invariant (guaranteed) behavior in tests.  I think the
same is true of good docs.  If you can't say anything about the output, you
can't test it, and the documentation must be fuzzy too.

> Also, it's not fun to manipulate input code that has prompts and is
> within a multi-line string.

The files DocFileSuite chews on are text files, not necessarily acceptable
input to Python (in fact, they never are in practice).  So the multi-line
string concern doesn't exist in these.  Mucking with prompts is indeed a
price of entry.

> Syntax highlighting is not available, for one thing.

That depends on the editor, but I agree it's at best clumsy to set up.

> I've made my own tutorial generator.  Given this input script:
>
>      ## Double hash comments treated as meta comments and not output to
>      ## the tutorial.
>
>      """
>      Triple-quote comments that start in the first column are treated as
>      tutorial text.
>
>      First we'll import some modules:
>      """
>      import os
>
>      """
>      Some code follows.  In addition to hash comments, and triple-quote
>      comments not starting in the first column are preserved.  Note that
>      code blocks such as functions must end with a blank line, just as
>      is done in the interactive interpreter.
>      """
>
>      def foo():
>          """foo description here"""
>          print 'hello'  # be friendly
>
>      #my_path = 'foo/bar'
>      my_path = os.path.join('foo', 'bar')
>      my_path
>      foo()
>
> the output is:
>
>      Triple-quote comments that start in the first column are treated as
>      tutorial text.
>
>      First we'll import some modules:
>
>          >>> import os
>
>      Some code follows.  In addition to hash comments, and triple-quote
>      comments not starting in the first column are preserved.  Note that
>      code blocks such as functions must end with a blank line, just as
>      is done in the interactive interpreter.
>
>          >>> def foo():
>          ...     """foo description here"""
>          ...     print 'hello'  # be friendly
>          ...
>          >>> #my_path = 'foo/bar'
>          >>> my_path = os.path.join('foo', 'bar')
>          >>> my_path
>          'foo/bar'
>          >>> foo()
>          hello

So if you put your output in a file, you could feed it exactly as-is to
DocFileSuite(), and it would create a unittest for you that verified the
output in your examples is in fact what it claims to be.

From aahz at pythoncraft.com  Fri Aug  6 22:40:36 2004
From: aahz at pythoncraft.com (Aahz)
Date: Fri Aug  6 22:40:38 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <e8bf7a5304080607524990942d@mail.gmail.com>
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org>
	<e8bf7a5304080607524990942d@mail.gmail.com>
Message-ID: <20040806204035.GA18606@panix.com>

On Fri, Aug 06, 2004, Jeremy Hylton wrote:
> On Fri, 06 Aug 2004 07:45:51 -0700, Guido van Rossum <guido@python.org> wrote:
>> [Anthony]
>>>
>>> I'm having trouble finding a single piece from you where you
>>> outline the problems you have with the [decorators] after args,
>>> before :. Could you possibly post a message to python-dev
>>> stating your problems with it? It's the major one I've been
>>> unable to find your objections to.
>> 
>> And nobody else has expressed their objections to it either? :-)
> 
> I thought it was obvious that this option was not acceptable.

It's not obvious to me, but given Guido's concerns, what are the
arguments against

    def foo [deco1(xyz), deco2] (arg1, arg2, *args, **kwargs):

Since all functions MUST have a parameter list, everyone knows to keep
scanning after the decorators.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From martin at v.loewis.de  Fri Aug  6 22:51:53 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug  6 22:51:55 2004
Subject: [Python-Dev] PyRun_ with file name
Message-ID: <4113EF69.2020102@v.loewis.de>

There is a constant issue of mixing CRTs on Windows. Patch 1003535
proposes to work around them by adding yet another PyRun_ function.
I'm in favour, except that:
- it should be called PyRun_Filename
- it should have the maximum current parameter set, i.e. including
   CompilerFlags*

I'm doubtful as to whether specifying an opening mode is necessary.

What do you think?

Martin
From guido at python.org  Fri Aug  6 23:16:22 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug  6 23:16:28 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: Your message of "Fri, 06 Aug 2004 16:40:36 EDT."
	<20040806204035.GA18606@panix.com> 
References: <41136453.1040908@interlink.com.au>
	<200408061445.i76EjqT07026@guido.python.org>
	<e8bf7a5304080607524990942d@mail.gmail.com> 
	<20040806204035.GA18606@panix.com> 
Message-ID: <200408062116.i76LGMO08465@guido.python.org>

> It's not obvious to me, but given Guido's concerns, what are the
> arguments against
> 
>     def foo [deco1(xyz), deco2] (arg1, arg2, *args, **kwargs):

Didn't you see my keynote at PyCon?  I showed all 5 places where
[deco] can be inserted in a function declaration (before the colon),
because all 5 had been proposed, and 4 out of 5 had been rejected by
then.

> Since all functions MUST have a parameter list, everyone knows to keep
> scanning after the decorators.

Pretty much the same arguments apply as with the decorators after the
argument list, plus some people like to grep for the function name
followed by a left paren.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From python at rcn.com  Fri Aug  6 11:17:55 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug  6 23:19:47 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <1091823347.10212.17.camel@localhost>
Message-ID: <019d01c47b96$429cce20$e841fea9@oemcomputer>

> On Fri, 2004-08-06 at 15:02, Anthony Baxter wrote:
> 
> > To jump on the band-wagon - I can live with | as well. I was going
to
> > add it to the PEP rewrite, but I'll wait for a decision, to save
myself
> > the writing ;)
> 
> Sure, me too (obviously).  In a perfect world, I'd still prefer @, but
> if using | puts this issue to bed, +1 from me.

FWIW, +1.


Raymond

From pyth at devel.trillke.net  Sat Aug  7 00:05:42 2004
From: pyth at devel.trillke.net (Holger Krekel)
Date: Sat Aug  7 00:05:45 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <019d01c47b96$429cce20$e841fea9@oemcomputer>
References: <1091823347.10212.17.camel@localhost>
	<019d01c47b96$429cce20$e841fea9@oemcomputer>
Message-ID: <20040806220542.GE5208@solar.trillke>

Raymond Hettinger wrote:
> > On Fri, 2004-08-06 at 15:02, Anthony Baxter wrote:
> > 
> > > To jump on the band-wagon - I can live with | as well. I was going
> to
> > > add it to the PEP rewrite, but I'll wait for a decision, to save
> myself
> > > the writing ;)
> > 
> > Sure, me too (obviously).  In a perfect world, I'd still prefer @, but
> > if using | puts this issue to bed, +1 from me.
> 
> FWIW, +1.

| FWIW
def holger():
    if anything:
        return +1

From jepler at unpythonic.net  Sat Aug  7 00:09:26 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Sat Aug  7 00:09:53 2004
Subject: [Python-Dev] PyRun_ with file name
In-Reply-To: <4113EF69.2020102@v.loewis.de>
References: <4113EF69.2020102@v.loewis.de>
Message-ID: <20040806220926.GA2225@unpythonic.net>

[python.org/sf/1003535]

Another alternative would be to complete the PyFile_* family of APIs
(PyFile_Seek, etc), so that the C runtime routines would never be used.
The "new" function in the PyRun_ family would take a PyFileObject*.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040806/ce72a219/attachment.pgp
From ncoghlan at iinet.net.au  Sat Aug  7 00:55:51 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Aug  7 00:56:09 2004
Subject: [Python-Dev] python-dev Summary for 2004-07-16 through 2004-07-31
	[draft]
In-Reply-To: <4111DC2C.8070302@ocf.berkeley.edu>
References: <4111DC2C.8070302@ocf.berkeley.edu>
Message-ID: <41140C77.1040802@iinet.net.au>

Brett Cannon wrote:

> ------------------------------------------------------------------------------------------------------------------------------------------ 
> 
> How to get Python to compile with Microsoft's free compiler that should 
> just come with the OS standard
> ------------------------------------------------------------------------------------------------------------------------------------------ 
> 
> 1. Download the free .NET compiler
> 2. Download the Windows SDK (at 
> http://www.microsoft.com/msdownload/platformsdk/sdkupdate/ with only IE)
> 3. Download Garth's tools to generate a Makefile from the .sln files (at 
> http://mail.python.org/pipermail/python-dev/2004-February/042595.html )
> 4. Compile
> 5. Realize you should be using an OS that doesn't make you go through 
> this many hoops just to have a compiler for your platform
> 
> Contributing threads:
>   - `Non-Visual Studio builds on Windows XP? 
> <http://mail.python.org/pipermail/python-dev/2004-July/046324.html>`__

Actually, step 4 is more complicated than that ;)

Which was why I abandoned it at that point. . . and then discovered that
  my laptop's modem and built-in wireless don't work properly in Linux,
unless I feel like paying for the Linuxant drivers, or getting coLinux
to work (I'm partway through the latter with Fedora Core 3, since there
won't be Linuxant drivers for Core 3 until it gets released).

So. . .

4a. Get MINGW, since Garth's tool uses make, not nmake. Either that, or
write the nmake makefile generation module for Garth's tool. Guess which
one is likely to be less work ;) (especially since I already had mingw)

4b. Edit the .vcproj files and the .py files in Garth's tool to remove
all references to 'odbc32.lib' and 'odbccp32.lib', since the free tools
don't have those, and Python doesn't seem to actually need them (Visual
Studio links to them by default).

4c. In gnu_make.py, line 49, change it to be:
        write(escape(makedef['all']))
   (The version in the python-dev is missing the escape, which meant
make couldn't match up the targets properly)

4d. Run Garth's tool to generate your makefiles for either debug or release.

4e. Set up a build environment with PATH, INCLUDE and LIB set correctly

4f. Run 'make -f pcbuild.gnu.mak python'

4g. Try to run the tests, and see the following:
C:\DEVELO~1\projects\python\dist\src\PCbuild>python ..\PC\testpy.py
'import site' failed; use -v for traceback
Traceback (most recent call last):
   File "..\PC\testpy.py", line 31, in ?
     import regrtest # Standard Python tester.
   File "C:\DEVELO~1\projects\python\dist\src\lib\test\regrtest.py", line
  126, in ?
     from test import test_support
   File "C:\DEVELO~1\projects\python\dist\src\lib\test\test_support.py",
line 154, in ?
     TESTFN_UNICODE_UNENCODEABLE.encode("Latin1")
   File "C:\development\projects\python\dist\src\lib\encodings\__init__.p
y", line 30, in ?
     import codecs, exceptions, types, aliases
   File "C:\development\projects\python\dist\src\lib\codecs.py", line 76,
  in ?
     class Codec:
   File "C:\development\projects\python\dist\src\lib\codecs.py", line 76,
  in Codec
     class Codec:
SystemError: no locals found when storing '__module__'

Or, in debug:
C:\DEVELO~1\projects\python\dist\src\PCbuild>python_d ..\PC\testpy.py
'import site' failed; use -v for traceback
Traceback (most recent call last):
   File "..\PC\testpy.py", line 31, in ?
     import regrtest # Standard Python tester.
   File "C:\DEVELO~1\projects\python\dist\src\lib\test\regrtest.py", line
  126, in ?
     from test import test_support
   File "C:\DEVELO~1\projects\python\dist\src\lib\test\test_support.py",
line 154, in ?
     TESTFN_UNICODE_UNENCODEABLE.encode("Latin1")
   File "C:\development\projects\python\dist\src\lib\encodings\__init__.p
y", line 30, in ?
     import codecs, exceptions, types, aliases
   File "C:\development\projects\python\dist\src\lib\codecs.py", line 76,
  in ?
     class Codec:
   File "C:\development\projects\python\dist\src\lib\codecs.py", line 76,
  in Codec
     class Codec:
SystemError: no locals found when storing '__module__'
[8229 refs]

Still, even building is progress, right?

Cheers,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA


From tismer at stackless.com  Sat Aug  7 01:04:22 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sat Aug  7 01:01:53 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <4113CD18.8080803@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de>
Message-ID: <41140E76.2050601@stackless.com>

Martin v. L?wis wrote:

> Christian Tismer wrote:
> 
>> the primary purpose of all the decorator syntax effort
>> is to get the
>>
>> foo = classmethod(foo)
>>
>> into the right place, not after but before the definition.
>>
>> Dunno if this was mentioned before, but why not simply do that?
>> Move the statement right before the def and provide a way
>> that this gets recognized correctly.
>>
>> No syntax change, just the convenience people asked for.
> 
> 
> I don't think this can work. How precisely would you implement
> it? Python traditionally executes source code from top to bottom,
> except for loops.

Hey, this is an implementation detail, isn't it? :-)

I can think of a couple of ways to implement it. One would be
to defer execution of a statement with undefined names until
the end of the whole code block in the case of a class
definition and to resolve the names late.
Cleaner would probably be to have something like a placeholder
for the undefined function, assign to the name a special
object and trap the def statement to resolve the object.
(this comes to me since I'm just dealing with reference objects
in PDF files, which are filled in later).

No, I was really not concerned about how to do this, surely
it won't work with plain sequential execution, but other
proposals (which Guido considered once) like just putting
[decorator]
in front of things also needs some (simpler of course) special
handling.

At the risk of getting slogged, I also thought of

myfunc := classmethod(myfunc)

for a short time, with ":=" as a late binding assignment
operator which defers its operation until myfunc is
defined, but I think this is not a popular idea. :-)

To save *some* typing (laziness was also a driver for this
whole decorator nightmare), we could also remove the assignment
and just have a function call like

classmethod(myfunc)

which reduces the re-typing to two.

Anyway, compared to the expected effect, this whole discussion
appears worse to me than the ternary expression thread last
year, where one at least could learn something new.
Fine that a?b:c and friends were put down. I really hate to
see so much time wasted just for decorators. I could live
with a special rule like my little proposal. I also agree
that all the syntax pirouettes tried so far just add more
ugliness to the language, just for nothing.
Using a single char as a prefix might be a compromise
after all. But must it be "@", such an eye-catching, ugly piece
of flie-dirt?
Looks like macro-alert, hey we get into special mode and
decorate tokens in special contexts for special meaning.
This is even worse than what Prothon tries with ? and !.
If @, why not $, this might attract Perl users to Python.

If we are getting perverse, why not simply go and add decorators
using XML syntax?

<decorator kind="classmethod" comment="we deserve what we ask for"/>

I'd-like-to-ask-green-peace-to-protect-my-language -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Sat Aug  7 01:20:53 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sat Aug  7 01:18:30 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <4113D5E1.6040006@interlink.com.au>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<1f7befae040806090644be9c04@mail.gmail.com>	<cf0gd1$smv$1@sea.gmane.org>	<1091818207.7529.34.camel@localhost>
	<4113D5E1.6040006@interlink.com.au>
Message-ID: <41141255.5010408@stackless.com>

Anthony Baxter wrote:

> Mark Russell wrote:
> 
>> I too like |.  It is not quite as visible as @, but I don't think that's
>> a serious problem, and not breaking ipython and other tools is a big
>> win.  I just tried implementing this change, and all the tests still
>> pass (not a big surprise, but worth confirming I guess).
> 
> 
> To jump on the band-wagon - I can live with | as well. I was going to
> add it to the PEP rewrite, but I'll wait for a decision, to save myself
> the writing ;)

Ok, I dislike special prefix chars at all, in a language that
doesn't have this concept elsewhere (despite strings of course,
but their prefixes are just regular chars),
but the bar "|" appears a lot nicer than "@" to me.
The bar is small and friendly, a small character for a small
feature. With "@", I associate a powerful, magical Voodoo
thing, something that I never expected to see in Python.
Unless there is a real need, I'd say save "@" for a feature that
really deserves such a powerful character.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From nhodgson at bigpond.net.au  Sat Aug  7 01:27:02 2004
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Sat Aug  7 01:27:07 2004
Subject: [Python-Dev] 318: Annotation background
Message-ID: <02e801c47c0c$e129eed0$e615a88d@neil>

   It appears from the discussion and PEP that there has not been a lot of
experience here with the Java and C# analogues of decorators.

   In Java, especially server edition features like Entity Java Beans, there
is a requirement for large volumes of supporting files for a system. Some of
these are Java files defining a set of related classes and some are XML
files defining how these can be used. For example, say I have two database
tables, Master and Detail, defined as expected. These will need to be
defined as Java classes, with fields corresponding to the database columns.
Extra fields will be needed for transient attributes, some of which will be
defined by the developer and others by the infrastructure such as dirty
bits, transaction IDs, and remote connection handles. As well as the main
class, there is a need for derived classes such as a remote interface that
looks like the class, a value class that contains the fields (or a subset of
the fields) that can be sensibly copied together to a remote system, and
views of the data that turn relational artifacts like keys into a more
object-oriented form, such as providing a pointer from a Detail to a Master
or having the Detail records appear to be a list on the Master object.

   Maintaining all these files requires far too much work and you will
always be coping with updates that forgot one file. Then along came XDoclet
[1]. XDoclet uses doc comment tags to move the information about associated
source and XML files into the main class file. For example,

/**
 * @ejb.bean
 *   type="CMP"
 *   name="Master"
 */
public abstract class MasterEJB implements EntityBean {
    /**
     * @ejb.value-object
     *   aggregate="example.vo.DetailValue"
     *   aggregate-name="Details"
     *   type="Collection"
     * @ejb.relation
     *   name="Master-Detail"
     */
    public abstract Collection getDetails();
    public abstract void setDetails(Collection details);

   However, comments should be for documentation, not implementation code,
and these 'comments' are now some of the most important and most difficult
to maintain parts of the source. So there was a desire to move this into the
main language where there could be more consistency and earlier checking for
correctness. This has led to JSR 175 [2] which allows @ annotations to be
applied directly to identifiers rather than indirectly through preceding
comments.

[1] http://xdoclet.sourceforge.net/xdoclet/index.html
[2] http://www.jcp.org/aboutJava/communityprocess/review/jsr175/

   Neil

From tismer at stackless.com  Sat Aug  7 01:29:39 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sat Aug  7 01:27:19 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <20040805194523.GA20561@mems-exchange.org>
References: <200408051636.i75Gaak03654@guido.python.org>	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>	<20040805184845.GB29796@burma.localdomain>	<8547DC34-E711-11D8-93D5-000A95686CD8@redivi.com>	<20040805191710.GD29796@burma.localdomain>	<9819B22A-E715-11D8-89DE-000D93AD379E@mac.com>
	<20040805194523.GA20561@mems-exchange.org>
Message-ID: <41141463.9050500@stackless.com>

Neil Schemenauer wrote:

> On Thu, Aug 05, 2004 at 09:28:13PM +0200, Ronald Oussoren wrote:
> 
>>On 5-aug-04, at 21:17, Gustavo Niemeyer wrote:
>>
>>>  def funcname(...):
>>>     ...
>>>  funcname.signature = "v@:@i"
>>
>>That should be workable for this specific example.
> 
> 
> Even nicer if '_' is bound to the last function defined.
> 
>     class SomeClass(objc):
>         def funcname(...):
>            ...
>         _.signature = "v@:@i"

What if, by some special magic, the *next* function defined
were bound to '_' ? This would give us the desired prefix
behavior.

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Sat Aug  7 01:38:30 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sat Aug  7 01:36:04 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>	<20040805184845.GB29796@burma.localdomain>
	<AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <41141676.6060602@stackless.com>

Ronald Oussoren wrote:

...

> Whoops, I used the wrong word, I meant 'generic functions' instead of 
> 'generic example'. He's doing something like this:
> 
> @when("isinstance(db, OracleDB")
> def storeInDB(db, object):
>     pass
> 
> @when("isinstance(db, BerkelyDB")
> def storeInDB(db, object):
>     pass

Why is this needed? Can't we continue to use

if isinstance(db, OracleDB):
   def storeInDB(db, object):
     pass

if isinstance(db, BerkelyDB):
   def storeInDB(db, object):
     pass

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From mkc at mathdogs.com  Sat Aug  7 01:46:27 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Sat Aug  7 01:46:32 2004
Subject: [Python-Dev] Re: pre-PEP: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<2mbrho5o6c.fsf@starship.python.net>
Message-ID: <lkdvffvx224.fsf@panix2.panix.com>

Michael Hudson <mwh@python.net> writes:
> Generally, it strikes me as mildly useful.  An implementation would
> surely help :-)

If yours is the most positive review, there probably won't be one.  :-)

Seriously, though, I'd readily concede that there are different approaches to
solving this problem, and I was hoping for some feedback and suggestions
before I start whacking at the code.  I also have considered the possibility
that although this has been driving me nuts that few others will care--if this
is true, it probably *shouldn't* go in the core libraries.

> Presumably 'structmatch' would also become a metho on pattern objects?

Yes.  Good point.

>> [mismatch index return]
> I don't really like this idea; it seems to be that it would be more
> helpful to raise an exception and attach this data to the exception.
> But that's a bit inconsisent with how match and search work.

Hmm.  Yeah, if it weren't for the value of matching re.match semantics, it
would make a lot more sense for this to be an exception.  As it is, I could
see it going either way.  Or I suppose you could add a flag to determine which
(but ugh).

Thanks for your feedback!
Mike

From pje at telecommunity.com  Sat Aug  7 01:58:46 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat Aug  7 01:54:39 2004
Subject: [Python-Dev] Call for defense of @decorators
In-Reply-To: <41141676.6060602@stackless.com>
References: <AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
	<200408051636.i75Gaak03654@guido.python.org>
	<FFA02177-E70C-11D8-89DE-000D93AD379E@mac.com>
	<20040805184845.GB29796@burma.localdomain>
	<AB9C950A-E712-11D8-89DE-000D93AD379E@mac.com>
Message-ID: <5.1.1.6.0.20040806194231.027540b0@mail.telecommunity.com>

At 01:38 AM 8/7/04 +0200, Christian Tismer wrote:
>Ronald Oussoren wrote:
>
>>Whoops, I used the wrong word, I meant 'generic functions' instead of 
>>'generic example'. He's doing something like this:
>>@when("isinstance(db, OracleDB")
>>def storeInDB(db, object):
>>     pass
>>@when("isinstance(db, BerkelyDB")
>>def storeInDB(db, object):
>>     pass
>
>Why is this needed? Can't we continue to use
>
>if isinstance(db, OracleDB):
>   def storeInDB(db, object):
>     pass
>
>if isinstance(db, BerkelyDB):
>   def storeInDB(db, object):
>     pass

No...  the equivalent code generated is more like:

     def storeInDB(db,object):
         if isinstance(db,OracleDB):
             ...
         elif isinstance(db,BerkleyDB):
             ...

with these important differences:

  * The behavior can be defined modularly, with different modules 
contributing possible execution paths, rather than requiring the function 
body to be defined in a single place.  This makes functions "open" to 
extension, by adding more cases.  For example, one might define a generic 
function to visit various document node types, and then developers who add 
new node types can write additional cases for the existing visitor function.

  * The dispatch algorithm supports multiple dispatch on arbitrary 
conditions on any parameters, even though the example Ronald gave only does 
single-dispatch on one parameter.  'and', 'or', and 'not' can be used to 
construct complex predicates.

  * An arbitrary number of tests may be applied to arbitrary expressions 
involving the parameters, but for each invocation of the function, each 
expression will be computed at most once and each test will be performed at 
most once, no matter how many 'when()' invocations comprise the total 
generic function body.  Conditions which are "less discriminating" will be 
tested after conditions that are "more discriminating", in order to avoid 
computing expressions that may not have any effect on what implementation 
is finally selected.

  * The hierarchy of 'if' tests is automatically generated, based on 
logical implication relationships between the predicates, overlapping of 
ranges, etc.  This removes a lot of tedious reasoning and coding that a 
human programmer would otherwise have to do to ensure a correct decision 
tree when the input is just a bunch of "business rules".

The net effect is to have generic functions in Python, similar to those of 
Lisp or Dylan, but with the addition of arbitrary predicate evaluation, as 
in the research language Cecil.  If you're curious about the basic concept 
and algorithm, see:

     http://citeseer.ist.psu.edu/chambers99efficient.html

and the current Python implementation can be found in the CVS trunk version 
(1.0a0) of PyProtocols.  It does not yet support constrained evaluation 
order, but it does extend the Chambers and Chen algorithm to support 
efficient range and equality comparisons (e.g. for numbers and strings), 
and has other extensions to support dispatching on class and interface 
using Python's particular MRO rules, which are a bit different from those 
of Dylan, Lisp, and Cecil.


From fperez528 at yahoo.com  Sat Aug  7 02:25:47 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Sat Aug  7 02:25:53 2004
Subject: [Python-Dev] IPython, @, and option E from the wiki
Message-ID: <cf17ic$83o$1@sea.gmane.org>

Hi all,

since the developers have been kind enough to take ipython's fate into account
in the decorator debate, I figured I'd post an additional piece of info.

Option E on the Wiki http://www.python.org/moin/PythonDecorators is:

E. pie decorator at top of function body syntax 

def foo(arg1,arg2):
    @classmethod
    ...

def bar(low,high):
    @accepts(int,int)
    @returns(float)
    ...

I just want to mention a couple of things regarding this one, in case it
becomes a contender in Guido's mind:

1. It would NOT cause ipython any problems whatsoever.  Ipython only
special-cases @foo at the beginning of a line, so this would be perfectly OK
with ipython.  I don't know if Leo would fare equally well with it.

2. I happen to like it quite a bit, since I already consider (because of the
special role of docstrings) the top area of a function 'special'.  So that
area seems to me a reasonable choice for additional meta-information about the
function's behavior.  I agree decorators are more radical than docstrings, but
at least there's a certain conceptual precedent.  And it seems to leave the
cleanness of the 'def' word which is a nice visible highlight of a function
declaration, without causing the deep indentation issues of other proposals.

3. In this kind of setup, using | instead of @ would be ok as well, I guess:

def bar(low,high):
    |accepts(int,int)
    |returns(float)
    ...

Ok, nobody really wanted my opinion, so feel free to ignore #2/3 :)  The main
point was #1.

Best,

f

From edloper at gradient.cis.upenn.edu  Sat Aug  7 03:01:47 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Sat Aug  7 03:00:50 2004
Subject: [Python-Dev] pre-PEP: Complete, Structured Regular Expression
	Group Matching
Message-ID: <411429FB.6080603@gradient.cis.upenn.edu>

Michael Hudson wrote:
>> Generally, it strikes me as mildly useful.  An implementation would
>> surely help  :-) 

Mike wrote:
> If yours is the most positive review, there probably won't be one.   :-)

I wouldn't lose heart over the lack of response -- the current flood of 
decorator messages tends to drown out everything else.

I like your idea in the abstract, but I have trouble following many of 
the examples in your pre-pep (and as a result, I'm not sure if I 
*really* understand what you're proposing).  E.g., you have:

|  >>> m1 = re.match(r'("([A-Z]|[a-z])*"\s*)*', '"Xx" "yy" "ZzZ"')
|  >>> m1.group(2)
|  [['X', 'x'], ['yy'], ['ZzZ']]

But it seems to me like the output should instead be:

|  [['X', 'x'], ['y', 'y'], ['Z', 'z', 'Z']]

In particular, group two is "([A-Z]|[a-z])", which matches a single 
character; so it seems like the value of m1.group(2) should be a tree 
with leaves that are single characters.  (Looking at it from another 
angle, I can't see any difference between "Xx" and "ZzZ" that would 
cause one to be decomposed and the other to be left as a string.)

Similar comments apply for many of your other examples.  So either:
   - Your examples are incorrect
   - My understanding of your proposed algorithm is incorrect

If it's the former, then please fix the examples.  If not, then perhaps 
we can talk about your algorithm some more off-list, so I can try to see 
where my misunderstanding is coming from.

-Edward

From python at rcn.com  Fri Aug  6 16:30:10 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sat Aug  7 04:31:53 2004
Subject: [Python-Dev] pre-PEP: Complete,
	Structured Regular Expression Group Matching
In-Reply-To: <lkdekmlvv8k.fsf@panix2.panix.com>
Message-ID: <000f01c47bc1$e1da4820$e841fea9@oemcomputer>

> A notable limitation of the ``re.match`` method is that it fails to
> capture all group match information for repeatedly matched groups.
> For example, in a call like this ::
> 
>     m0 = re.match(r'([A-Z]|[a-z])*', 'XxxxYzz')
> 
> one would like to see that the group which matched four times matched
> the strings ``'X'``, ``'xxx'``, ``'Y'`` and ``'zz'``.  

I suspect there is a typo in the example and that the asterisk should
precede the right paren:

     m0 = re.match(r'([A-Z]|[a-z]*)', 'XxxxYzz')


Raymond

From mkc at mathdogs.com  Sat Aug  7 05:59:04 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Sat Aug  7 05:59:09 2004
Subject: [Python-Dev] Re: pre-PEP: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<000f01c47bc1$e1da4820$e841fea9@oemcomputer>
Message-ID: <lkd657vegzb.fsf@panix2.panix.com>

"Raymond Hettinger" <python@rcn.com> writes:
> > A notable limitation of the ``re.match`` method is that it fails to
> > capture all group match information for repeatedly matched groups.
> > For example, in a call like this ::
> > 
> >     m0 = re.match(r'([A-Z]|[a-z])*', 'XxxxYzz')
> > 
> > one would like to see that the group which matched four times matched
> > the strings ``'X'``, ``'xxx'``, ``'Y'`` and ``'zz'``.  
> 
> I suspect there is a typo in the example and that the asterisk should
> precede the right paren:
> 
>      m0 = re.match(r'([A-Z]|[a-z]*)', 'XxxxYzz')

Oops, you are correct.  In fact, I was thinking of 

       m0 = re.match(r'([A-Z]*|[a-z]*)', 'XxxxYzz')

when I wrote this, but since only single capitals are given in my example, it
could be either.

Fixed.  Thanks!
Mike

From tjreedy at udel.edu  Sat Aug  7 06:12:19 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat Aug  7 06:12:24 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de><20040805215331.GB31288@burma.localdomain><ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost><1f7befae040806090644be9c04@mail.gmail.com>
	<4113AF47.1010405@interlink.com.au>
	<200408061634.i76GYEZ07623@guido.python.org>
Message-ID: <cf1kr3$qe4$1@sea.gmane.org>


"Guido van Rossum" <guido@python.org> wrote in message
news:200408061634.i76GYEZ07623@guido.python.org...
> > - change the language ref that talks about this to state that
> > although Python currently doesn't use these symbols, this is not
> > stopping a future version of Python from using them for some new
> > feature.
>
> Does that really have to be stated?

All that is needed is the addition of 'currently' in the sentence about
illegality.  Compare with "The Python compiler currently generates the
following byte code instructions. " (LibRef 18.10.1) which has the same
word to guard against the same false presumption of stability and
consequent complaints.

Terry J. Reedy



From martin at v.loewis.de  Sat Aug  7 06:19:08 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug  7 06:19:12 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <41140E76.2050601@stackless.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de>
	<41140E76.2050601@stackless.com>
Message-ID: <4114583C.4070500@v.loewis.de>

Christian Tismer wrote:
> I can think of a couple of ways to implement it. One would be
> to defer execution of a statement with undefined names until
> the end of the whole code block in the case of a class
> definition and to resolve the names late.

That would be counter-intuitive. If I have

    foo = func1(foo)
    def foo(self):
      body
    bar = foo

then your proposal is to execute foo=func1(foo) *after*
bar=foo. I doubt most users can understand that easily.

> Cleaner would probably be to have something like a placeholder
> for the undefined function, assign to the name a special
> object and trap the def statement to resolve the object.

That would require a new keyword, right? Which one?

> At the risk of getting slogged, I also thought of
> 
> myfunc := classmethod(myfunc)
> 
> for a short time, with ":=" as a late binding assignment
> operator which defers its operation until myfunc is
> defined, but I think this is not a popular idea. :-)

This is also a syntax change, and one that is going to
break existing tools: it reuses the colon, which many
tools expect to introduce a block of some sort.

> To save *some* typing (laziness was also a driver for this
> whole decorator nightmare), we could also remove the assignment
> and just have a function call like
> 
> classmethod(myfunc)

This breaks backward compatibility. This is already legal
syntax, and should raise a NameError if myfunc is not
defined (actually, this holds for the first proposal, too).
Furthermore, it might be that myfunc *is* defined, so
given

classmethod(myfunc)
def myfunc():
   body1

if some_condition():
   classmethod(myfunc)
   def myfunc():
     body2

you couldn't make the second definition of myfunc a classmethod.

> I could live
> with a special rule like my little proposal. 

Your proposals are all by no means little. They are significant,
counter-intuitive changes of the language.

> But must it be "@", such an eye-catching, ugly piece
> of flie-dirt?

If you don't like the current proposal, try to find objective
reasons, in addition to the subjective ones. Also, try to come
up with counter-proposals that are well thought-out, instead of
being created in a rush. In particular, if all you are concerned
with is the specific choice of operator, propose a different one.

Regards,
Martin
From martin at v.loewis.de  Sat Aug  7 06:23:42 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug  7 06:23:43 2004
Subject: [Python-Dev] python-dev Summary for 2004-07-16 through 2004-07-31
	[draft]
In-Reply-To: <41140C77.1040802@iinet.net.au>
References: <4111DC2C.8070302@ocf.berkeley.edu> <41140C77.1040802@iinet.net.au>
Message-ID: <4114594E.2030606@v.loewis.de>

Nick Coghlan wrote:
> 4b. Edit the .vcproj files and the .py files in Garth's tool to remove
> all references to 'odbc32.lib' and 'odbccp32.lib', since the free tools
> don't have those, and Python doesn't seem to actually need them (Visual
> Studio links to them by default).
> 
> 4c. In gnu_make.py, line 49, change it to be:
>        write(escape(makedef['all']))
>   (The version in the python-dev is missing the escape, which meant
> make couldn't match up the targets properly)

Feel free to contribute patches. I think I already said that I see no
problem with the Python script being distributed along with Python,
provided its author(s) are willing to contribute it (it would be good
if there was a single dedicated contact for maintaining it, though).

For the odbc references, please contribute patches to take the
references out of the projects. We really don't need odbc in Python.

Regards,
Martin
From martin at v.loewis.de  Sat Aug  7 06:28:46 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug  7 06:28:48 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <41141255.5010408@stackless.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<1f7befae040806090644be9c04@mail.gmail.com>	<cf0gd1$smv$1@sea.gmane.org>	<1091818207.7529.34.camel@localhost>	<4113D5E1.6040006@interlink.com.au>
	<41141255.5010408@stackless.com>
Message-ID: <41145A7E.8010500@v.loewis.de>

Christian Tismer wrote:
> Ok, I dislike special prefix chars at all, in a language that
> doesn't have this concept elsewhere (despite strings of course,
> but their prefixes are just regular chars),

Lists, tuples, and dictionaries are also introduced with non-letter
characters. Furthermore, nested expressions use the same prefix
char that lists use. # introduces comments.

Regards,
Martin
From edloper at gradient.cis.upenn.edu  Sat Aug  7 06:35:17 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Sat Aug  7 06:34:14 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
Message-ID: <41145C05.1020501@gradient.cis.upenn.edu>

The new "vertical bar" syntax was proposed recently, and has gotten
almost unanimously positive feedback.  In particular, of 17 responses
to the syntax, only one has been negative (see table, below).

I'm including both of the following variants:

     |classmethod
     |accepts(int, int)
     |returns(float)
     def foo(arg1, arg2):
         ...

     def foo(arg1, arg2):
         |classmethod
         |accepts(int, int)
         |returns(float)
         ...

(I prefer the second, but would be happy with either.)

The one negative comment was that "|" can look similar to "I" or "l",
depending on the font; but it's only an issue for some fonts, and it
should be largely mitigated by syntax-highlighting.

Guido's response to the vertical bar syntax is:
 > If the community can rally behind one of these, I think that would
 > be acceptable.  They all seem arbitrary, but so is the choice of
 > '@'. :-)
<http://mail.python.org/pipermail/python-dev//2004-August/047143.html>

So I wanted to see if anyone has anything *negative* to say about this
syntax (or one of its variants).  If possible, please respond by
updating the wiki, rather than by email.  This syntax is listed under
H (pre-def) and E2 (post-def).  You will need to sign in before you
can edit the wiki.

     Wiki: <http://www.python.org/cgi-bin/moinmoin/PythonDecorators>

(Feel free to update the wiki with positives, too, but I'm mainly trying 
to get a feel for whether anyone is strongly opposed to this syntax.)

-Edward

======================================================================
                  The Vertical Bar Decorator Syntax
======================================================================
History:                                             URL [1]
   First proposed (by Shalabh Chaturvedi) .../2004-August/047005.html
   Nested version proposed (by IxokaI)    .../2004-August/047170.html
   All tests pass if @ changed to |       .../2004-August/047189.html

Responses (sorted):                                  URL [1]
   +1    Shalabh Charturvedi              .../2004-August/047005.html
   +1    Richie Hindle                    .../2004-August/047078.html
   +1    Barry Warsaw                     .../2004-August/047133.html
                                     also .../2004-August/047207.html
   +1    Tim Peters                       .../2004-August/047147.html
   +1    Johannes Gijsbers                .../2004-August/047162.html
   +1    IxokaI                           .../2004-August/047170.html
   +1    Fernando Perez                   .../2004-August/047176.html
                                     also .../2004-August/047223.html
   +1    Mark Russell                     .../2004-August/047189.html
   +1    Anthony Baxter                   .../2004-August/047195.html
   +1    Raymond Hettinger                .../2004-August/047212.html
   +1    Edward Loper                            (this email)
   +1?   Holger Krekel                    .../2004-August/047213.html
   +0.5  Timothy Delaney                  .../2004-August/047007.html
   +0    GvR                              .../2004-August/047143.html
   +0?   Skip Montanaro                   .../2004-August/047192.html
   +0    Christian Tismer                 .../2004-August/047217.html
   -1    Bob Ippolito                     .../2004-August/047200.html

[1] All URLs are written in condensed form, 'cuz this email is too
     long already; to get a real URL, replace the "..." with
     "http://mail.python.org/pipermail/python-dev".

From tjreedy at udel.edu  Sat Aug  7 06:34:13 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat Aug  7 06:34:20 2004
Subject: [Python-Dev] Re: PEP 0008 confusion - here it is, but don't use it?
References: <E1Bt9kW-0006Lb-NH@sc8-pr-cvs1.sourceforge.net><16659.54650.788854.375861@montanaro.dyndns.org>
	<4113E512.8050407@v.loewis.de>
Message-ID: <cf1m45$sbg$1@sea.gmane.org>


"Martin v. Löwis" <martin@v.loewis.de> wrote in message
news:4113E512.8050407@v.loewis.de...
> > I'm a bit confused.  Why bother to speed up s+=t if we aren't supposed
to
> > use it?
>
> Because it helps users who aren't aware of the problem, and use the
> obvious approach. Developers of the standard libraries are supposed to
> know, and use the "better, but non-obvious" technique.

Also, users are freer to discard cross implementation portability than
library.  Dependence on ref counting to close files is another example.

tjr



From pythondev at bitfurnace.com  Sat Aug  7 06:40:44 2004
From: pythondev at bitfurnace.com (damien morton)
Date: Sat Aug  7 06:44:32 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <4114583C.4070500@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<200408061557.i76FvGo07481@guido.python.org>	<4113BC98.3020407@stackless.com>
	<4113CD18.8080803@v.loewis.de>	<41140E76.2050601@stackless.com>
	<4114583C.4070500@v.loewis.de>
Message-ID: <41145D4C.7040200@bitfurnace.com>

Even though Im basically in favour of the C# style syntax, and mostly 
because I have been using C# a lot and find no problems recognising the 
syntax when I see it, I also find the "decorator on the line after the 
def" syntax appealing.

Allow me to throw this syntax into the ring:

def foo(arg1,arg2):
    \(classmethod)
    blah
    ...

def bar(low,high):
    \(accepts(int,int), returns(float))
    blah
    ...

Allow me to also throw a possible variant

def foo(arg1,arg2): \(classmethod)
    blah
    ...

def bar(low,high): \(accepts(int,int), returns(float))
    blah
    ...

I strongly feel that any decorator syntax will need some kind of matched 
pair to delimit the decorator(s). Angle-brackets (single or double) seem 
very appealing

<classmethod>
def foo(arg1,arg2):
    blah
    ...
<accepts(int,int), returns(float)>
def bar(low,high):
    blah
    ...


From python-kbutler at sabaydi.com  Sat Aug  7 06:55:35 2004
From: python-kbutler at sabaydi.com (Kevin J. Butler)
Date: Sat Aug  7 06:55:42 2004
Subject: [Python-Dev] Suggesting '.' decorators (PEP318)
Message-ID: <411460C7.4030708@sabaydi.com>

Many people wrote:
    > Lots of options

I haven't seen this mentioned, so I'm going to throw in my suggestion.

The following makes sense to me, uses previously invalid syntax, and
uses the '.' punctuation in a way similar to existing/proposed usage:

       def longMethodNameForEffect(longArgumentOne=None,
                                   longArgumentTwo=42):
           .staticmethod
           .funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
                      status="experimental", author="BDFL")
           """
           asdfasdf
           """
           raise NotYetImplemented

If we want to allow one-liner decorators (questionable to me):

       def myMethod(self): .staticmethod
           raise NotYetImplemented

Advantages/disadvantages of positioning at start of block:

+ Unambiguous "target" for decorators, matches Python's precedents for
indentation and modify-what-came-before

+ Decorators won't get lost in long argument lists (they are indented
differently in Guido's preferred formatting, and are separated by the
smiley in the all-args-on-individual-lines formatting)

+ Separate from def syntax (to keep def the same avoiding breakage)

0 Perhaps decorators should be allowed before or after the docstring. If
you have to choose, I'd choose making it before the docstring.

+ No extra cut-and-paste issues, no extra indentation level.

+ No smileys.  (Does that get a :)  or a  ]:  ?)

Advantages/disadvantages of .decorators:

+ Will not be silently ignored (doesn't use currently legal syntax)

+ Simple for simple decorators, supports complex decorators as cleanly
as possible

+ Less ugly (YMMV). Doesn't feel like executable line noise (to 1 out of
1 pythonistas polled...)

+ No new punctuation or keywords (avoids breaking Leo/IPython/existing code)

+ Really nice syntax for assigning attributes, if desired (Not sure I
like this overloading, but it /looks/ good)

       def func():
           .author = "Kevin Butler"
           pass

+/0 Syntax obvious visually (Someone will complain that the leading
period gets lost - that person should switch to a fixed-width font, as
used when coding.  <.5 wink>) Easy to highlight.

0 Although it is a punctuation-based syntax, it is compatible with
existing/proposed '.' usage ('.' can mean subordination to a containing
construct like 'with', and passing the implicit "func" argument is
analogous to passing "self")

0 Compatible with some future "with ...:" syntax, as decorators must
immediately follow a 'def ...:' (or possibly a 'class ...:'), so if
there is a 'with ...:' what follows cannot be a decorator.

- Minor extension to use of '.'

- Some people may have too much dust on their monitors

Misc items from the PEP applied to .decorators:

+ One decorator per line

+ Could use ".doc()" or ".doc = " as a docstring alternative

?+ Compile time (? as much as the @ syntax is?)

+ Works in interactive mode

Public or private feedback appreciated, I will summarize private
feedback to the list.

kb
From nbastin at opnet.com  Sat Aug  7 06:56:34 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Sat Aug  7 06:58:33 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu>
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <27F1C2B9-E82E-11D8-BC7A-000D932927FE@opnet.com>


On Aug 7, 2004, at 12:35 AM, Edward Loper wrote:

> The new "vertical bar" syntax was proposed recently, and has gotten
> almost unanimously positive feedback.  In particular, of 17 responses
> to the syntax, only one has been negative (see table, below).
>
> I'm including both of the following variants:
>
>     |classmethod
>     |accepts(int, int)
>     |returns(float)
>     def foo(arg1, arg2):
>         ...
>
>     def foo(arg1, arg2):
>         |classmethod
>         |accepts(int, int)
>         |returns(float)

I have to admit that at first look, the decorator-after-def syntax 
wasn't very appealing.  However, although it's not how other languages 
generally do it (does anybody know of one?), it might be easier for new 
users to understand.  i.e., I'm decorating 'this' thing, rather than 
the 'next' thing.

Does anybody suggest how this syntax plays with docstrings?  Are 
decorators after the doc string, or before, or doesn't it matter?  My 
vote would be for after the doc string, but I thought I'd throw that 
out there.  It's also entirely possible that no one else likes the 
after-def syntax, so maybe this isn't a bridge we'll have to cross.. 
:-)

As for pipe versus @, I couldn't really care less, since my syntax 
highlighter will take care of it, but for new users I think '@' stands 
out much better.  On the other hand, we might want to reserve '@' for 
some future use more aligned to its' usual meaning.  Heck, I'm 
surprised that any rational discussion is happening at all at this 
point.

Seriously, we've been discussing this for longer than I care to think 
about, and it's obvious that we're not going to get something that 
everyone agrees upon.  If there were something that everyone could 
agree upon, we'd already have found it - it's not like a pile of new 
discussion is likely to change that, so if we stick with 
'@'-before-def, that's fine by me too.

--
Nick

From mkc at mathdogs.com  Sat Aug  7 07:03:30 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Sat Aug  7 07:03:35 2004
Subject: [Python-Dev] Re: pre-PEP: Complete,
	Structured Regular Expression Group Matching
References: <411429FB.6080603@gradient.cis.upenn.edu>
Message-ID: <lkd1xijedzx.fsf@panix2.panix.com>

Edward Loper <edloper@gradient.cis.upenn.edu> writes:
> I wouldn't lose heart over the lack of response -- the current flood of
> decorator messages tends to drown out everything else.

No, not at all.  I was hoping I might benefit from @-fatigue, though.  :-)

> Similar comments apply for many of your other examples.  So either:
>    - Your examples are incorrect
>    - My understanding of your proposed algorithm is incorrect

It's the former, unfortunately--sorry about that.  I will post a corrected
version shortly.

Thanks for your feedback,
Mike

From tjreedy at udel.edu  Sat Aug  7 07:05:55 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat Aug  7 07:06:00 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <cf1nvj$uk0$1@sea.gmane.org>


"Edward Loper" <edloper@gradient.cis.upenn.edu> wrote in message
news:41145C05.1020501@gradient.cis.upenn.edu...
> The new "vertical bar" syntax was proposed recently, and has gotten
> almost unanimously positive feedback.  In particular, of 17 responses
> to the syntax, only one has been negative (see table, below).

Add another +1.  Verticle rules have been used before to tie several lines
together, either as continuation lines or as sub-lines.

>      |classmethod
>      |accepts(int, int)
>      |returns(float)
>      def foo(arg1, arg2):
>          ...
>
>      def foo(arg1, arg2):
>          |classmethod
>          |accepts(int, int)
>          |returns(float)
>          ...
> (I prefer the second, but would be happy with either.)

Both logically and visually, I prefer the second also.  By logically, I
mean that it is more meaningful to me to read the decorators after reading
that we are defining a function.

My main objection is that I like the idea that the header is definition
time info and the body is run time code.  I consider the doc string in the
body to be a wart, not a precedence to by copied, but maybe I should think
of the body as starting after the last | or the closing ''' :-).

> The one negative comment was that "|" can look similar to "I" or "l",
> depending on the font; but it's only an issue for some fonts, and it
> should be largely mitigated by syntax-highlighting.

Is a space allowed after the prefic character, whatever it might be?  With

def foo():
   | classmethod
   | accepts(int,int)
   | returns(float)

the visual similarity to I doesn't matter much.  Actual typos leading to
test such as 'I returns' are guaranteed syntax errors, whereas a name such
as Ireturns is not.  The space even more clearly matches standard usage of
a vertical rule and more clearly denotes the list as hanging off of 'def',
so I would be tempted to require the space.

Terry J. Reedy



From mkc at mathdogs.com  Sat Aug  7 07:08:08 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Sat Aug  7 07:10:42 2004
Subject: [Python-Dev] pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com>
Message-ID: <lkdwu0bcz7r.fsf@panix2.panix.com>

The first version contained several errors.  Here is a new version with
corrections.  It also incorporates the feedback received so far.

Mike

##############################################################################
PEP: XXX
Title: Complete, Structured Regular Expression Group Matching
Version: $Revision: 1.4 $
Last-Modified: $Date: 2004/08/07 04:57:57 $
Author: Mike Coleman <mkc@mathdogs.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 1-Aug-2004
Python-Version: 2.4
Post-History: 1-Aug-2004


Abstract
========

This proposal is to add a new method ``re.structmatch`` that fully
captures matches for groups that match repeatedly.  The method returns
a structured match tree whose shape is determined by the pattern
groups.  Ultimately this will allow a string that can be described by
a Python regular expressions (e.g., the contents of ``/etc/passwd`` or
``.ini`` files, or the output of ``diff``) to be parsed into the
obvious parse tree with a single call to ``re.structmatch``.


Motivation
==========

A notable limitation of the ``re.match`` method is that it fails to
capture all group match information for repeatedly matched groups.
For example, in a call like this ::

    m0 = re.match(r'([A-Z]+|[a-z]+)*', 'XxxxYzz')

one would like to see that the group which matched four times matched
the strings ``'X'``, ``'xxx'``, ``'Y'`` and ``'zz'``.  In the current
implementation, only the last match for each group (``'zz'`` in this
case) is available, even though the other matches are calculated
implicitly during the matching process.  For simple cases, the missing
strings might be easily recovered by other means, but for more
complicated cases this is a significant annoyance.

The simplest extension would be to collect all of the matches for each
group in a list, so that in the call above for example ::

    m0.group(1) --> ['X', 'xxx', 'Y', 'zz']

(A mechanism like this is apparently to be added to Perl in version 6
[#PERL6]_, though that did not inspire this PEP.)

This immediately suggests the question of what is to be done for
nested repeats, like so ::

    m1 = re.match(r'("([A-Z]+|[a-z]+)*"\s*)*', '"Xx" "yy" "ZzZ"')

We could have ::

    m1.group(2) --> ['X', 'x', 'yy', 'Z', 'z', 'Z' ]

but this flattened result would discard useful information about the
structure of the match.  A more useful result would be ::

    m1.group(2) --> [['X', 'x'], ['yy'], ['Z', 'z', 'Z']]

This is definitely an improvement.  Consider the following slightly
more complicated case, though ::

    m1 = re.match(r'("([A-Z]+|[a-z]+)*"(\s*))*', '"Xx" "yy" "ZzZ"')

which would give ::

    m1.group(2) --> [['X', 'x'], ['yy'], ['Z', 'z', 'Z']]
    m1.group(3) --> [' ', ' ', '']

This is less useful than the putative conceptual structure of the
match, which would be something like ::

    [[['X', 'x'], ' '],
     [['yy'], ' '], 
     [['Z', 'z', 'Z'], '']]

In this simple case, this structure could be recovered using the
``zip`` function, but in the general case this is a non-trivial
transformation.

This PEP proposes a mechanism to address the general case directly.


Proposal
========

The new function ``re.structmatch`` has the same arguments as
``re.match`` and will match using exactly the same algorithm.  Instead
of returning a MatchObject upon success, ``re.structmatch`` will
return a "tree" (or rather, a forest) in the form of a Python list.

First we will define some terms.  Groups in the input pattern are
parenthesized subexpressions that "capture" strings when matched, and
are of the form ``'(abc)'`` or ``'(?P<name>abc)'``.  Leaf groups are
groups that do not contain any subgroups.  So, for example, the group
``'(abc)'`` is a leaf group and the group ``'(a(xyz)b)'`` is not a
leaf group (but it does contain the leaf group ``'(xyz)'``).  We shall
call parenthesized expressions that are not groups "nongroups"; these
include constructs like ``'(?:abc)'`` as well as lookahead and
lookbehind assertions.

The structure of the returned tree is determined from the grouping
tree present in the pattern in the following manner:

* Leaf groups not followed immediately by a repeat operator map to a
  single string::

    re.structmatch(r'(...)', 'abcdef') --> ['abc']
    re.structmatch(r'(...).(..)', 'abcdef') --> ['abc', 'ef']

* Leaf groups followed immediately by a repeat operator map to a list
  of strings::

    re.structmatch(r'([^d])*', 'abcdef') --> [['a', 'b', 'c']]
    re.structmatch(r'([^d])*(.)*', 'abcdef')
        --> [['a', 'b', 'c'], ['d', 'e', 'f']]
    re.structmatch(r'(..)*', 'abcdef') --> [['ab', 'cd', 'ef']]
    re.structmatch(r'(.){2}', 'abcdef') --> [['a', 'b']]

* Non-leaf groups not followed immediately by a repeat operator map to
  a list of the mappings of their subgroups::

    re.structmatch(r'(...)', 'abcdef') --> ['abc']
    re.structmatch(r'((...))', 'abcdef') --> [['abc']]
    re.structmatch(r'(((...)))', 'abcdef') --> [[['abc']]]
    re.structmatch(r'((...)())', 'abcdef') --> [['abc'], []]
    re.structmatch(r'(.(.)(.(.)).(.))', 'abcdef')
       --> ['a', ['b'], ['c', ['d']], 'e', ['f']]

* Non-leaf groups followed immediately by a repeat operator map to a
  list of the mappings of their repeated matches::

    re.structmatch(r'((.).(.))*', 'abcdef') --> [[['a', 'c'], ['d', 'f']]]
    re.structmatch(r'(([ab])*(x)*)*', 'baxbxx')
       --> [[['b', 'a'], ['x']], [['b'], ['x', 'x']]]

* In the case of alternation, only the matched groups appear in the
  output::

    re.structmatch(r'([^a])+|([^d])+', 'abcdef') --> [['a', 'b', 'c']]

  If it's important to know which alternative matched, named groups
  can be used.

* Named groups map to a pair where the first member is the name and
  the second member is what the unnamed group would have mapped to::

    re.structmatch(r'([^d])*(?P<rest>.)*', 'abcdef')
        --> [['a', 'b', 'c'], ('rest', ['d', 'e', 'f'])]

* Nongroups do not affect the output structure.  Compared to non-leaf
  groups, nongroups have the effect of "flattening" the output, like
  so::

    re.structmatch(r'((.).(.))', 'abcdef') --> [['a', 'c']]
    re.structmatch(r'(.).(.)', 'abcdef') --> ['a', 'c']
    re.structmatch(r'(?:(.).(.))', 'abcdef') --> ['a', 'c']

    re.structmatch(r'((.).(.))*', 'abcdef') 
        --> [[['a', 'c'], ['d', 'f']]]
    re.structmatch(r'(?:(.).(.))*', 'abcdef') 
        --> [['a', 'c', 'd', 'f']]

  (Nongroups that do not contain leaf groups have no effect whatsoever
  on the output structure.)


Additional Features
-------------------

By analogy with ``'re.match'``, regular expression objects will also
have a ``'structmatch'`` method, with the same signature as the
``'match'`` method.

In many cases in which ``'re.structmatch'`` fails to match, the cause will
be due to an unexpected error in the format of the string being
matched.  In order to assist the calling program in generating a more
meaningful possible error message, ``'re.structmatch'`` will return the
endpoint of the largest match against the searched string.  So, for
example ::

        re.structmatch('abcd', 'abxxx') --> 2
        re.structmatch('abcde|z', 'abxxx') --> 2
        re.structmatch('z|abcde', 'abxxx') --> 2
        re.structmatch('x*?y', 'xxx') --> 3

(This return value should be considered advisory rather than exact, as
future improvements in the match algorithm may make it difficult to
calculate the exact value.)

An alternative suggested by Michael Hudson would be to throw an
exception in this case.  This makes more sense, but would have the
disadvantage of diverging from the behavior of ``'re.match'``.


Examples
========

Parsing ``/etc/group``
----------------------

If ``/etc/group`` contains the following lines ::

    root:x:0:
    daemon:x:1:
    bin:x:2:
    sys:x:3:

then it can be parsed as follows ::

    p = r'''(?xm)           # VERBOSE, MULTILINE
            (
              (?:
                # one field, preceded by a ':' if
                # the field is not the line's first
                (?:^|:) ([^:\n]*)
              )*
              \n
            )*
            \Z              # assert that the entire
                            #   input was matched
         '''
    s = open('/etc/group').read()
    tree = re.structmatch(p, s)

which will give ``tree`` the following value::

    [['root', 'x', '0', ''],
     ['daemon', 'x', '1', ''],
     ['bin', 'x', '2', ''],
     ['sys', 'x', '3', '']]

Note that the above pattern is written to allow any ``/etc/group``
field to be empty.  The pattern won't work correctly for versions of
Python prior to 2.4 because of the possibility of repeating empty
matches.  This problem seems to have been fixed in Python 2.4.  (A
slightly more complicated pattern would work for pre-2.4 versions.)


Parsing ``.ini`` files
----------------------

The Microsoft ``.ini`` file format is found in various contexts (there
is one in the Python source distribution:
``Tools/freeze/extensions_win32.ini``).  Given a file with contents ::

    [singers]
    good=Annie Lennox
    bad=William Shatner

    [comedians]
    good=Monty Python

the file can be parsed with pattern ::

    r'''(?xm)           # VERBOSE, MULTILINE
        \s*             # leading whitespace
        (               # begin sections
          ^\[ ([^]]+) \] \s*  # section header
          (             # begin params
            ^([^=]+) =  # param name
            (.*) $      # param value
            \s*         # intervening whitespace
          )*            # end params
        )*              # end sections
        \Z              # assert that the entire
                        #   input was matched
     '''

to give ::

    [['singers',
      ['good', 'Annie Lennox'],
      ['bad', 'William Shatner']],
     ['comedians',
      ['good', 'Monty Python']]]

The pattern given omits support for ``.ini`` file comments for the
sake of simplicity, but this could be added.


Details
=======

The proposal states that ``re.structmatch`` will match using exactly
the same algorithm as ``re.match``.  This might be a little odd for a
pattern like ``r'(x|y|z)*aaa\1'``, where the algorithm will require
that the ``\1`` back-reference match (at most) one character.  It's
not obvious whether there is any better option, though, and there are
advantages of implementation and simplicity for keeping the match
algorithms of these two methods identical.  (A similar argument
applies to ``'(?P=NAME)'``.)


Discussion
==========

Part of the reason the proposed method would be so useful is that
``re.split`` currently does not split on empty matches.  If it had
this feature, one could split on lookahead and lookbehind assertions,
which would significantly ease parsing of strings with a recursive
regular structure (e.g., ``.ini`` files).  Patch `#988761`_ will
correct this ``re.split`` deficiency, if it is accepted.

.. _#988761: https://sourceforge.net/tracker/?func=detail&aid=988761&group_id=5470&atid=305470


For particularly complex patterns, the current 99 group limit might
actually become a practical problem.

One could imagine a variation in which subtrees of named groups might
be captured in dictionaries rather than lists, with the group names
used as keys.



Rejected Alternatives
=====================

Several simpler alternatives are rejected in the `Motivation`_ section
above.  Although these alternatives would be better than nothing, they
would not adequately satisfy the goal of this proposal, which is to
allow the programmer to extract the structure of a string in an
immediate manner.

It would be possible to use tuples for some substructures in the
return tree, rather than composing it strictly of lists.  This
alternative was rejected because it was thought useful to be able to
modify the resulting tree in place, perhaps to add decorations, etc.,
and tuples would make this more difficult.


References
==========

.. [#PERL6] Multiple regex matches in Perl Apocalypse 5
   (http://www.perl.com/pub/a/2002/06/04/apo5.html?page=22#rfc%20360:%20allow%20multiply%20matched%20groups%20in%20regexes%20to%20return%20a%20listref%20of%20all%20matches)


Acknowledgements
================

Thanks to Raymond Hettinger, Michael Hudson and Edward Loper for
feedback and improvements.


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:

From bob at redivi.com  Sat Aug  7 07:14:39 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sat Aug  7 07:14:50 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu>
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <AED65FB6-E830-11D8-B298-000A95686CD8@redivi.com>


On Aug 7, 2004, at 12:35 AM, Edward Loper wrote:

> The new "vertical bar" syntax was proposed recently, and has gotten
> almost unanimously positive feedback.  In particular, of 17 responses
> to the syntax, only one has been negative (see table, below).
---
> The one negative comment was that "|" can look similar to "I" or "l",
> depending on the font; but it's only an issue for some fonts, and it
> should be largely mitigated by syntax-highlighting.
>   -1    Bob Ippolito                     .../2004-August/047200.html

I'm actually -0, it doesn't really bother me, but I saw the potential 
issue immediately.  When the | is viewed with common fonts in email, 
web pages, or maybe even print (if done semi-carelessly) it has the 
potential to be confusing.

-bob
From nhodgson at bigpond.net.au  Sat Aug  7 08:10:54 2004
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Sat Aug  7 08:12:24 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <041501c47c45$54e3f630$e615a88d@neil>

Edward Loper:


> The new "vertical bar" syntax was proposed recently, and has gotten
> almost unanimously positive feedback.  In particular, of 17 responses
> to the syntax, only one has been negative (see table, below).
> ...
> So I wanted to see if anyone has anything *negative* to say about this
> syntax (or one of its variants).

   One advantage of '@' is that, since it is currently not found in Python
code, it is a very strong escape that can be inserted anywhere. For example,
if there is later a need or desire to annotate function arguments
individually as well as functions, or individual statements or expressions,
then @ is a better choice. For example

@RemoteCall
def Ignite(engine @Remote, environment @Copy, thrust @Float):

   may be better at associating annotations more closely with their targets
than

@RemoteCall
@Types(["p", "p", "f"])
@Copy("environment")
def Ignite(engine, environment, thrust):

   Neil

From nhodgson at bigpond.net.au  Sat Aug  7 08:10:54 2004
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Sat Aug  7 08:13:05 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <041901c47c45$95c30470$e615a88d@neil>

Edward Loper:


> The new "vertical bar" syntax was proposed recently, and has gotten
> almost unanimously positive feedback.  In particular, of 17 responses
> to the syntax, only one has been negative (see table, below).
> ...
> So I wanted to see if anyone has anything *negative* to say about this
> syntax (or one of its variants).

   One advantage of '@' is that, since it is currently not found in Python
code, it is a very strong escape that can be inserted anywhere. For example,
if there is later a need or desire to annotate function arguments
individually as well as functions, or individual statements or expressions,
then @ is a better choice. For example

@RemoteCall
def Ignite(engine @Remote, environment @Copy, thrust @Float):

   may be better at associating annotations more closely with their targets
than

@RemoteCall
@Types(["p", "p", "f"])
@Copy("environment")
def Ignite(engine, environment, thrust):

   Neil

From tim.peters at gmail.com  Sat Aug  7 08:27:20 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 08:27:29 2004
Subject: [Python-Dev] RELEASED Python 2.4, alpha 2
In-Reply-To: <411274C0.4070300@v.loewis.de>
References: <411231C8.3020308@interlink.com.au> <411241F5.3080602@v.loewis.de>
	<1f7befae040805103911610582@mail.gmail.com>
	<411274C0.4070300@v.loewis.de>
Message-ID: <1f7befae04080623276089908e@mail.gmail.com>

[Tim, tries the "upgrade" install for 2.4a2, notes that it's pretty slow]

[Martin v. L?wis]
> Yes. I should probably speed it up by changing the UI sequence number
> of RemoveExistingApplications from 1510 to 1450. In case you wonder
> what this means:

It's obvious:  you would add 3000, then cyclically permute the first 3
digits right one position.  Tell me something I don't know <wink>.

> Action 1500 is InstallInitialize; anything from then
> on is under transaction control. This means that installer will roll
> back the uninstall in case the subsequent install fails. This is
> expensive, as it first moves all old files out of the place, then puts
> the new ones in, then deletes the old ones. At 1450, the files would
> be deleted immediately (I believe), at the cost of not being able to
> undo the deinstallation if the installation later fails (e.g. by
> user Cancel).

I wouldn't do this; it's good that as much of the process as possible
is transactional.  I doubt it took longer overall than what I normally
do (run a full uninstall of the old version, then install the new
version).

>> One glitch:  I'm pretty sure I selected "install only for me" in the
>> first screen.  I wanted to go back to check, but it doesn't seem
>> possible to go back to the first screen.
 
> Yes. I don't know what to do about this: the screen sets ALLUSERS=1,
> then performs FindRelatedProducts to find the old installation. If the
> old installation was per-machine, we only find it if we commit to making
> the new installation also per-machine.

Except that in this case, I'm pretty sure my 2.4a1 installation was
also "only for me".  I could be wrong about that.

> That decision cannot be taken back; the user would have to cancel and start
> over.

Which is an option, so that's cool.

>> Anyway, the Python DLL ended up in a system directory, and that's "the
>> glitch".  I expected/wanted it to end up in my Python directory.

> Ok. I will see whether I can reproduce that.

I believe I later saw a checkin that intended to fix this, and, if so,
that's cool too.  I hereby nominate you as MSI-builder For Life
<wink>.
From tim.peters at gmail.com  Sat Aug  7 08:54:35 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 08:54:38 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <f8ef9ae40408052303384b55c@mail.gmail.com>
References: <200408060207.i7627gmi000871@cosc353.cosc.canterbury.ac.nz>
	<e443ad0e04080522023e4029e6@mail.gmail.com>
	<1f7befae04080522314d818d8f@mail.gmail.com>
	<f8ef9ae40408052303384b55c@mail.gmail.com>
Message-ID: <1f7befae040806235454e01df9@mail.gmail.com>

[ixokai@gmail.com]
>  Doesn't the fact that the @stuff is at the same level and has no
> connection to the def besides 'above' bother you in our universe of
> indentation-love?

No, it doesn't.  I routinely put comment blocks before my functions,
"at the same level", and they're strongly connected to the function in
reality if not by syntax rule.  I *like* them at that level too,
because they're usually about the external behavior of the function,
not about its internals.  Decorators are also more about externals
than internals, so it feels right to me to see them at the same level
as the function's comment block.  Guido agrees, although he doesn't
yet realize it <wink>, as evidenced by his earlier comment that
docstrings probably would have been better placed before defs too. 
Comment blocks, docstrings, and decorators are all about what the
function is like "from the outside".  If Python supported design by
contract natively, I'd also like to see method pre- and
post-conditions at this level.

>  And the fact that the pie *is* "at", and why that it just doesn't
> "read"? '"define methy1 at staticmethod" is wrong, but "at
> staticmethod define methy2" is right.

When I read

# Here's a comment.

I don't verbalize it as "splat here's a comment".  I don't feel a need
to verbalize "@" either.

> I like Python to be able to be read very easily.

I care about readability, but little about pronouncability.

> If we have to have it at the same level as def and above, we soo need a
> keyword..pweeze?

Pick the keyword you'd like.  Then whenever you see "@", pronounce
that word instead <wink>.

> class MyLittleToy:
>   def methy1(self):
>       print 'whee'
>   @staticmethod
>   def methy2():
>        print 'woo'
>   @staticmethod
>   def methy3(arg):
>        for i in arg:
>           print 'woohoo'
> 
> ? That bothers me a looot... and is the primary reason I hate the pie.
> That and I really don't want anymore punctuation.

That bothers me a lot too, but because the world really isn't running
out of vertical whitespace:

class MyLittleToy:

    def methy1(self):
        print 'whee'

    @staticmethod
    def methy2():
        print 'woo'

    @staticmethod
    def methy3(arg):
        for i in arg:
            print 'woohoo'
 
doesn't bother me.
From mike.spam.filter at day8.com.au  Sat Aug  7 09:00:01 2004
From: mike.spam.filter at day8.com.au (Mike Thompson)
Date: Sat Aug  7 09:00:12 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <cf1nvj$uk0$1@sea.gmane.org>
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<cf1nvj$uk0$1@sea.gmane.org>
Message-ID: <cf1ulk$75r$1@sea.gmane.org>

Terry Reedy wrote:

> 
> Is a space allowed after the prefic character, whatever it might be?  With
> 
> def foo():
>    | classmethod
>    | accepts(int,int)
>    | returns(float)
> 


Thats it!  I love it.


From t-meyer at ihug.co.nz  Sat Aug  7 09:14:29 2004
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Sat Aug  7 09:14:34 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E874FB36@its-xchg4.massey.ac.nz>
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E86C36A8@its-xchg4.massey.ac.nz>

[Tim Peters explaining that you don't have to crunch everything up together]
> That bothers me a lot too, but because the world really isn't running
> out of vertical whitespace:
> 
> class MyLittleToy:
> 
>     def methy1(self):
>         print 'whee'
> 
>     @staticmethod
>     def methy2():
>         print 'woo'
> 
>     @staticmethod
>     def methy3(arg):
>         for i in arg:
>             print 'woohoo'

If the syntax does end up along these lines (and I'm expressing no opinion
either way), then maybe it would be worth explicitly telling people (i.e. in
PEP 8) that they ought to put a blank line before decorators to make it
clear?  Yes, people should figure this themselves, but they probably won't
<0.5 wink>, and it does make for ugly code.

Same deal for "don't do '@this @and_this @on_the_same_line'" (unless that
loses it's validity).

(And if some other syntax 'wins', then a guide for using that couldn't hurt,
too).

=Tony Meyer

p.s. Anyone else have a suspicion that Anthony checked in the @decorators to
2.4a2 as a sneaky way of getting more people than normal (all them riled up
ones) to download the alpha and try it out? <wink>

From s.percivall at chello.se  Sat Aug  7 09:33:00 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Sat Aug  7 09:33:03 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu>
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <02B028A7-E844-11D8-8B30-0003934AD54A@chello.se>


On 2004-08-07, at 06.35, Edward Loper wrote:
> The new "vertical bar" syntax was proposed recently, and has gotten
> almost unanimously positive feedback.  In particular, of 17 responses
> to the syntax, only one has been negative (see table, below).

-1 from me, so that's two. | just isn't a good sign.

From s.percivall at chello.se  Sat Aug  7 10:03:24 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Sat Aug  7 10:03:27 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <000401c47bec$588620a0$e841fea9@oemcomputer>
References: <000401c47bec$588620a0$e841fea9@oemcomputer>
Message-ID: <41F91659-E848-11D8-8B30-0003934AD54A@chello.se>


On 2004-08-06, at 21.34, Raymond Hettinger wrote:

>> | just isn't a good sign.
>
> How about the exclamation point?
>
>
> Raymond
>
>

If "@" is controversial enough to be a no-no,
"!" gets +1 from me.

From florian.proff.schulze at gmx.net  Sat Aug  7 10:42:16 2004
From: florian.proff.schulze at gmx.net (Florian Schulze)
Date: Sat Aug  7 10:37:44 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
Message-ID: <opsccs4qc55c87rg@news.gmane.org>

On Fri, 06 Aug 2004 11:39:34 -0400, Barry Warsaw <barry@python.org> wrote:

> So I spent a little time futzing with python-mode and some of my
> decorator code, to try some alternative leading characters.  Of the ones
> that cannot be used in valid Python code today (i.e. no backward
> compatibility issues), I tried ':', '/', '=', '*', and '|'.
>
> [snipped]
>
> My preference then would be to use = with | and : as alternatives, in
> that order.

Does it really need to be only one character? Would this make the parser  
to complicated? If not, I would propose '::' which stands out much more  
than ':'

What about '~'?

More:
  '|='
  '|~'
  ':~'
  ':=' - This was brought up as late binding as well:
    name := staticmethod(name)
    def name(self):
        ...
  '&' - Used for too much else IMO.

Now in context:

   ::accepts(int,int)
   ::returns(float)
   def bar(low,high):
       ...

   |=accepts(int,int)
   |=returns(float)
   def bar(low,high):
       ...

   |~accepts(int,int)
   |~returns(float)
   def bar(low,high):
       ...

   :~accepts(int,int)
   :~returns(float)
   def bar(low,high):
       ...

   :=accepts(int,int)
   :=returns(float)
   def bar(low,high):
       ...

   ~accepts(int,int)
   ~returns(float)
   def bar(low,high):
       ...

   ~~accepts(int,int)
   ~~returns(float)
   def bar(low,high):
       ...


If all this is not possible, then I +1 for '|'.
   |accepts(int,int)
   |returns(float)
   def bar(low,high):
       ...

Regards,
Florian Schulze
ps: Just for reference:

   @accepts(int,int)
   @returns(float)
   def bar(low,high):
       ...

From martin at v.loewis.de  Sat Aug  7 11:26:07 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug  7 11:26:15 2004
Subject: [Python-Dev] MSI Installer issues
In-Reply-To: <1f7befae04080623276089908e@mail.gmail.com>
References: <411231C8.3020308@interlink.com.au>
	<411241F5.3080602@v.loewis.de>	<1f7befae040805103911610582@mail.gmail.com>	<411274C0.4070300@v.loewis.de>
	<1f7befae04080623276089908e@mail.gmail.com>
Message-ID: <4114A02F.7060906@v.loewis.de>

Tim Peters wrote:
>>Yes. I don't know what to do about this: the screen sets ALLUSERS=1,
>>then performs FindRelatedProducts to find the old installation. If the
>>old installation was per-machine, we only find it if we commit to making
>>the new installation also per-machine.
> 
> 
> Except that in this case, I'm pretty sure my 2.4a1 installation was
> also "only for me".  I could be wrong about that.

Indeed. Upgrading from a per-user installation is easy, as the whole
process starts with the assumption that it is going to be per-user. Only
per-machine were previously not upgradable.

> I believe I later saw a checkin that intended to fix this, and, if so,
> that's cool too.  I hereby nominate you as MSI-builder For Life
> <wink>.

Yes, and thanks :-) Part of my recent talking goes along with the hope
that others become interested enough in MSI as a technology to be able
to contribute, though.

While I have your attention: you have commented that the installer logo
needs attribution, during the installation process. Is this still the
case, and what is the text I should use? Any other attribution that
needs to be made?

A colleague has commented that logo is too low-res, is ugly, and does
not feature reptiles. Should we look for an update to the logo?

Regards,
Martin
From heikowu at ceosg.de  Sat Aug  7 11:27:04 2004
From: heikowu at ceosg.de (Heiko Wundram)
Date: Sat Aug  7 11:26:24 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu>
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <200408071127.04568.heikowu@ceosg.de>

Am Samstag, 7. August 2004 06:35 schrieb Edward Loper:
> I'm including both of the following variants:
>      |classmethod
>      |accepts(int, int)
>      |returns(float)
>
>      def foo(arg1, arg2):
>          ...
>
>      def foo(arg1, arg2):
>          |classmethod
>          |accepts(int, int)
>          |returns(float)
>
>          ...
>
> (I prefer the second, but would be happy with either.)

-0.5 for the first, because | hard to read (I just noticed that in my eMail 
client), and -1 for the second, because decorating a function (some of you 
suggested even _after_ the docstring) in the actual function takes away all 
there is to decorators: making it immediately clear that a function is being 
decorated. If we would have to specify decorators after the actual def and 
the doc-string, there'd be no need for decorators, as that's what we can do 
at the moment too.

Heiko.
From heikowu at ceosg.de  Sat Aug  7 11:28:56 2004
From: heikowu at ceosg.de (Heiko Wundram)
Date: Sat Aug  7 11:28:15 2004
Subject: [Python-Dev] Suggesting '.' decorators (PEP318)
In-Reply-To: <411460C7.4030708@sabaydi.com>
References: <411460C7.4030708@sabaydi.com>
Message-ID: <200408071128.56011.heikowu@ceosg.de>

Am Samstag, 7. August 2004 06:55 schrieb Kevin J. Butler:
> The following makes sense to me, uses previously invalid syntax, and
> uses the '.' punctuation in a way similar to existing/proposed usage:
>
>        def longMethodNameForEffect(longArgumentOne=None,
>                                    longArgumentTwo=42):
>            .staticmethod
>            .funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
>                       status="experimental", author="BDFL")
>            """
>            asdfasdf
>            """
>            raise NotYetImplemented

-1

Same objection from me as to the proposal of decorating with | in the 
function: it simply takes away the actual meaning from the decorator (which 
actually is to make it very clear to see that the function is special in some 
way).

Heiko.
From martin at v.loewis.de  Sat Aug  7 11:31:29 2004
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug  7 11:31:32 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <opsccs4qc55c87rg@news.gmane.org>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>
	<opsccs4qc55c87rg@news.gmane.org>
Message-ID: <4114A171.3050809@v.loewis.de>

Florian Schulze wrote:
> Does it really need to be only one character? Would this make the 
> parser  to complicated? If not, I would propose '::' which stands out 
> much more  than ':'

Anything involving colon might break tools who believe that in Python,
the colon opens a block. Of course, it might then also be that these
tools recognize that there text after the colon (both after the first
and after the second one), and assume that this uses the single-line
version of suite.

> What about '~'?

It already is a unary operator:

 >>> def foo():
...   return 1
...
 >>> ~foo()
-2

Regards,
Martin
From martin at v.loewis.de  Sat Aug  7 12:30:16 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug  7 12:30:18 2004
Subject: [Python-Dev] PyRun_ with file name
In-Reply-To: <20040806220926.GA2225@unpythonic.net>
References: <4113EF69.2020102@v.loewis.de>
	<20040806220926.GA2225@unpythonic.net>
Message-ID: <4114AF38.5040107@v.loewis.de>

Jeff Epler wrote:
> Another alternative would be to complete the PyFile_* family of APIs
> (PyFile_Seek, etc), so that the C runtime routines would never be used.
> The "new" function in the PyRun_ family would take a PyFileObject*.

I'm not sure I like this alternative, though. It is more complex both
for the caller and for the Python runtime.

Regards,
Martin
From anthony at interlink.com.au  Sat Aug  7 12:42:46 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Aug  7 12:43:06 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <200408071127.04568.heikowu@ceosg.de>
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<200408071127.04568.heikowu@ceosg.de>
Message-ID: <4114B226.4020405@interlink.com.au>

Lest my previous post saying I could "live with" the
pipe symbol be misinterpreted, I still prefer @ over |.
My main reason for preferring the former is that it's
far more obvious in a page of code (I tried both). The
pipe tends to blend in to a disturbing degree. Making it
a different colour in an editor also doesn't seem to help
that much.

I'm also concerned that we don't make a horrible
precedent here - that a new language feature is
changed solely because other tools that were playing
outside the boundaries of the language will be
impacted. This would be an utter pain in the arse
going forward.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From anthony at interlink.com.au  Sat Aug  7 12:43:50 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Aug  7 12:44:10 2004
Subject: [Python-Dev] Suggesting '.' decorators (PEP318)
In-Reply-To: <411460C7.4030708@sabaydi.com>
References: <411460C7.4030708@sabaydi.com>
Message-ID: <4114B266.3020306@interlink.com.au>

Kevin J. Butler wrote:
> Many people wrote:
>    > Lots of options
> 
> I haven't seen this mentioned, so I'm going to throw in my suggestion.
> 
> The following makes sense to me, uses previously invalid syntax, and
> uses the '.' punctuation in a way similar to existing/proposed usage:

-1. This violates my "syntax should not look like grit" rule.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From adurdin at gmail.com  Sat Aug  7 14:06:36 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Sat Aug  7 14:06:38 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <4114A171.3050809@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>
	<opsccs4qc55c87rg@news.gmane.org> <4114A171.3050809@v.loewis.de>
Message-ID: <59e9fd3a04080705067633a790@mail.gmail.com>

I don't like the pipe at all, for two reasons:

One: as has already been mentioned, in many fonts it is easily
confused with lowercase L: l |

Two, and more importantly: the pipe is in the shifted position of the
backslash key, which on different keyboards is in different locations,
many of them awkward to reach while typing at speed. This problem is
particularly severe on laptop keyboards.

@ at least is consistently on the 2 key (except, of course, for all
those European keyboards. I guess their differences are enough to
knock down any argument about typing convenience :-)

Andrew Durdin
From adurdin at gmail.com  Sat Aug  7 14:13:57 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Sat Aug  7 14:13:59 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <4114B226.4020405@interlink.com.au>
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<200408071127.04568.heikowu@ceosg.de>
	<4114B226.4020405@interlink.com.au>
Message-ID: <59e9fd3a04080705132e5cf5d2@mail.gmail.com>

Edward Loper wrote:

>So I wanted to see if anyone has anything *negative* to say about this
>syntax (or one of its variants).  If possible, please respond by
>updating the wiki, rather than by email.  This syntax is listed under
>H (pre-def) and E2 (post-def). 

My only problem with the pipe is that it's often awkward to type
particularly on laptop keyboards where it gets squeezed into odd
locations. I've added a comment to this effect on the wiki.
From adurdin at gmail.com  Sat Aug  7 14:36:59 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Sat Aug  7 14:37:02 2004
Subject: [Python-Dev] Decorators with arguments are curries!
Message-ID: <59e9fd3a04080705361b6314be@mail.gmail.com>

Having waded through all the decorator syntax discussions recently,
and some of the historical ones, I haven't found this point mentioned
before, so I'll bring it up now:

Consider the following code:

def foo(bar):
   return 1

a = foo
a = foo(bar)

The first assignment to a is binding a reference to a function; the
second is calling the function. This is a very significant difference
in python, and I'm concerned that all the current proposed decorator
syntaxes[*] are liable to cause confusion on this point. For example:

def foo_decorator(func):
    print "no params to this"
    return func

def bar_decorator(func, param):
    print param
    return func

@foo_decorator
@bar_decorator("one param here")
def decorated_func():
    pass

Here the first decorator statement is bare, while the second one
includes parentheses and an argument; the first one looks like a
function reference, while the second looks like a function call.
I find this confusing; semantically, this appears to equivalent to
(assuming expressions are allowed in decorator statements):

# using curry as proposed in PEP 309 [**]
@foo_decorator
@curry(bar_decorator, "one param here")
def decorated_func():
    pass

Most of my concern here is that this aspect of decorator syntax
appears to be implicitly introducing a currying syntax in one special
circumstance, which is then *not* transferable to currying functions
in normal situations, as it would conflict with function calling. And
if a currying syntax (or built-in) was introduced down the track, then
these decorators would be inconsistent with them.

Has anyone else noticed this issue?

Andrew Durdin

[*] Except those which don't allow for arguments
[**] For a nice coincidence, PEP 309 suggested using '@' to mark curries
From martin at v.loewis.de  Sat Aug  7 14:52:04 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug  7 14:52:17 2004
Subject: [Python-Dev] PEP-0006, Bug Fix Releases
In-Reply-To: <4113C900.6060902@interlink.com.au>
References: <4113C900.6060902@interlink.com.au>
Message-ID: <4114D074.6080404@v.loewis.de>

Anthony Baxter wrote:
>     1. Backport bug fixes. If you fix a bug, and it seems appropriate,
>        port it to the CVS branch for the current bug fix release. If
>        you're unwilling or unable to backport it yourself, make a note
>        in the commit message.

Would it be helpful to mandate a precise wording?

>     Starting with Python 2.0, all feature releases are required to have
>     a version number of the form X.Y; patch releases will always be of
>     the form X.Y.Z.

Here is one instance of "patch release" left.

>     The process for managing patch releases is modeled in part on the cl
>     Tsystem [1].

This should have been left at "Tcl system".

>     [1] http://dev.scriptics.com:8080/cgi-bin/tct/tip/28.html

This should be

http://www.tcl.tk/cgi-bin/tct/tip/28.html

at the moment, as Scriptics is gone.

Regards,
Martin
From adurdin at gmail.com  Sat Aug  7 14:59:02 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Sat Aug  7 14:59:08 2004
Subject: [Python-Dev] Re: Decorators with arguments are curries!
In-Reply-To: <59e9fd3a04080705361b6314be@mail.gmail.com>
References: <59e9fd3a04080705361b6314be@mail.gmail.com>
Message-ID: <59e9fd3a040807055971518181@mail.gmail.com>

On Sat, 7 Aug 2004 22:36:59 +1000, Andrew Durdin <adurdin@gmail.com> wrote:
> # using curry as proposed in PEP 309 [**]
> @foo_decorator
> @curry(bar_decorator, "one param here")

And that really should have been:

# using rightcurry as mentioned in PEP 309 [**]
@foo_decorator
@rightcurry(bar_decorator, "one param here")

But that's not a primary issue.
The primary issues were: (1) That decorators with vs. without
arguments look inconsistent, like function calls vs. function
references, and (2) that consequentially, decorator syntax is
implicitly introducing a non-viable currying syntax.
From martin at v.loewis.de  Sat Aug  7 15:00:13 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug  7 15:00:19 2004
Subject: [Python-Dev] Decorators with arguments are curries!
In-Reply-To: <59e9fd3a04080705361b6314be@mail.gmail.com>
References: <59e9fd3a04080705361b6314be@mail.gmail.com>
Message-ID: <4114D25D.7090207@v.loewis.de>

Andrew Durdin wrote:
> def bar_decorator(func, param):
>     print param
>     return func
> 
> @foo_decorator
> @bar_decorator("one param here")
> def decorated_func():
>     pass
> 
> Here the first decorator statement is bare, while the second one
> includes parentheses and an argument; the first one looks like a
> function reference, while the second looks like a function call.

Correct. And that is indeed the intended meaning. Did you try this
out? It gives

Traceback (most recent call last):
   File "b.py", line 9, in ?
     @foo_decorator
TypeError: bar_decorator() takes exactly 2 arguments (1 given)

(although, as you can see, the line number is off by one)

See

http://www.python.org/dev/doc/devel/ref/function.html

on why this is so.

> Most of my concern here is that this aspect of decorator syntax
> appears to be implicitly introducing a currying syntax in one special
> circumstance, which is then *not* transferable to currying functions
> in normal situations, as it would conflict with function calling. 

And yet, the proposal does no such thing.

Regards,
Martin
From adurdin at gmail.com  Sat Aug  7 15:13:50 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Sat Aug  7 15:13:52 2004
Subject: [Python-Dev] Decorators with arguments are curries!
In-Reply-To: <4114D25D.7090207@v.loewis.de>
References: <59e9fd3a04080705361b6314be@mail.gmail.com>
	<4114D25D.7090207@v.loewis.de>
Message-ID: <59e9fd3a04080706135e4f622b@mail.gmail.com>

On Sat, 07 Aug 2004 15:00:13 +0200, "Martin v. L?wis"
<martin@v.loewis.de> wrote:
> Andrew Durdin wrote:
> >
> > Here the first decorator statement is bare, while the second one
> > includes parentheses and an argument; the first one looks like a
> > function reference, while the second looks like a function call.
> 
> Correct. And that is indeed the intended meaning. Did you try this
> out? It gives
> 
> Traceback (most recent call last):
>    File "b.py", line 9, in ?
>      @foo_decorator
> TypeError: bar_decorator() takes exactly 2 arguments (1 given)
> 
> (although, as you can see, the line number is off by one)
> 
> See
> 
> http://www.python.org/dev/doc/devel/ref/function.html
> 
> on why this is so.

Ah! Quite so. The relevant quote would be: "The result [of the
decorator expression] must be a callable, which is invoked with the
function object as the only argument".
I guess I didn't pay quite enough attention to the examples in PEP 318
to understand them properly :)
From mwh at python.net  Sat Aug  7 15:23:39 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Aug  7 15:23:40 2004
Subject: [Python-Dev] PEP-0006, Bug Fix Releases
In-Reply-To: <4113C900.6060902@interlink.com.au> (Anthony Baxter's message
	of "Sat, 07 Aug 2004 04:08:00 +1000")
References: <4113C900.6060902@interlink.com.au>
Message-ID: <2mllgr3wv8.fsf@starship.python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

>     In addition, anyone from the Python community is free to suggest
>     patches for inclusion. Patches may be submitted specifically for
>     patch releases; they should follow the guidelines in PEP 3 [2].
>     In general, though, it's probably better that a bug in a specific
>     release also be fixed on the HEAD as well as the branch.

I thought the consensus policy was now stronger than this: bugfixes
are tested on the trunk (unless the bug has gone away on the trunk as
part of feature work or something).  Submitting a patch that only
applies to release2x-maint is not good form.

>     Bug fix releases are expected to occur at an interval of roughly
>     six months. This is only a guideline, however - obviously, if a
>     major bug is found, a bugfix release may be appropriate sooner. In
>     general, only the N-1 release will be under active maintenance at
>     any time. That is, during Python 2.4's development, Python 2.3 gets
                                                         ^
                                                         only?

>     bugfix releases.

Cheers,
mwh

-- 
  But since your post didn't lay out your assumptions, your goals,
  or how you view language characteristics as fitting in with 
  either, you're not a *natural* candidate for embracing Design by 
  Contract <0.6 wink>.    -- Tim Peters, giving Eiffel adoption advice
From mwh at python.net  Sat Aug  7 15:26:18 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Aug  7 15:26:21 2004
Subject: [Python-Dev] pie-thon redux
In-Reply-To: <5.1.1.6.0.20040806142605.03303d90@mail.telecommunity.com>
	(Phillip J. Eby's message of "Fri, 06 Aug 2004 14:28:30 -0400")
References: <5.1.1.6.0.20040806142605.03303d90@mail.telecommunity.com>
Message-ID: <2mhdrf3wqt.fsf@starship.python.net>

"Phillip J. Eby" <pje@telecommunity.com> writes:

> At 02:13 AM 8/7/04 +1000, Anthony Baxter wrote:
>>As a break from the decorator thread, NTK (www.ntk.net)
>>had this writeup of the Pie-Thon. It amused me.
>>
>>[snip]
>
> I'm not usually one for today's IM slang, but...
>
> OMG!!!!  RTFLOL!!!  :)

Does this mean you're new to NTK?  They also had the best write-up
anywhere of the open sourcing of Principia/Zope:

                                >> TRACKING <<
                  making good use of the things that we find 

         Professional Web content systems companies, switching to
         Open Source - as recommended by their own *venture
         capitalist*? That's *sick*. ZOPE is a PERVERTED hybrid of
         the widely-regarded BOBO object-oriented CGI replacement,
         and the previously commercial PRINCIPIA Web application
         system. It's also got the Aqueduct relational database
         system thrown in. It's all free, source code is included,
         and even more horrifically, it's centered on the PYTHON
         language, that politically-correct, nicely-formatted,
         easy-to-read OPEN BLOODY SOURCE scripting language. ZOPE's
         creators will be releasing the source later today, so we
         haven't had a chance to play, but we hope it's bloody
         rubbish, otherwise we'll have nothing nasty to say at all.
         http://www.zope.org/
                  - nice clean well-written Website too. CURSE THEM! 

http://www.ntk.net/

Cheers,
mwh

-- 
    -Dr. Olin Shivers,
     Ph.D., Cranberry-Melon School of Cucumber Science
                                           -- seen in comp.lang.scheme
From python at rcn.com  Sat Aug  7 03:31:19 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sat Aug  7 15:33:06 2004
Subject: [Python-Dev] Suggesting '.' decorators (PEP318)
In-Reply-To: <4114B266.3020306@interlink.com.au>
Message-ID: <001c01c47c1e$3e0fc320$e841fea9@oemcomputer>

> > I haven't seen this mentioned, so I'm going to throw in my
suggestion.
> >
> > The following makes sense to me, uses previously invalid syntax, and
> > uses the '.' punctuation in a way similar to existing/proposed
usage:
> 
> -1. This violates my "syntax should not look like grit" rule.

Also, the dot prefix should probably be kept free in case a WITH
statement is ever introduced.


Raymond


From mwh at python.net  Sat Aug  7 15:56:07 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Aug  7 15:56:09 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <59e9fd3a04080705067633a790@mail.gmail.com> (Andrew Durdin's
	message of "Sat, 7 Aug 2004 22:06:36 +1000")
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<opsccs4qc55c87rg@news.gmane.org> <4114A171.3050809@v.loewis.de>
	<59e9fd3a04080705067633a790@mail.gmail.com>
Message-ID: <2md6233vd4.fsf@starship.python.net>

Andrew Durdin <adurdin@gmail.com> writes:

> @ at least is consistently on the 2 key (except, of course, for all
> those European keyboards. I guess their differences are enough to
> knock down any argument about typing convenience :-)

@ is shift-2 on my Mac and shift-' on my PC.  But, hell, # is option-3
on my Mac and I live with that.

Cheers,
mwh

-- 
  I've even been known to get Marmite *near* my mouth -- but never
  actually in it yet.  Vegamite is right out.
 UnicodeError: ASCII unpalatable error: vegamite found, ham expected
                                       -- Tim Peters, comp.lang.python
From pf_moore at yahoo.co.uk  Sat Aug  7 16:12:53 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sat Aug  7 16:12:46 2004
Subject: [Python-Dev] Re: pre-PEP: Complete,
	Structured Regular Expression Group Matching
References: <411429FB.6080603@gradient.cis.upenn.edu>
	<lkd1xijedzx.fsf@panix2.panix.com>
Message-ID: <brhn9guy.fsf@yahoo.co.uk>

Mike Coleman <mkc@mathdogs.com> writes:

> Edward Loper <edloper@gradient.cis.upenn.edu> writes:
>> I wouldn't lose heart over the lack of response -- the current
>> flood of decorator messages tends to drown out everything else.
>
> No, not at all. I was hoping I might benefit from @-fatigue, though.
> :-)

I was interested in your message as a break from @decorators, but I
couldn't muster the energy to think about the proposal because I've
been reading too many @decorator messages. I guess you won one way and
lost the other :-)

Seriously, although I don't use the re module much (overreaction to an
early exposure to Perl, I guess) the idea seems useful. I'm not
entirely sure about the implementation, though - I dislike the name
structmatch, as it doesn't say what's going on very well. At first I
thought that having a separate method was overkill, but given that
re.search and re.match both exist, I take that back.

Maybe a better name would be re.parse? The function has a "feel" of
parsing a string according to a pattern.

The only other comment I have is that the semantics seem pretty
complex - I think that in practice, they do more or less what you want
them to, but the description is pretty obscure. And although I can see
that the error return has some value, I suspect that it might actually
complicate real use.

A suggestion - would it be possible to implement re.structmatch as a
pure Python prototype, to thrash out some of the usability questions?
If the function appears valuable in such a form, arguing for
incorporation into the re module would be a lot easier.

Hope this helps,
Paul.
-- 
The only reason some people get lost in thought is because it's
unfamiliar territory -- Paul Fix

From mwh at python.net  Sat Aug  7 16:16:09 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Aug  7 16:16:11 2004
Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular
	Expression Group Matching
In-Reply-To: <lkdvffvx224.fsf@panix2.panix.com> (Mike Coleman's message of
	"06 Aug 2004 19:46:27 -0400")
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<2mbrho5o6c.fsf@starship.python.net>
	<lkdvffvx224.fsf@panix2.panix.com>
Message-ID: <2m7jsb3ufq.fsf@starship.python.net>

Mike Coleman <mkc@mathdogs.com> writes:

> Michael Hudson <mwh@python.net> writes:
>> Generally, it strikes me as mildly useful.  An implementation would
>> surely help :-)
>
> If yours is the most positive review, there probably won't be one.  :-)

Well, I thought *someone* should say *something*, just to stop the
proto-pep being lost.  Have you sent it to the PEP editor yet?  I
think it's ready enough.

The reason that I only said "mildly useful" was because I didn't think
"OMG!  I would so have liked to use this yesterday" or anything like
that...

> Seriously, though, I'd readily concede that there are different
> approaches to solving this problem, and I was hoping for some
> feedback and suggestions before I start whacking at the code.  I
> also have considered the possibility that although this has been
> driving me nuts that few others will care--if this is true, it
> probably *shouldn't* go in the core libraries.

It depends a bit on whether it can be implemented as an addition in a
non-painful way, or whether editing the sre code makes it very much
easier.  I don't know the answer, mind.

>>> [mismatch index return]
>> I don't really like this idea; it seems to be that it would be more
>> helpful to raise an exception and attach this data to the exception.
>> But that's a bit inconsisent with how match and search work.
>
> Hmm.  Yeah, if it weren't for the value of matching re.match semantics, it
> would make a lot more sense for this to be an exception.  As it is, I could
> see it going either way. 

I think you could make a case for raising an exception; the point of
structmatch is to extract information, whereas at least sometimes you
call re.match just to see if the data matches the pattern.

> Or I suppose you could add a flag to determine which (but ugh).

Err, let's not go there.

Cheers,
mwh

-- 
  Lisp nearing the age of 50 is the most modern language out
  there. GC, dynamic, reflective, the best OO model extant including
  GFs, procedural macros, and the only thing old-fashioned about it 
  is that it is compiled and fast.   -- Kenny Tilton, comp.lang.python
From amk at amk.ca  Sat Aug  7 16:23:27 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Sat Aug  7 16:23:48 2004
Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows
Message-ID: <20040807142327.GA2529@rogue.amk.ca>

A recent patch makes asyncore.py's .set_reuse_addr method work on Windows, 
which needs a different socket option:

            if sys.platform == 'win32':
                reuse_constant = socket.SO_EXCLUSIVEADDRUSE
            else:
                reuse_constant = socket.SO_REUSEADDR

Question: should the condition be sys.platform.startswith('win') for
the sake of win64, or other Windows variants?  (I don't know which
values sys.platform can have on Windows.)

--amk
From pf_moore at yahoo.co.uk  Sat Aug  7 16:27:10 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sat Aug  7 16:27:02 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<cf1nvj$uk0$1@sea.gmane.org> <cf1ulk$75r$1@sea.gmane.org>
Message-ID: <7jsb9g75.fsf@yahoo.co.uk>

Mike Thompson <mike.spam.filter@day8.com.au> writes:

> Terry Reedy wrote:
>
>> Is a space allowed after the prefic character, whatever it might be?
>> With
>> def foo():
>>    | classmethod
>>    | accepts(int,int)
>>    | returns(float)
>>
>
>
> Thats it!  I love it.

Note that there is no implementation of this currently available
(whereas changing @ to | in the existing syntax is likely to be a
trivial change). Proponents of this variant (or any other without an
implementation available) will ultimately have to deliver code.

I've added notes to the Wiki as to which variants have
implementations, FWIW.

Paul
-- 
Instant gratification takes too long -- Carrie Fisher

From martin at v.loewis.de  Sat Aug  7 16:29:39 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug  7 16:29:40 2004
Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows
In-Reply-To: <20040807142327.GA2529@rogue.amk.ca>
References: <20040807142327.GA2529@rogue.amk.ca>
Message-ID: <4114E753.2050303@v.loewis.de>

A.M. Kuchling wrote:
> Question: should the condition be sys.platform.startswith('win') for
> the sake of win64, or other Windows variants?  (I don't know which
> values sys.platform can have on Windows.)

Win64 also uses "win32" as sys.platform. If you really want to find
out this is not x86, you need to look at the compiler ID in sys.version.

Regards,
Martin
From amk at amk.ca  Sat Aug  7 16:51:42 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Sat Aug  7 16:52:01 2004
Subject: [Python-Dev] re.split on empty patterns
Message-ID: <20040807145142.GB2529@rogue.amk.ca>

The re.split() method ignores zero-length pattern matches.  Patch
#988761 adds an emptyok flag to split that causes zero-length matches
to trigger a split.  For example:

>>> re.split(r'\b', 'this is a sentence')# does nothing; \b is always length 0
['this is a sentence']
>>> re.split(r'\b', 'this is a sentence', emptyok=True)
['', 'this', ' ', 'is', ' ', 'a', ' ', 'sentence', '']

Without the patch, the various zero-length assertions are 
pretty useless; with it, they can serve a purpose with split():

>>> re.split(r'(?m)$', 'line1\nline2\n', emptyok=True)
['line1', '\nline2', '\n', '']
>>> # Split file into sections
>>> re.split("(?m)(?=^[[])", """[section1]
foo=bar

[section2]
coyote=wiley
""", emptyok=True)
['', '[section1]\nfoo=bar\n\n', '[section2]\ncoyote=wiley\n']

Zero-length matches often result in a '' at the beginning or end, or
between characters, but I think users can handle that.  IMHO this
feature is clearly useful, and would be happy to commit the patch
as-is.

Question: do we want to make this option the new default?  Existing
patterns that can produce zero-length matches would change their
meanings:

>>> re.split('x*', 'abxxxcdefxxx')
['ab', 'cdef', '']
>>> re.split('x*', 'abxxxcdefxxx', emptyok=True)
['', 'a', 'b', '', 'c', 'd', 'e', 'f', '', '']

(I think the result of the second match points up a bug in the patch;
the empty strings in the middle seem wrong to me.  Assume that gets
fixed.)

Anyway, we therefore can't just make this the default in 2.4.  We
could trigger a warning when emptyok is not supplied and a split
pattern results in a zero-length match; users could supply
emptyok=False to avoid the warning.  Patterns that never have a
zero-length match would never get the warning.  2.5 could then set
emptyok to True.  

Note: raising the warning might cause a serious performance hit for
patterns that get zero-length matches a lot, which would make 2.4
slower in certain cases.

Thoughts?  Does this need a PEP?

--amk


From amk at amk.ca  Sat Aug  7 17:02:29 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Sat Aug  7 17:02:48 2004
Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows
In-Reply-To: <4114E753.2050303@v.loewis.de>
References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de>
Message-ID: <20040807150229.GC2529@rogue.amk.ca>

On Sat, Aug 07, 2004 at 04:29:39PM +0200, "Martin v. L?wis" wrote:
> Win64 also uses "win32" as sys.platform. If you really want to find
> out this is not x86, you need to look at the compiler ID in sys.version.

OK; I just care about platforms where the alternative constant is
necessary.  It sounds to me like win32 is therefore the only relevant
sys.platform string to check for.  Thanks!

(Hm... should it check for "cygwin", too?  If any Cygwin maintainers
have suggestions, please let me know.)

--amk

From amk at amk.ca  Sat Aug  7 17:27:08 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Sat Aug  7 17:27:26 2004
Subject: [Python-Dev] Pickler()'s bin argument
Message-ID: <20040807152708.GA2670@rogue.amk.ca>

PEP 307 says this about Pickler:

    Passing 'bin' as a keyword argument is deprecated,
    and a PendingDeprecationWarning is issued in this case.  You have
    to invoke the Python interpreter with -Wa or a variation on that
    to see PendingDeprecationWarning messages.  In Python 2.4, the
    warning class may be upgraded to DeprecationWarning.

Should I change it to become a DeprecationWarning in 2.4?

--amk
From barry at python.org  Sat Aug  7 17:54:07 2004
From: barry at python.org (Barry Warsaw)
Date: Sat Aug  7 17:54:11 2004
Subject: [Python-Dev] RE: [spambayes-dev] bug in imap filter or in email
	package
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E86C36A1@its-xchg4.massey.ac.nz>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E86C36A1@its-xchg4.massey.ac.nz>
Message-ID: <1091894046.1064.29.camel@anthem.wooz.org>

On Wed, 2004-08-04 at 21:56, Tony Meyer wrote:
> There has been a little bit of discussion on spambayes-dev about a bug with
> the 2.4a1 email package, where header lines that end with \r\n are not
> treated correctly (the value ends up with a \r at the end).
> 
> A SF bug was opened for this:
> 
> [ 1002475 ] email message parser doesn't handle \r\n correctly
> <http://sf.net/tracker/?group_id=5470&atid=105470&func=detail&aid=1002475>
> 
> I've created a patch to fix this, and a couple of tests to add to
> test_email.py:
> [ 1003693 ] Fix for 1002475 (Feedparser not handling \r\n correctly)
> <http://sf.net/tracker/index.php?func=detail&aid=1003693&group_id=5470&atid=
> 305470>
> 
> If someone would like to review this and check it in after 2.4a2 is all done
> that would be great.  Maybe someone at the bug day?  (I might come along to
> that, but it's the middle of the night, so probably not).

Patch looks good to me.  I'm checking them in (after a little stylistic
hydrant-pissing :).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040807/be5d5348/attachment.pgp
From pje at telecommunity.com  Sat Aug  7 17:59:04 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat Aug  7 17:55:00 2004
Subject: [Python-Dev] Decorators with arguments are curries!
In-Reply-To: <59e9fd3a04080705361b6314be@mail.gmail.com>
Message-ID: <5.1.1.6.0.20040807115729.028f3ce0@mail.telecommunity.com>

At 10:36 PM 8/7/04 +1000, Andrew Durdin wrote:
>The first assignment to a is binding a reference to a function; the
>second is calling the function. This is a very significant difference
>in python, and I'm concerned that all the current proposed decorator
>syntaxes[*] are liable to cause confusion on this point. For example:
>
>def foo_decorator(func):
>     print "no params to this"
>     return func
>
>def bar_decorator(func, param):
>     print param
>     return func
>
>@foo_decorator
>@bar_decorator("one param here")
>def decorated_func():
>     pass
>
>Here the first decorator statement is bare, while the second one
>includes parentheses and an argument; the first one looks like a
>function reference, while the second looks like a function call.

Your example will fail, saying that bar_decorator is being called without 
enough arguments.

Decorator syntax does *not* provide currying.  You have to write something 
like this:

     def bar_decorator(param):
         def decorate(func):
             print param
             return func
         return decorate

in order for your example to work.

From tim.peters at gmail.com  Sat Aug  7 17:56:09 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 17:56:11 2004
Subject: [Python-Dev] Pickler()'s bin argument
In-Reply-To: <20040807152708.GA2670@rogue.amk.ca>
References: <20040807152708.GA2670@rogue.amk.ca>
Message-ID: <1f7befae04080708563a3735be@mail.gmail.com>

[A.M. Kuchling]
> PEP 307 says this about Pickler:
> 
>    Passing 'bin' as a keyword argument is deprecated,
>    and a PendingDeprecationWarning is issued in this case.  You have
>    to invoke the Python interpreter with -Wa or a variation on that
>    to see PendingDeprecationWarning messages.  In Python 2.4, the
>    warning class may be upgraded to DeprecationWarning.
>
> Should I change it to become a DeprecationWarning in 2.4?

+1.  2.3's been current a long time, and AFAICT
PendingDeprecationWarning isn't a lot more warning than a comment in
the middle of cPickle.c.
From guido at python.org  Sat Aug  7 18:12:38 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug  7 18:12:44 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: Your message of "Sat, 07 Aug 2004 00:12:19 EDT."
	<cf1kr3$qe4$1@sea.gmane.org> 
References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de><20040805215331.GB31288@burma.localdomain><ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost><1f7befae040806090644be9c04@mail.gmail.com>
	<4113AF47.1010405@interlink.com.au>
	<200408061634.i76GYEZ07623@guido.python.org> 
	<cf1kr3$qe4$1@sea.gmane.org> 
Message-ID: <200408071612.i77GCcK10255@guido.python.org>


> > > - change the language ref that talks about this to state that
> > > although Python currently doesn't use these symbols, this is not
> > > stopping a future version of Python from using them for some new
> > > feature.
> >
> > Does that really have to be stated?
> 
> All that is needed is the addition of 'currently' in the sentence about
> illegality.  Compare with "The Python compiler currently generates the
> following byte code instructions. " (LibRef 18.10.1) which has the same
> word to guard against the same false presumption of stability and
> consequent complaints.

I still think it shouldn't be needed.  Do we have to add 'currently'
to every statement about the language?  That doesn't make sense.  The
reference manual's title page already includes a version number.
Shouldn't that be sufficient warning for those who want to interpret
any part of the manual as a promise for all future?

I really want to take a hard stance on this, because I believe the
only reason this came up was that someone needed to find an argument
against '@'.  I don't think their argument would have a chance in
court, so there's no reason to give in to them.

Fight the trend to add silly disclaimers everywhere!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Aug  7 18:17:42 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug  7 18:17:47 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: Your message of "Sat, 07 Aug 2004 00:35:17 EDT."
	<41145C05.1020501@gradient.cis.upenn.edu> 
References: <41145C05.1020501@gradient.cis.upenn.edu> 
Message-ID: <200408071617.i77GHgs10281@guido.python.org>

> The new "vertical bar" syntax was proposed recently, and has gotten
> almost unanimously positive feedback.  In particular, of 17 responses
> to the syntax, only one has been negative (see table, below).

I want to mention one argument for '@' that doesn't apply to '|': it
will be familiar to Java programmers.

> I'm including both of the following variants:
> 
>      |classmethod
>      |accepts(int, int)
>      |returns(float)
>      def foo(arg1, arg2):
>          ...
> 
>      def foo(arg1, arg2):
>          |classmethod
>          |accepts(int, int)
>          |returns(float)
>          ...
> 
> (I prefer the second, but would be happy with either.)

In the discussion on decorators months ago, solutions involving
special syntax inside the block were ruled out early on.  Basically,
you shouldn't have to peek inside the block to find out important
external properties of the function.  (Yes, I know that docstrings
violate this principle.  For the record, it would be better if they
were prefix too; and a prefix decorator syntax will let us fix this in
the future but introducing a "doc" decorator.)

Please add this to the Wiki for me.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Sat Aug  7 18:29:44 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug  7 18:29:49 2004
Subject: [Python-Dev] Re: Decorators with arguments are curries!
In-Reply-To: Your message of "Sat, 07 Aug 2004 22:59:02 +1000."
	<59e9fd3a040807055971518181@mail.gmail.com> 
References: <59e9fd3a04080705361b6314be@mail.gmail.com>  
	<59e9fd3a040807055971518181@mail.gmail.com> 
Message-ID: <200408071629.i77GTiT10357@guido.python.org>

> The primary issues were: (1) That decorators with vs. without
> arguments look inconsistent, like function calls vs. function
> references, and (2) that consequentially, decorator syntax is
> implicitly introducing a non-viable currying syntax.

I think you're mistaken.  When using @foo, foo should be a function
taking one argument; when using @foo(x,y,z), the *call* to foo(x,y,z)
should return a function of one argument (or an equivalent callable
object, of course).  This is often done by defining an inner "helper"
function, e.g.:

    def funcattrs(**kwds):
	def helper(func):
	    func.__dict__.update(kwds)
	return helper

    @funcattr(counter=42, obsolete=True)
    def foobar():
	pass

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Aug  7 18:37:00 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug  7 18:37:06 2004
Subject: [Python-Dev] Recommended way to tell platform
In-Reply-To: Your message of "Sat, 07 Aug 2004 16:29:39 +0200."
	<4114E753.2050303@v.loewis.de> 
References: <20040807142327.GA2529@rogue.amk.ca>  
	<4114E753.2050303@v.loewis.de> 
Message-ID: <200408071637.i77Gb0w10415@guido.python.org>

> A.M. Kuchling wrote:
> > Question: should the condition be sys.platform.startswith('win') for
> > the sake of win64, or other Windows variants?  (I don't know which
> > values sys.platform can have on Windows.)
> 
> Win64 also uses "win32" as sys.platform. If you really want to find
> out this is not x86, you need to look at the compiler ID in sys.version.
> 
> Regards,
> Martin

This does bring up an interesting question: what's the recommended way
to distinguish a certain platform?  There's os.name, sys.platform, the
platform module, and I've also seen code testing for os.sep=='/'.

I still prefer hasattr(<module>, <attribute>) whenever applicable,
e.g. preferring hasattr(os, 'fork') over os.name=='posix' (or
os.name!='nt' :-), but sometimes that's not possible.

What should be the preferred way?  (It may be impossible to say
because there are different use cases, but probably one of the most
important cases is simply distinguishing Windows from the rest -- how
should that be done?)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Sat Aug  7 18:38:34 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug  7 18:38:40 2004
Subject: [Python-Dev] Pickler()'s bin argument
In-Reply-To: Your message of "Sat, 07 Aug 2004 11:27:08 EDT."
	<20040807152708.GA2670@rogue.amk.ca> 
References: <20040807152708.GA2670@rogue.amk.ca> 
Message-ID: <200408071638.i77GcYm10437@guido.python.org>

> PEP 307 says this about Pickler:
> 
>     Passing 'bin' as a keyword argument is deprecated,
>     and a PendingDeprecationWarning is issued in this case.  You have
>     to invoke the Python interpreter with -Wa or a variation on that
>     to see PendingDeprecationWarning messages.  In Python 2.4, the
>     warning class may be upgraded to DeprecationWarning.
> 
> Should I change it to become a DeprecationWarning in 2.4?

Sure.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From colanderman at gmail.com  Sat Aug  7 18:57:48 2004
From: colanderman at gmail.com (Chris King)
Date: Sat Aug  7 18:57:51 2004
Subject: [Python-Dev] Suggesting '.' decorators (PEP318)
In-Reply-To: <001c01c47c1e$3e0fc320$e841fea9@oemcomputer>
References: <001c01c47c1e$3e0fc320$e841fea9@oemcomputer>
Message-ID: <875c7e07040807095770541225@mail.gmail.com>

On Fri, 6 Aug 2004 21:31:19 -0400, Raymond Hettinger <python@rcn.com> wrote:
> > > I haven't seen this mentioned, so I'm going to throw in my
> suggestion.
> > >
> > > The following makes sense to me, uses previously invalid syntax, and
> > > uses the '.' punctuation in a way similar to existing/proposed
> usage:
> >
> > -1. This violates my "syntax should not look like grit" rule.
> 
> Also, the dot prefix should probably be kept free in case a WITH
> statement is ever introduced.

Same here -- I'd rather see .foo at function level (i.e. not nested in
a with block) provide true function attributes, not decorators.
From colanderman at gmail.com  Sat Aug  7 19:11:04 2004
From: colanderman at gmail.com (Chris King)
Date: Sat Aug  7 19:11:06 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu>
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <875c7e070408071011495d1033@mail.gmail.com>

On Sat, 07 Aug 2004 00:35:17 -0400, Edward Loper
<edloper@gradient.cis.upenn.edu> wrote:
>      |classmethod
>      |accepts(int, int)
>      |returns(float)
>      def foo(arg1, arg2):
>          ...

Personally, I'd prefer @ over |: | looks too much like ASCII art (like
someone might draw in a multiline comment in C); whereas @ at least
looks like it's doing something.  Though for me, @ brings to mind
preprocessor directives, maybe that's for the best.

>      def foo(arg1, arg2):
>          |classmethod
>          |accepts(int, int)
>          |returns(float)
>          ...

This does look prettier (especially the space version), but it looks
even more like ASCII art.

So for the record, assuming I'm -1 on @, I'm -1.5 on this ;)
From bob at redivi.com  Sat Aug  7 19:18:23 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sat Aug  7 19:18:31 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <59e9fd3a04080705067633a790@mail.gmail.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>
	<opsccs4qc55c87rg@news.gmane.org> <4114A171.3050809@v.loewis.de>
	<59e9fd3a04080705067633a790@mail.gmail.com>
Message-ID: <C95074A4-E895-11D8-B7F3-000A95686CD8@redivi.com>


On Aug 7, 2004, at 8:06 AM, Andrew Durdin wrote:

> I don't like the pipe at all, for two reasons:
>
> One: as has already been mentioned, in many fonts it is easily
> confused with lowercase L: l |

Actually when I initially brought up this concern, it was an uppercase 
"i" that I was talking about.  Lowercase L of course also has this 
problem, but uppercase "i" is a relatively commonly used prefix.

-bob

From mkc at mathdogs.com  Sat Aug  7 19:33:05 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Sat Aug  7 19:33:10 2004
Subject: [Python-Dev] Re: pre-PEP: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<2mbrho5o6c.fsf@starship.python.net>
	<lkdvffvx224.fsf@panix2.panix.com>
	<2m7jsb3ufq.fsf@starship.python.net>
Message-ID: <lkd4qnec0q6.fsf@panix2.panix.com>

Michael Hudson <mwh@python.net> writes:
> Well, I thought *someone* should say *something*, just to stop the
> proto-pep being lost.  Have you sent it to the PEP editor yet?  I
> think it's ready enough.

I'm happy you did--I was just joking.  Yes, I've sent it to the PEP guys, but
no response yet.

> The reason that I only said "mildly useful" was because I didn't think
> "OMG!  I would so have liked to use this yesterday" or anything like
> that...

Sure, I understand.  I run into a lot of these cases at work, where the domain
is bioinformatics (lots of pattern matching and parsing of ad hoc file formats).

> It depends a bit on whether it can be implemented as an addition in a
> non-painful way, or whether editing the sre code makes it very much
> easier.  I don't know the answer, mind.

I'm thinking (hoping) that it could piggyback on the current re.match
implementation in a fairly clean way.  The algorithm is the same--we just have
to "notice" a few more things while we're carrying it out and arrange to
return those noticed things at the end.

> I think you could make a case for raising an exception; the point of
> structmatch is to extract information, whereas at least sometimes you
> call re.match just to see if the data matches the pattern.

Yeah, I'm leaning in this direction, too.

Mike

From tim.peters at gmail.com  Sat Aug  7 19:33:44 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 19:33:47 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <200408071612.i77GCcK10255@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de><20040805215331.GB31288@burma.localdomain><ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost><1f7befae040806090644be9c04@mail.gmail.com>
	<4113AF47.1010405@interlink.com.au>
	<200408061634.i76GYEZ07623@guido.python.org> 
	<cf1kr3$qe4$1@sea.gmane.org>
	<200408071612.i77GCcK10255@guido.python.org>
Message-ID: <1f7befae04080710335001c6b9@mail.gmail.com>

[Guido]
> I still think it shouldn't be needed.  Do we have to add 'currently'
> to every statement about the language?  That doesn't make sense.  The
> reference manual's title page already includes a version number.
> Shouldn't that be sufficient warning for those who want to interpret
> any part of the manual as a promise for all future?

Yes.

> I really want to take a hard stance on this, because I believe the
> only reason this came up was that someone needed to find an argument
> against '@'.

At least two reasonably popular Python tools use @ heavily now, and
their authors didn't appear to give a rip about decorators one way or
the other.  The use of @ for any purpose in the core would have
elicited similar concern.

> I don't think their argument would have a chance in court,

I believe they agree with that (partly because they both said so <wink>).

> so there's no reason to give in to them.

Courts are adversarial.  You don't want an adversarial relationship
with Python users -- there are lots of things to consider besides what
a court would say.

> Fight the trend to add silly disclaimers everywhere!

+1.  OTOH, I'm also +1 on picking a character and promising (in the
reference manual) that the language will never use it, to give authors
of these kinds of tools a way to live peacefully with Python
evolution.  @ seems like a good choice for that.
From mkc at mathdogs.com  Sat Aug  7 19:46:10 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Sat Aug  7 19:46:16 2004
Subject: [Python-Dev] Re: pre-PEP: Complete,
	Structured Regular Expression Group Matching
References: <411429FB.6080603@gradient.cis.upenn.edu>
	<lkd1xijedzx.fsf@panix2.panix.com> <brhn9guy.fsf@yahoo.co.uk>
Message-ID: <lkdzn56aljx.fsf@panix2.panix.com>

Paul Moore <pf_moore@yahoo.co.uk> writes:
> I was interested in your message as a break from @decorators, but I
> couldn't muster the energy to think about the proposal because I've
> been reading too many @decorator messages. I guess you won one way and
> lost the other :-)

@:-)

> Maybe a better name would be re.parse? The function has a "feel" of
> parsing a string according to a pattern.

This is a nice suggestion.  The main objection I can think of is that we might
be stealing that name from some future idea that would more deserve it.

> The only other comment I have is that the semantics seem pretty
> complex - I think that in practice, they do more or less what you want
> them to, but the description is pretty obscure.

Yeah, probably the doc will need to be carefully written, with good examples,
to make this go down easily.  I hope that for practical cases it'll be
relatively straightforward.

> And although I can see that the error return has some value, I suspect that
> it might actually complicate real use.

Someone else suggested throwing an exception instead, which I'm now leaning in
favor of, too.

> A suggestion - would it be possible to implement re.structmatch as a
> pure Python prototype, to thrash out some of the usability questions?
> If the function appears valuable in such a form, arguing for
> incorporation into the re module would be a lot easier.

I suspect that in this case a pure Python implementation might actually be a
lot harder than implementing it as a C patch.  This is because so much of the
work is already done in the current re module; the C patch would probably not
be that big.  In order to do it in Python I'd end up reimplementing re.match
in Python, which I wouldn't look forward to.

It's because of this that I'm trying to get as much preliminary comment as I
can.

Cheers,
Mike

From guido at python.org  Sat Aug  7 19:53:28 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug  7 19:53:34 2004
Subject: [Python-Dev] Suggesting '.' decorators (PEP318)
In-Reply-To: Your message of "Sat, 07 Aug 2004 12:57:48 EDT."
	<875c7e07040807095770541225@mail.gmail.com> 
References: <001c01c47c1e$3e0fc320$e841fea9@oemcomputer>  
	<875c7e07040807095770541225@mail.gmail.com> 
Message-ID: <200408071753.i77HrTL10619@guido.python.org>

> Same here -- I'd rather see .foo at function level (i.e. not nested in
> a with block) provide true function attributes, not decorators.

Bleah on that idea.  The body of the function should express what it
does when it is called.  Function attributes are for who calls it
(roughly).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Sat Aug  7 19:58:16 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 19:58:19 2004
Subject: [Python-Dev] re.split on empty patterns
In-Reply-To: <20040807145142.GB2529@rogue.amk.ca>
References: <20040807145142.GB2529@rogue.amk.ca>
Message-ID: <1f7befae040807105878b47c61@mail.gmail.com>

[A.M. Kuchling]
 <amk@amk.ca> wrote:
> The re.split() method ignores zero-length pattern matches.  Patch
> #988761 adds an emptyok flag to split that causes zero-length matches
> to trigger a split.

...

> IMHO this feature is clearly useful,

Yes it is!  Or, more accurately, it can be, when it's intended to
match an empty string.  It's a bit fuzzy because regexps are so
error-prone, and writing a regexp that matches an empty string by
accident is easy.

> and would be happy to commit the patch as-is.

Haven't looked at the patch, though.

> Question: do we want to make this option the new default?  Existing
> patterns that can produce zero-length matches would change their
> meanings:
>
> >>> re.split('x*', 'abxxxcdefxxx')
> ['ab', 'cdef', '']
> >>> re.split('x*', 'abxxxcdefxxx', emptyok=True)
> ['', 'a', 'b', '', 'c', 'd', 'e', 'f', '', '']
>
> (I think the result of the second match points up a bug in the patch;
> the empty strings in the middle seem wrong to me.  Assume that gets
> fixed.)

Agreed.

> Anyway, we therefore can't just make this the default in 2.4.  We
> could trigger a warning when emptyok is not supplied and a split
> pattern results in a zero-length match; users could supply
> emptyok=False to avoid the warning.  Patterns that never have a
> zero-length match would never get the warning.  2.5 could then set
> emptyok to True.
>
> Note: raising the warning might cause a serious performance hit for
> patterns that get zero-length matches a lot, which would make 2.4
> slower in certain cases.

If you don't intend to change the default, there's no problem.  I like
"no problem".  This isn't so useful so often that it can't afford to
wait for Python 3 to change.  In the meantime, "emptyok" is an odd
name since it's always "ok" to have an empty match.  "split0=True"
reads better to me, since the effect is to split on a 0-length match. 
split_on_empty_match would be too wordy.

> Thoughts?  Does this need a PEP?

It will if an argument starts now <wink>.
From s.percivall at chello.se  Sat Aug  7 20:02:34 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Sat Aug  7 20:02:36 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: <4113BACA.6040801@ieee.org>
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org>
Message-ID: <F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>

I just wanted to say that I believe it should be allowed to decorate
classes. There's not reason enough (reading the thread linked to by the
PEP) to limit decorators this way. Like said several times in that
thread, metaclasses are harder (to write and to understand) than
decorators.

Also, considering that you said you would have actually wanted
docstrings above the definition of a function, wouldn't this apply
to classes as well?

Anyway, it's an artificial limit and it would be better to be able
to test it out during the alpha.

//Simon

> Guido van Rossum wrote:
> >
> > Some people have argued that it should work for classes (Java and C#
> > decorators work for any declaration).  I've never been convinced,
> > although I'm not dead set against it either.
> >
> > It would seem that the functionality isn't quite as useful there; you
> > can get most of the same effects with metaclasses.  I'm not aware of
> > any idiom that takes a class and passes it through some kind of
> > wrapper or transformation *after* the class declaration; if somebody
> > is using this, it surely must be a very rare use indeed.
> >

From colanderman at gmail.com  Sat Aug  7 20:07:35 2004
From: colanderman at gmail.com (Chris King)
Date: Sat Aug  7 20:07:39 2004
Subject: [Python-Dev] Suggesting '.' decorators (PEP318)
In-Reply-To: <200408071753.i77HrTL10619@guido.python.org>
References: <001c01c47c1e$3e0fc320$e841fea9@oemcomputer>  
	<875c7e07040807095770541225@mail.gmail.com>
	<200408071753.i77HrTL10619@guido.python.org>
Message-ID: <875c7e070408071107cfab038@mail.gmail.com>

On Sat, 07 Aug 2004 10:53:28 -0700, Guido van Rossum <guido@python.org> wrote:
> > Same here -- I'd rather see .foo at function level (i.e. not nested in
> > a with block) provide true function attributes, not decorators.
> 
> Bleah on that idea.  The body of the function should express what it
> does when it is called.  Function attributes are for who calls it
> (roughly).

I'd still rather see .foo kept free for that purpose (in that
context), in case we can convince you of the merits of function
attributes sometime in the future ;)
From bob at redivi.com  Sat Aug  7 20:18:51 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sat Aug  7 20:19:01 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: <F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org>
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
Message-ID: <3BCE7F74-E89E-11D8-B7F3-000A95686CD8@redivi.com>

On Aug 7, 2004, at 2:02 PM, Simon Percivall wrote:

>> Guido van Rossum wrote:
>> >
>> > Some people have argued that it should work for classes (Java and C#
>> > decorators work for any declaration).  I've never been convinced,
>> > although I'm not dead set against it either.
>> >
>> > It would seem that the functionality isn't quite as useful there; 
>> you
>> > can get most of the same effects with metaclasses.  I'm not aware of
>> > any idiom that takes a class and passes it through some kind of
>> > wrapper or transformation *after* the class declaration; if somebody
>> > is using this, it surely must be a very rare use indeed.
>> >
> I just wanted to say that I believe it should be allowed to decorate
> classes. There's not reason enough (reading the thread linked to by the
> PEP) to limit decorators this way. Like said several times in that
> thread, metaclasses are harder (to write and to understand) than
> decorators.
>
> Also, considering that you said you would have actually wanted
> docstrings above the definition of a function, wouldn't this apply
> to classes as well?
>
> Anyway, it's an artificial limit and it would be better to be able
> to test it out during the alpha.

I've always been +1 for allowing them on classes too for the reasons 
that you state, and I would use them if they were there.  However, 
since we do have metaclasses and I know how to use them it's not nearly 
as important to me as function decorators and I'd rather not get in 
another big thread about it.

Note that you could make a docstring decorator that does just this..

def doc(docstring):
	def doc(fn):
		fn.__doc__ = docstring
		return fn
	return doc

@doc("""foo does nothing and takes no arguments""")
def foo():
	pass

It could of course work the same way for classes too.

-bob
From tim.peters at gmail.com  Sat Aug  7 20:25:09 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 20:25:13 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <4114B226.4020405@interlink.com.au>
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<200408071127.04568.heikowu@ceosg.de>
	<4114B226.4020405@interlink.com.au>
Message-ID: <1f7befae040807112543238881@mail.gmail.com>

[Anthony Baxter]
> Lest my previous post saying I could "live with" the
> pipe symbol be misinterpreted, I still prefer @ over |.
> My main reason for preferring the former is that it's
> far more obvious in a page of code (I tried both).

Hmm.  Why would you *want* decorator annotations to be especially
visible on a page of code?  Indentation and blank lines still make
class, method, and function boundaries clear, and that's the important
"page level" structure.  Decorators are details.

> The pipe tends to blend in to a disturbing degree.

Indeed, for me that's a reason to prefer | over @.  BTW, while the PEP
wasn't clear on this point last I looked, I hope it allows for
whitespace after the | (or @, or =, or whatever it is).

> Making it a different colour in an editor also doesn't seem to help
> that much.

Even better <wink>.

> I'm also concerned that we don't make a horrible
> precedent here - that a new language feature is
> changed solely because other tools that were playing
> outside the boundaries of the language will be
> impacted. This would be an utter pain in the arse
> going forward.

Well, this particular language feature has changed every week for most
of a year. If we're at the point where a change actually *helps*
someone, I'm not gonna hold that against it.
From tim.peters at gmail.com  Sat Aug  7 20:35:25 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 20:35:27 2004
Subject: [Python-Dev] Recommended way to tell platform
In-Reply-To: <200408071637.i77Gb0w10415@guido.python.org>
References: <20040807142327.GA2529@rogue.amk.ca>  
	<4114E753.2050303@v.loewis.de>
	<200408071637.i77Gb0w10415@guido.python.org>
Message-ID: <1f7befae04080711351466da96@mail.gmail.com>

[Guido]
> This does bring up an interesting question: what's the recommended way
> to distinguish a certain platform?  There's os.name, sys.platform, the
> platform module, and I've also seen code testing for os.sep=='/'.

I ask myself that question every time I need to do it.  Out of sheer
momentum now, I almost always use

    if sys.platform in ('win32',):

I dislike using os.name == 'nt', mostly because if we asked all
Windows Python users which OS they use, only a tiny percentage would
answer "NT".  Since NT is a dying OS, 'nt' is an increasingly
odd-looking thing to test agaist.

OTOH, I have no idea when or why sys.platform == "win32" would give a
different result than os.name == "nt".

> I still prefer hasattr(<module>, <attribute>) whenever applicable,
> e.g. preferring hasattr(os, 'fork') over os.name=='posix' (or
> os.name!='nt' :-), but sometimes that's not possible.

Like whether to use time.time or time.clock.

> What should be the preferred way?  (It may be impossible to say
> because there are different use cases, but probably one of the most
> important cases is simply distinguishing Windows from the rest -- how
> should that be done?)

My way <wink> -- unless it's broken in some way I haven't bumped into yet.
From rafaelemoranb at yahoo.es  Sat Aug  7 20:36:10 2004
From: rafaelemoranb at yahoo.es (Rafael Ernesto Moran Bautista)
Date: Sat Aug  7 20:36:12 2004
Subject: [Python-Dev] hi I am new in the list
Message-ID: <20040807183610.77951.qmail@web41410.mail.yahoo.com>

hi, my name is Rafael Morán and I am new in this list,
I hope help in this an interesting project called
Python.

I think could help in these topics: Translations (my
native language is spanish and know a little of
english), Python programming, C/C++ programming.

Att.
Rafael Morán


		
__________________________________
Do you Yahoo!?
Yahoo! Mail - Helps protect you from nasty viruses.
http://promotions.yahoo.com/new_mail
From mwh at python.net  Sat Aug  7 20:40:46 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Aug  7 20:40:48 2004
Subject: [Python-Dev] Recommended way to tell platform
In-Reply-To: <1f7befae04080711351466da96@mail.gmail.com> (Tim Peters's
	message of "Sat, 7 Aug 2004 14:35:25 -0400")
References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de>
	<200408071637.i77Gb0w10415@guido.python.org>
	<1f7befae04080711351466da96@mail.gmail.com>
Message-ID: <2mvffu3i6p.fsf@starship.python.net>

Tim Peters <tim.peters@gmail.com> writes:

> OTOH, I have no idea when or why sys.platform == "win32" would give a
> different result than os.name == "nt".

Probably never, but os.name == 'posix' covers a huge variety of sins,
so which one you test affects the 'feel' of the code somewhat.

Cheers,
mwh

-- 
  We did requirements and task analysis, iterative design, and user
  testing. You'd almost think programming languages were an interface
  between people and computers.                    -- Steven Pemberton
          (one of the designers of Python's direct ancestor ABC)
From tjreedy at udel.edu  Sat Aug  7 20:45:32 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat Aug  7 20:45:39 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
References: <ECBA357DDED63B4995F5C1F5CBE5B1E874FB36@its-xchg4.massey.ac.nz>
	<ECBA357DDED63B4995F5C1F5CBE5B1E86C36A8@its-xchg4.massey.ac.nz>
Message-ID: <cf380e$9rl$1@sea.gmane.org>


"Tony Meyer" <t-meyer@ihug.co.nz> wrote in message
news:ECBA357DDED63B4995F5C1F5CBE5B1E86C36A8@its-xchg4.massey.ac.nz...
>If the syntax does end up along these lines (and I'm expressing no opinion
>either way), then maybe it would be worth explicitly telling people (i.e.
in
>PEP 8) that they ought to put a blank line before decorators to make it
>clear?

Does PEP 8 tell people that they should put a blank before comment lines
introducing a function?

>Same deal for "don't do '@this @and_this @on_the_same_line'"
> (unless that loses it's validity).

In PEP 8, maybe, but not the Ref Manual (unless grammatically prohibited).
While uglier, this form does not stike me as being wholely different from
the quasilist proposals that would spell the above [this, and_this,
on_the_same_line].

Terry J. Reedy






From pf_moore at yahoo.co.uk  Sat Aug  7 21:13:37 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sat Aug  7 21:13:26 2004
Subject: [Python-Dev] Re: pre-PEP: Complete,
	Structured Regular Expression Group Matching
References: <411429FB.6080603@gradient.cis.upenn.edu>
	<lkd1xijedzx.fsf@panix2.panix.com>
	<brhn9guy.fsf@yahoo.co.uk> <lkdzn56aljx.fsf@panix2.panix.com>
Message-ID: <brhmn4m6.fsf@yahoo.co.uk>

Mike Coleman <mkc@mathdogs.com> writes:

> I suspect that in this case a pure Python implementation might
> actually be a lot harder than implementing it as a C patch. This is
> because so much of the work is already done in the current re
> module; the C patch would probably not be that big. In order to do
> it in Python I'd end up reimplementing re.match in Python, which I
> wouldn't look forward to.

Fairy nuff. I was assuming (on no particular basis - you've obviously
looked into this in a lot more detail than me) that you could
implement it using re.match - but on reflection, I can see that this
would probably not be true.

> It's because of this that I'm trying to get as much preliminary
> comment as I can.

That's reasonable. As I said before, I have a tendency to avoid the re
module too much (a perfect recent example was some code I wrote
splitting a string on a ":" character, then laboriously trimming each
part - I really should have done re.split on \s*:\s*). On that basis,
your examples give me a mixed reaction - the results impress me with
the ability to get exactly the right information out in a single
parse, but the regular expressions needed look utterly unmaintainable.

I suppose I'm saying that if I liked re's, I'd love this feature. As
it is, it leaves me looking for the same feature but with a better
pattern syntax.

But your examples reinforce for me that "parse" is a reasonable name
for the method.

Paul.
-- 
Home computers are being called upon to perform many new functions,
including the consumption of homework formerly eaten by the dog --
Doug Larson

From jhylton at gmail.com  Sat Aug  7 21:29:19 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Sat Aug  7 21:29:25 2004
Subject: [Python-Dev] new failing test -- test_compiler
Message-ID: <e8bf7a53040807122937ba5105@mail.gmail.com>

I added a test for the compiler package, because I realized it would
easily fall out of date if the language changed (say, decorators were
added) and the compiler package wasn't updated.  I added a test with a
resource, so it would only fail if you run regrtest.py -u all.  I
think that's a little lame, but better than letting us get into beta
without getting the compiler package fixed.

At any rate, the test become somewhat more interesting than I expected
because it fails in an unexpected way:

Fatal Python error: deletion of interned string failed
Aborted (core dumped)

My best guess is that this is a stack overflow.  In particular, gdb
shows that the failure is 388 stack levels deep.  The compiler package
has very deep function calls, because it uses recursion to traverse
abstract syntax trees.  (I know, I know, we should just rewrite the
compiler to use iterator instead of recursion <wink>.)  Inside one of
those deeply nested calls, it triggers GC which attempts to free a
deeply nested tree that is now garbage.  I think that's where it blows
up.  If anyone has suggestions for debugging, I'm all ears.

Jeremy
From mwh at python.net  Sat Aug  7 21:53:00 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Aug  7 21:53:01 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test
 test_builtin.py, 1.33, 1.34
In-Reply-To: <E1BtWjf-0002Q8-Ew@sc8-pr-cvs1.sourceforge.net>
	(jhylton@users.sourceforge.net's
	message of "Sat, 07 Aug 2004 12:20:07 -0700")
References: <E1BtWjf-0002Q8-Ew@sc8-pr-cvs1.sourceforge.net>
Message-ID: <2mr7qi3eub.fsf@starship.python.net>

jhylton@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Lib/test
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9225/Lib/test
>
> Modified Files:
> 	test_builtin.py 
> Log Message:
> Subclasses of string can no longer be interned.  The semantics of
> interning were not clear here -- a subclass could be mutable, for
> example -- and had bugs.  Explicitly interning a subclass of string
> via intern() will raise a TypeError.  Internal operations that attempt
> to intern a string subclass will have no effect.
>
> Added a few tests to test_builtin that includes the old buggy code and
> verifies that calls like PyObject_SetAttr() don't fail.  Perhaps these
> tests should have gone in test_string.

Would have been nice if you'd run the test suite before checking this
in... test_descr contains stuff like:

    s = madstring("x y")
    vereq(s, "x y")
    verify(intern(s).__class__ is str)
    verify(intern(s) is intern("x y"))
    vereq(intern(s), "x y")

though if madstring was mad enough to redefine __hash__, maybe it
wouldn't.

Cheers,
mwh

-- 
  > It might get my attention if you'd spin around in your chair,
  > spoke in tongues, and puked jets of green goblin goo.
  I can arrange for this.  ;-)            -- Barry Warsaw & Fred Drake
From tim.peters at gmail.com  Sat Aug  7 21:53:22 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 21:53:24 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <e8bf7a53040807122937ba5105@mail.gmail.com>
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
Message-ID: <1f7befae0408071253544d2c31@mail.gmail.com>

[Jeremy Hylton]
> I added a test for the compiler package, because I realized it would
> easily fall out of date if the language changed (say, decorators were
> added) and the compiler package wasn't updated.  I added a test with a
> resource, so it would only fail if you run regrtest.py -u all.  I
> think that's a little lame, but better than letting us get into beta
> without getting the compiler package fixed.
>
> At any rate, the test become somewhat more interesting than I expected
> because it fails in an unexpected way:
>
> Fatal Python error: deletion of interned string failed
> Aborted (core dumped)
> 
> My best guess is that this is a stack overflow.

I assume you ran this on Linux.  If so, and it is a stack overflow,
then it's very peculiar that it dies in much the same way on Windows:

"""
$ regrtest.py -uall test_compiler
test_compiler
Fatal Python error: deletion of interned string failed

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
"""

Anyone know who the support team for Python might be?

> In particular, gdb shows that the failure is 388 stack levels deep.

I've seen Zope gdb stack traces a lot deeper than that, so it's not
necessarily indicative of a blown stack.

> The compiler package has very deep function calls, because it uses
> recursion to traverse abstract syntax trees.  (I know, I know, we should just
> rewrite the compiler to use iterator instead of recursion <wink>.)  Inside one of
> those deeply nested calls, it triggers GC which attempts to free a
> deeply nested tree that is now garbage.  I think that's where it blows
> up.  If anyone has suggestions for debugging, I'm all ears.

Same thing on Windows.  This is the string it's trying to delete:

"abcdfeghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_"

If that's the string it's trying to delete for you too, my suggestion
is that you find out why <wink>.  Seriously, it smells more like "a
real bug" to me.

	op->ob_refcnt = 3;
	if (PyDict_DelItem(interned, op) != 0)
		Py_FatalError(
			"deletion of interned string failed");

sets the refcount to 3 first, and the refcount is 4 when the program
dies.  That means PyDict_DelItem took its

	if (ep->me_value == NULL) {
		PyErr_SetObject(PyExc_KeyError, key);
		return -1;
	}

path.  It's all consistent on Windows with a non-stack-overflowing
genuine error.
From tim.peters at gmail.com  Sat Aug  7 21:59:43 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 21:59:47 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <1f7befae0408071253544d2c31@mail.gmail.com>
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
	<1f7befae0408071253544d2c31@mail.gmail.com>
Message-ID: <1f7befae04080712597b9e7e0b@mail.gmail.com>

> This is the string it's trying to delete:
> 
> "abcdfeghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_"

FYI, that string appears only in shlex.py (split across two source
lines).  Maybe that will help whittle it down.
From tim.peters at gmail.com  Sat Aug  7 22:13:19 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 22:13:24 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <1f7befae04080712597b9e7e0b@mail.gmail.com>
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
	<1f7befae0408071253544d2c31@mail.gmail.com>
	<1f7befae04080712597b9e7e0b@mail.gmail.com>
Message-ID: <1f7befae04080713137fc55b79@mail.gmail.com>

BTW, think it's just coincidence that you also checked in a patch
changing something-or-other having to do with string interning?  I do
<wink>.
From mwh at python.net  Sat Aug  7 22:14:53 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Aug  7 22:14:56 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <1f7befae04080713137fc55b79@mail.gmail.com> (Tim Peters's
	message of "Sat, 7 Aug 2004 16:13:19 -0400")
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
	<1f7befae0408071253544d2c31@mail.gmail.com>
	<1f7befae04080712597b9e7e0b@mail.gmail.com>
	<1f7befae04080713137fc55b79@mail.gmail.com>
Message-ID: <2mn0163dtu.fsf@starship.python.net>

Tim Peters <tim.peters@gmail.com> writes:

> BTW, think it's just coincidence that you also checked in a patch
> changing something-or-other having to do with string interning?  I do
> <wink>.

We found that looking for the cause of the test_compiler problems.  It
doesn't seem to be related.

Cheers,
mwh

-- 
  (Unfortunately, while you get Tom Baker saying "then we 
   were attacked by monsters", he doesn't flash and make 
  "neeeeooww-sploot" noises.)
      -- Gareth Marlow, ucam.chat, from Owen Dunn's review of the year
From mwh at python.net  Sat Aug  7 22:47:22 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Aug  7 22:47:24 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <1f7befae04080712597b9e7e0b@mail.gmail.com> (Tim Peters's
	message of "Sat, 7 Aug 2004 15:59:43 -0400")
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
	<1f7befae0408071253544d2c31@mail.gmail.com>
	<1f7befae04080712597b9e7e0b@mail.gmail.com>
Message-ID: <2misbu3cbp.fsf@starship.python.net>

Tim Peters <tim.peters@gmail.com> writes:

>> This is the string it's trying to delete:
>> 
>> "abcdfeghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_"
>
> FYI, that string appears only in shlex.py (split across two source
> lines).  Maybe that will help whittle it down.

Here's some code from Lib/compiler/transformer.py:

    def atom_string(self, nodelist):
        k = ''
        for node in nodelist:
            k += self.decode_literal(node[1])
        n = Const(k)
        n.lineno = nodelist[0][2]
        return n

what does this make you think of?  Yes, it's the new += string
optimizations; Python gets upset if you mutate interned strings...

Armin is on the case...

Cheers,
mwh

-- 
  Programming languages should be designed not by piling feature on
  top of feature, but by removing the weaknesses and restrictions
  that make the additional features appear necessary.
               -- Revised(5) Report on the Algorithmic Language Scheme
From tim.peters at gmail.com  Sat Aug  7 22:56:46 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 22:56:49 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <1f7befae04080712597b9e7e0b@mail.gmail.com>
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
	<1f7befae0408071253544d2c31@mail.gmail.com>
	<1f7befae04080712597b9e7e0b@mail.gmail.com>
Message-ID: <1f7befae04080713564dd7086e@mail.gmail.com>

Weird.  The very first time PyString_InternInPlace() sees

"abcdfeghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_"

it's *already* marked as SSTATE_INTERNED_MORTAL .  So
PyString_InternInPlace() just returns, without adding it to the
interned dict.  The error we get later is inevitable then.

This happens while compile.c's PyCode_New is doing its "Intern
selected string constants" loop.

PyString_InternInPlace()  is the only thing that marks a string as
being in the SSTATE_INTERNED_MORTAL state, is it's A Mystery how this
string got so marked.  Unless it's a wild store.
From amk at amk.ca  Sat Aug  7 22:57:03 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Sat Aug  7 22:57:23 2004
Subject: [Python-Dev] Removing stuff from 2.4
Message-ID: <20040807205703.GA3501@rogue.amk.ca>

Hunting for deprecated modules.

* Deprecated in 2.1: TERMIOS.py, whrandom.py.  
* Deprecated in 2.2: mpz module, statcache module.  

  There are also various 2.2-deprecated methods in the email package;
  Barry can decide if he wants to bother with them or not.

* Deprecated in 2.3: xreadlines module, rotor module, rfc822 module,
  mimetools module, mimify module, MimeWriter module.  

  Only rotor and xreadlines raise a DeprecationWarning in 2.3, so
  they're the only candidates for removal.  The other modules, except
  for xreadlines, are marked as deprecated in PEP4; should we make
  them trigger DeprecationWarnings?

My proposal: 
* Remove TERMIOS, mpz, statcache, xreadlines, rotor.
* Make rfc822 raise PendingDeprecationWarning (so nothing will happen in 2.4
  unless you have -Wa)
* Make mimetools, mimify, Mimewrite raise DeprecationWarnings.  (This 
  is the part I'm least sure about; comments from people who know about
  these modules would be appreciated.  Should they raise Pending instead?)

I don't know what to do about whrandom, if anything; remove it?  Leave
it for 2.5?  suggestions?

--amk
From guido at python.org  Sat Aug  7 23:05:14 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug  7 23:05:21 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: Your message of "Sat, 07 Aug 2004 20:02:34 +0200."
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se> 
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org> 
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se> 
Message-ID: <200408072105.i77L5Ea10965@guido.python.org>

> I just wanted to say that I believe it should be allowed to decorate
> classes. There's not reason enough (reading the thread linked to by the
> PEP) to limit decorators this way. Like said several times in that
> thread, metaclasses are harder (to write and to understand) than
> decorators.
> 
> Also, considering that you said you would have actually wanted
> docstrings above the definition of a function, wouldn't this apply
> to classes as well?
> 
> Anyway, it's an artificial limit and it would be better to be able
> to test it out during the alpha.

Are you volunteering to implement it?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Sat Aug  7 23:07:28 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug  7 23:07:37 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: Your message of "Sat, 07 Aug 2004 14:25:09 EDT."
	<1f7befae040807112543238881@mail.gmail.com> 
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<200408071127.04568.heikowu@ceosg.de>
	<4114B226.4020405@interlink.com.au> 
	<1f7befae040807112543238881@mail.gmail.com> 
Message-ID: <200408072107.i77L7TM10984@guido.python.org>

> Indeed, for me that's a reason to prefer | over @.  BTW, while the PEP
> wasn't clear on this point last I looked, I hope it allows for
> whitespace after the | (or @, or =, or whatever it is).

The @ is a token by itself so there's nothing to stop you to insert
whitespace after it.  The PEP should make this clear though.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Sat Aug  7 23:11:08 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug  7 23:11:15 2004
Subject: [Python-Dev] Recommended way to tell platform
In-Reply-To: Your message of "Sat, 07 Aug 2004 14:35:25 EDT."
	<1f7befae04080711351466da96@mail.gmail.com> 
References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de>
	<200408071637.i77Gb0w10415@guido.python.org> 
	<1f7befae04080711351466da96@mail.gmail.com> 
Message-ID: <200408072111.i77LB8q10997@guido.python.org>

> [Guido]
> > This does bring up an interesting question: what's the recommended
> > way to distinguish a certain platform?  There's os.name,
> > sys.platform, the platform module, and I've also seen code testing
> > for os.sep=='/'.

[Tim]
> I ask myself that question every time I need to do it.  Out of sheer
> momentum now, I almost always use
> 
>     if sys.platform in ('win32',):

Why the 'in singleton tuple'?  Still holding out for that compiler
optimization that promises to make tuples of constants code constants
themselves?

> I dislike using os.name == 'nt', mostly because if we asked all
> Windows Python users which OS they use, only a tiny percentage would
> answer "NT".  Since NT is a dying OS, 'nt' is an increasingly
> odd-looking thing to test agaist.

And so is 'posix' when you're looking at Linux. :-)

We can't change these values easily so they will look odder and odder
as time progresses.

> OTOH, I have no idea when or why sys.platform == "win32" would give
> a different result than os.name == "nt".

Maybe on cygwin?  I really don't know.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Sat Aug  7 23:14:00 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug  7 23:14:06 2004
Subject: [Python-Dev] Re: @decorators, the PEP and the "options" out there?
In-Reply-To: Your message of "Sat, 07 Aug 2004 14:45:32 EDT."
	<cf380e$9rl$1@sea.gmane.org> 
References: <ECBA357DDED63B4995F5C1F5CBE5B1E874FB36@its-xchg4.massey.ac.nz>
	<ECBA357DDED63B4995F5C1F5CBE5B1E86C36A8@its-xchg4.massey.ac.nz>
	<cf380e$9rl$1@sea.gmane.org> 
Message-ID: <200408072114.i77LE0b11018@guido.python.org>

> >If the syntax does end up along these lines (and I'm expressing no
> >opinion either way), then maybe it would be worth explicitly
> >telling people (i.e. in PEP 8) that they ought to put a blank line
> >before decorators to make it clear?
> 
> Does PEP 8 tell people that they should put a blank before comment
> lines introducing a function?

No, but it tells them to *separate* methods by blank lines.  So that's
pretty clear guidance for where the put blank lines relative to the
decorators.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From anthony at interlink.com.au  Sat Aug  7 23:19:32 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Aug  7 23:19:51 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <20040807205703.GA3501@rogue.amk.ca>
References: <20040807205703.GA3501@rogue.amk.ca>
Message-ID: <41154764.60809@interlink.com.au>

A.M. Kuchling wrote:
> * Make rfc822 raise PendingDeprecationWarning (so nothing will happen in 2.4
>   unless you have -Wa)

A few bits and pieces in the stdlib still use bits from rfc822. They all
mostly look pretty shallow, except:

> * Make mimetools, mimify, Mimewrite raise DeprecationWarnings.  (This 
>   is the part I'm least sure about; comments from people who know about
>   these modules would be appreciated.  Should they raise Pending instead?)

mimetools.Message subclasses rfc822.Message, and mimetools.Message is
used extensively in the std lib.



-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From barry at python.org  Sat Aug  7 23:27:05 2004
From: barry at python.org (Barry Warsaw)
Date: Sat Aug  7 23:27:10 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <20040807205703.GA3501@rogue.amk.ca>
References: <20040807205703.GA3501@rogue.amk.ca>
Message-ID: <1091914025.1064.48.camel@anthem.wooz.org>

On Sat, 2004-08-07 at 16:57, A.M. Kuchling wrote:
> Hunting for deprecated modules.
> 
> * Deprecated in 2.1: TERMIOS.py, whrandom.py.  
> * Deprecated in 2.2: mpz module, statcache module.  
> 
>   There are also various 2.2-deprecated methods in the email package;
>   Barry can decide if he wants to bother with them or not.

I'll probably remove those.  We're bumping the email package's own
version number to 3.

> My proposal: 
> * Remove TERMIOS, mpz, statcache, xreadlines, rotor.
> * Make rfc822 raise PendingDeprecationWarning (so nothing will happen in 2.4
>   unless you have -Wa)

+1

> * Make mimetools, mimify, Mimewrite raise DeprecationWarnings.  (This 
>   is the part I'm least sure about; comments from people who know about
>   these modules would be appreciated.  Should they raise Pending instead?)

DeprecationWarnings on mimify and MIMEWriter.  Not sure atm, what to do
about mimetools, but it's clearly destined for death.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040807/9b9165b5/attachment.pgp
From tim.peters at gmail.com  Sat Aug  7 23:31:31 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 23:31:33 2004
Subject: [Python-Dev] Recommended way to tell platform
In-Reply-To: <200408072111.i77LB8q10997@guido.python.org>
References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de>
	<200408071637.i77Gb0w10415@guido.python.org> 
	<1f7befae04080711351466da96@mail.gmail.com>
	<200408072111.i77LB8q10997@guido.python.org>
Message-ID: <1f7befae04080714317191726b@mail.gmail.com>

>> [Tim] almost always use
>>
>>      if sys.platform in ('win32',):

[Guido]
> Why the 'in singleton tuple'?

Just because it often turns out that someone else needs to add another
platform later.  Like

        if sys.platform in ('win32', 'os2emx', 'mac'):
            # There's no distinction among 'user', 'group' and 'world';
            # replicate the 'user' bits.
            user = expected >> 6
            expected = user * (1 + 8 + 64)

> Still holding out for that compiler optimization that promises to make tuples of
> constants code constants themselves?

I take it you haven't checked compile.c lately <wink>.

>> I dislike using os.name == 'nt', mostly because if we asked all
>> Windows Python users which OS they use, only a tiny percentage would
>> answer "NT".  Since NT is a dying OS, 'nt' is an increasingly
>> odd-looking thing to test agaist.

> And so is 'posix' when you're looking at Linux. :-)
> 
> We can't change these values easily so they will look odder and odder
> as time progresses.

Which is why I prefer using something with 'win32' in its name now. 
That's equally good for Windows 95, 98, 98SE, ME, NT, 2000, XP.  Never
mind that it also says 'win32' on Win64, since nobody uses the latter
yet.

>> OTOH, I have no idea when or why sys.platform == "win32" would give
>> a different result than os.name == "nt".
 
> Maybe on cygwin?  I really don't know.

Nope.  sys.platform is 'cygwin' there, and os.name is 'posix'.  So
it's impossible to mistake Cygwin for a real Windows app <wink>.
From python at rcn.com  Sat Aug  7 11:32:48 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sat Aug  7 23:34:37 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <20040807205703.GA3501@rogue.amk.ca>
Message-ID: <002401c47c61$813f55a0$e841fea9@oemcomputer>

> I don't know what to do about whrandom, if anything; remove it?  Leave
> it for 2.5?  suggestions?

>From PEP4:  "#This module has been documented as obsolete since
                   Python 2.1, but listing in this PEP was neglected.
                   The deprecation warning will be added to the module
                   one year after Python 2.3 is released, and the
                   module will be removed one year after that."

Though the module is useless, I think we have to follow the PEP.


Raymond

From tim.peters at gmail.com  Sat Aug  7 23:42:07 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug  7 23:42:09 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <002401c47c61$813f55a0$e841fea9@oemcomputer>
References: <002401c47c61$813f55a0$e841fea9@oemcomputer>
Message-ID: <1f7befae040807144233fee779@mail.gmail.com>

[Andrew]
>> I don't know what to do about whrandom, if anything; remove it?  Leave
>> it for 2.5?  suggestions?

[Raymond]
> From PEP4:  "#This module has been documented as obsolete since
>                   Python 2.1, but listing in this PEP was neglected.
>                   The deprecation warning will be added to the module
>                   one year after Python 2.3 is released, and the
>                   module will be removed one year after that."
>
> Though the module is useless, I think we have to follow the PEP.

whrandom should definitely get a DeprecationWarning for 2.4.  2.3
final was released in July of 2003, and it's already a year after.
From amk at amk.ca  Sat Aug  7 23:44:44 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Sat Aug  7 23:45:03 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <1f7befae040807144233fee779@mail.gmail.com>
References: <002401c47c61$813f55a0$e841fea9@oemcomputer>
	<1f7befae040807144233fee779@mail.gmail.com>
Message-ID: <20040807214444.GA3798@rogue.amk.ca>

On Sat, Aug 07, 2004 at 05:42:07PM -0400, Tim Peters wrote:
> whrandom should definitely get a DeprecationWarning for 2.4.  2.3
> final was released in July of 2003, and it's already a year after.

Done.

--amk
From tjreedy at udel.edu  Sat Aug  7 23:57:57 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat Aug  7 23:58:06 2004
Subject: [Python-Dev] Re: Re: Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de><20040805215331.GB31288@burma.localdomain><ceuk0e$rut$1@sea.gmane.org><1091806774.8414.63.camel@localhost><1f7befae040806090644be9c04@mail.gmail.com><4113AF47.1010405@interlink.com.au><200408061634.i76GYEZ07623@guido.python.org>
	<cf1kr3$qe4$1@sea.gmane.org>
	<200408071612.i77GCcK10255@guido.python.org>
Message-ID: <cf3j97$fqv$1@sea.gmane.org>


"Guido van Rossum" <guido@python.org> wrote in message
news:200408071612.i77GCcK10255@guido.python.org...
>Do we have to add 'currently' to every statement about the language?

Of course not.  I would draw the line with one more.

> I really want to take a hard stance on this

In that case, perhaps we should draw the line at one less and remove the
current 'currently'.  Then, no one like me could cite it as precedent;-).

> I don't think their argument would have a chance in court.

Depends on the lawyers.  As their lawyer, I would argue something like the
followihng:  1. Programming, like the law, requires careful writing.  In
addition to the C code, the Python Reference Manual is also carefully
written.  2. While anything and everything could theoretically change in
the next version, there are implicit and explicit promises that that will
not happen.  Also, some things are *much* more stable than others.  3. The
Python manual uses 'forbidden' rarely.  Its use is never necessary.  Since
the Ref Man is carefully written, its use must be intentional, to convey a
nuance of meaning.  The rational inference is that the nuance is that of
considerably more permanence than for other things.

Enough, I think.

Terry J. Reedy



From arigo at tunes.org  Sun Aug  8 00:23:57 2004
From: arigo at tunes.org (Armin Rigo)
Date: Sun Aug  8 00:29:03 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <2misbu3cbp.fsf@starship.python.net>
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
	<1f7befae0408071253544d2c31@mail.gmail.com>
	<1f7befae04080712597b9e7e0b@mail.gmail.com>
	<2misbu3cbp.fsf@starship.python.net>
Message-ID: <20040807222357.GA27723@vicky.ecs.soton.ac.uk>

Hi,

On Sat, Aug 07, 2004 at 09:47:22PM +0100, Michael Hudson wrote:
>         k = ''
>         for node in nodelist:
>             k += self.decode_literal(node[1])
> 
> what does this make you think of?  Yes, it's the new += string
> optimizations; Python gets upset if you mutate interned strings...
> 
> Armin is on the case...

Ahem, oups, yes, I am to blame for this memory corruption bug.  In the +=
string optimization I forgot that the string's ob_refcnt field is not always
"correct".  String interning messes with its value.  What occurred here is
that we mutated a string which had only one reference left -- that is, one
apart from the two that stringobject.c artificially removes to ob_refcnt when
interning.

This is now fixed by skipping the optimization if we got an interned string.  
While messing with ob_refcnt is always a bit fragile I am still convinced that
the += optimization is safe again.


A bient?t,

Armin.
From mkc at mathdogs.com  Sun Aug  8 00:36:17 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Sun Aug  8 00:36:21 2004
Subject: [Python-Dev] Re: re.split on empty patterns
References: <20040807145142.GB2529@rogue.amk.ca>
Message-ID: <lkd4qner2xq.fsf@panix2.panix.com>

"A.M. Kuchling" <amk@amk.ca> writes:
> >>> re.split('x*', 'abxxxcdefxxx')
> ['ab', 'cdef', '']
> >>> re.split('x*', 'abxxxcdefxxx', emptyok=True)
> ['', 'a', 'b', '', 'c', 'd', 'e', 'f', '', '']
> 
> (I think the result of the second match points up a bug in the patch;
> the empty strings in the middle seem wrong to me.  Assume that gets
> fixed.)

I believe it's correct in the sense that it matches the design I had in mind.
Of course, this could be the wrong design choice.

I think (some of) the alternatives can be stated this way:

1.  Empty matches are not considered at all for splitting.  (present behavior)

2.  Empty matches are only considered when they are not adjacent to non-empty
    matches.  (what you have in mind, I think)

3.  Empty matches are always considered.  (the patch behavior)

(It's understood that matches that fall within other matches are not
considered.  For empty matches, an adjacent match doesn't count as "within".)

I would say that alternatives 2 and 3 are better than 1 because they retain
more information, and I would argue that alternative 3 is better than
alternative 2 for the same reason.  I don't have a killer example, but here is
a somewhat abstract one that shows the difference between 2 and 3:

    # alternative 2:
    re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', 'bbczz']
    re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz']

    # alternative 3:
    re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', '', 'bbczz']
    re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz']

In English, the third alternative allows you to notice that there were two
adjacent matches, even if the second match was empty.  With the second
alternative, this is missed.

Of course, because of the match algorithm, we cannot notice an empty match
immediately followed by a non-empty match, so there's kind of an asymmetry
here.  With alternative 2 that asymmetry wouldn't be present, since we'd fail
to notice empty matches on either side.

Alternative 2 does have the advantage of matching the expectations of a naive
user a little better.  Alternative 3 is more powerful, but perhaps a little
less obvious.

(If you're reading this, what do you think?)


> Anyway, we therefore can't just make this the default in 2.4.  We
> could trigger a warning when emptyok is not supplied and a split
> pattern results in a zero-length match; users could supply
> emptyok=False to avoid the warning.  Patterns that never have a
> zero-length match would never get the warning.  2.5 could then set
> emptyok to True.  

I like this compromise.  A variant would be to warn if the pattern *could*
match empty, rather than warning when it *does* match empty.  I'm not sure
whether it would be easy to determine this, though.

> Note: raising the warning might cause a serious performance hit for
> patterns that get zero-length matches a lot, which would make 2.4
> slower in certain cases.

The above variant would ameliorate this, though in theory no one should be
using patterns that can empty match anyway.

Cheers,
Mike

From mkc at mathdogs.com  Sun Aug  8 00:44:06 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Sun Aug  8 00:44:10 2004
Subject: [Python-Dev] Re: re.split on empty patterns
References: <20040807145142.GB2529@rogue.amk.ca>
	<1f7befae040807105878b47c61@mail.gmail.com>
Message-ID: <lkdzn56po09.fsf@panix2.panix.com>

Tim Peters <tim.peters@gmail.com> writes:
> Yes it is!  Or, more accurately, it can be, when it's intended to
> match an empty string.  It's a bit fuzzy because regexps are so
> error-prone, and writing a regexp that matches an empty string by
> accident is easy.

I won't dispute that regexps are error-prone (having just proved it recently),
but I would point out that any calls to the current re.split with a pattern
that can match empty are probably kind of fuzzy anyway.  Consider, for
example, that re.search doesn't have an option that says "find matches but
skip any empty matches".

Along the same lines, I think re.findall doesn't really do what its doc
claims.  The doc says

    Empty matches are included in the result unless they touch the beginning
    of another match.

but, for example

    >>> re.findall(r'(?=.)|b', 'abcabc')
    ['', '', '', '', '', '']
    [24592 refs]
    >>> re.findall(r'b|(?=.)', 'abcabc')
    ['', 'b', '', '', 'b', '']
    [24601 refs]

with a recent python.  It's not clear what "another match" would be here.  The
only way its beginning could be touched by an empty match would be if they
both started in the same position.  So presumably that other match could only
be discovered by asking the RE engine to "find me the first match in this
position that's not empty".  It looks like the *actual* method being used,
though, is more like the method used in my patch:

    for each position that's not interior to some previous match:
        find the first match at this position (if any)
             and add it to the list

We could find the "longest" match instead of the "first" match, but that would
be a lot more expensive in general, and not necessarily easier to understand.


> In the meantime, "emptyok" is an odd
> name since it's always "ok" to have an empty match.  "split0=True"
> reads better to me, since the effect is to split on a 0-length match. 
> split_on_empty_match would be too wordy.

I was thinking "ok" in the sense of "okay to split on them" (empty matches).
I don't have any strong preference on the name, though--I'd just like to get
the feature in relatively soon (with semantics that everyone's happy with).

('split0' does look slightly '3133t', though.  :-)

Cheers,
Mike

From python at rcn.com  Sat Aug  7 13:01:26 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sun Aug  8 01:03:15 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <20040807222357.GA27723@vicky.ecs.soton.ac.uk>
Message-ID: <003b01c47c6d$e2fc94e0$e841fea9@oemcomputer>

> the string's ob_refcnt field is not
> always
> "correct".  String interning messes with its value.  What occurred
here is
> that we mutated a string which had only one reference left -- that is,
one
> apart from the two that stringobject.c artificially removes to
ob_refcnt
> when
> interning.

Try adding a guard using s->ob_sstate.


Raymond

From ark-mlist at att.net  Sun Aug  8 02:03:45 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Sun Aug  8 02:03:51 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <cf1ulk$75r$1@sea.gmane.org>
Message-ID: <004501c47cdb$355743d0$6602a8c0@arkdesktop>

> > def foo():
> >    | classmethod
> >    | accepts(int,int)
> >    | returns(float)
> >
> 
> 
> Thats it!  I love it.

I like the following variation even better:

	def foo() |
		classmethod |
		accepts(int, int) |
		returns(float):

Alternatively,

	def foo() | classmethod | accepts(int, int) | returns(float):

Yes, I understand that the first of these would require allowing a newline
after the | without ending the statement.  If it were up to me, I would
allow a newline to follow any operator, but if that's too radical, then
allowing newlines between the def and the following : would be fine.

From tim.peters at gmail.com  Sun Aug  8 02:35:44 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Aug  8 02:35:47 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <20040807222357.GA27723@vicky.ecs.soton.ac.uk>
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
	<1f7befae0408071253544d2c31@mail.gmail.com>
	<1f7befae04080712597b9e7e0b@mail.gmail.com>
	<2misbu3cbp.fsf@starship.python.net>
	<20040807222357.GA27723@vicky.ecs.soton.ac.uk>
Message-ID: <1f7befae04080717357814d502@mail.gmail.com>

[Armin Rigo]
> ...
> This is now fixed by skipping the optimization if we got an interned string.
> While messing with ob_refcnt is always a bit fragile I am still convinced that
> the += optimization is safe again.

Ya, this bug wasn't scary (an unlikely but clear-in-hindsight problem,
specifically due to a subsystem lying about true refcounts -- not much
of that goes on, thank God).

Alas, test_compiler still fails (but in an ordinary "doesn't work"
way, no longer in an internal-error way), so testing with -uall is
still a bit broken.
From fperez528 at yahoo.com  Sun Aug  8 03:26:18 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Sun Aug  8 03:26:34 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<200408071617.i77GHgs10281@guido.python.org>
Message-ID: <cf3vg5$1m5$1@sea.gmane.org>

Guido van Rossum wrote:

>> I'm including both of the following variants:

[Let's call this option #1]
>>      |classmethod
>>      |accepts(int, int)
>>      |returns(float)
>>      def foo(arg1, arg2):
>>          ...

[And this option #2]
>>      def foo(arg1, arg2):
>>          |classmethod
>>          |accepts(int, int)
>>          |returns(float)
>>          ...

> In the discussion on decorators months ago, solutions involving
> special syntax inside the block were ruled out early on.  Basically,
> you shouldn't have to peek inside the block to find out important
> external properties of the function.  (Yes, I know that docstrings
> violate this principle.  For the record, it would be better if they
> were prefix too; and a prefix decorator syntax will let us fix this in
> the future but introducing a "doc" decorator.)

Allow me to try and make a reasoned argument against your point, and in favor of
#2 (as labeled above), even for docstrings.  The reason why I find #1 more
difficult to mentally parse, is that as you read the code, you need to
accumulate information in your head which is not yet tied to anything in
particular (the def line hasn't arrived yet).  

In my brain, the process goes something like this (imagine this happening in
code with multiple decorators, each with multiple lines of comments detailing
its use):

      |classmethod

ok, a class method is coming, but I don't know yet what it's called.  Put this
on the 'mental stack', let's see where it goes...

      |accepts(int, int)

and it will take integer arguments; more for the stack...

      |returns(float)

and return a float; add to the stack...

      def foo(arg1, arg2):

Ah! It's function foo, with 2 args named arg1, arg2.  Now I go back (mentally)
and wrap together my knowledge about it, unwinding my stack, producing foo, a
classmethod with arg1,arg2 as integers, and which returns a float.

I find this unpleasant and inefficient from a cognitive viewpoint, because our
brain is best at processing information when it has 'buckets' where to put it:
if you read the def line first, the function name becomes the 'bucket' where
all further information goes.  That's why it's always harder to remember random
bits of information than things which connect to other things you already know.

For these reasons, I feel that option 2 is much more efficient.  A similar
outline of my mental process with option 2:

>>      def foo(arg1, arg2):

Ok, a new function foo() goes in, it takes 2 args, arg1 and arg2.  Make mental
bucket labeled 'foo'.  Anything that comes now, until we de-indent, goes into
box 'foo'.  No need to keep random bits of information up in the air until I
know where they're supposed to go.

>>          |classmethod

Store 'classmethod' in bucket 'foo'.

>>          |accepts(int, int)

Store 'accepts(int, int)' in bucket 'foo'.

>>          |returns(float)

Store 'returns(float)' in bucket 'foo'.

Done.  All information is stored in mental bucket 'foo', where it belongs.

Bucket 'foo' was created when I saw the def line, nicely separated by
indentation from all else, which is what my python mental processor is already
well trained to do.  Then, all the information that came afterwards (including
possibly the docstring) fell nicely into that mental bucket.

Option 1, which you seem to favor, reminds me of a problem I felt when studying
German for the first time (Spanish is my native language).  German has these
funny 'split-verbs', like anrufen, which are used in a sentence split in two,
and with the initial part of the verb only appearing at the very end of the
sentence.  A simple example (I don't recall the really painful ones anymore) is
"Ich rufe dich an".  You don't know that the verb is anrufen, until you see the
'an' at the end of the sentence.  With long sentences, I remember feeling very
frustrated, because I esentially had to buffer the whole thing in my head with
very little structure, waiting for the last bit to come in.  Then I needed to
quickly reconstruct the meaning, now that I knew which verb was being used, by
mentally reparsing the whole thing.  It's been many years, but I distinctly
remember this as being a significant hurdle in spoken conversation in German.  

I honestly feel adding this kind of process to python would be a significant
reduction to the fluidity with which the language 'parses mentally'.

And I like the fact that this syntax:

def foo(...):
    @dec1
    @dec2
    "docstring"

    # code begins...

sort of declares a 'header zone' for the function, which is visually well
defined by indentation and string highlighting, but allows the 'def' part to be
very easy to spot visually because it cuts the indentation cleanly.  It feels
like a quite natural convention for me to think that until I see the closing of
the docstring, I'm in the 'header' or 'interface' zone of the function.

I hope I've at least presented a clear argument in defense of this choice. 
Whether you agree or not, is up to you.  I deliberatly didn't mention the @ vs
| issue, since for this discussion it's completely irrelevant.

Best,

f

From jason at tishler.net  Sun Aug  8 04:32:19 2004
From: jason at tishler.net (Jason Tishler)
Date: Sun Aug  8 04:25:41 2004
Subject: [Python-Dev] Cygwin and sockets
In-Reply-To: <4110AABA.80407@v.loewis.de>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E86C541C@its-xchg4.massey.ac.nz>
	<4110AABA.80407@v.loewis.de>
Message-ID: <20040808023219.GE55644@tishler.net>

Martin,

On Wed, Aug 04, 2004 at 11:22:02AM +0200, "Martin v. L?wis" wrote:
> Tony Meyer wrote:
> >By "anybody", did you mean anybody that you know & trust?  <0.5 wink>.
> >
> >I can duplicate the process outlined in the bug report and get the
> >same failure to compile (I've added a note to it).
> 
> This is good enough for the moment, although I would have preferred an
> answer like "I know, and I have a fix sitting on my harddisk" :-)

I've *just* returned from vacation.  I will look into this problem after
catching up and will report back when I can add something useful.

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6
From raymond.hettinger at verizon.net  Sat Aug  7 16:45:47 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun Aug  8 04:47:40 2004
Subject: [Python-Dev] decimal.Context.copy() shallow or deep?
Message-ID: <004f01c47c8d$3a7cb1e0$e841fea9@oemcomputer>

Currently, the copy() method for Context objects makes a shallow copy of
the context.  To get a deepcopy, you have to use
copy.deepcopy(thecontext).

Though, copies are usually shallow and deepcopies are deep, I'm now
having misgivings about the copy method.  I would think that a deep copy
is almost invariably what you would want and expect, (i.e. the copy
doesn't share traps and flags with the original).

Do you guys think it should be left alone (shallow) or changed (made
deep)?

The wrinkle in all this is that internally the module makes many uses of
copy and expects it to be shallow (or least, tests fail in droves when
copy is made deep).  So I would have to rename the current copy to
_shallow_copy and replace the original with a deep version for the
public API.

Any thoughts?


Raymond

From tim.peters at gmail.com  Sun Aug  8 04:53:40 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Aug  8 04:53:43 2004
Subject: [Python-Dev] decimal.Context.copy() shallow or deep?
In-Reply-To: <004f01c47c8d$3a7cb1e0$e841fea9@oemcomputer>
References: <004f01c47c8d$3a7cb1e0$e841fea9@oemcomputer>
Message-ID: <1f7befae040807195334227984@mail.gmail.com>

[Raymond Hettinger]
> Currently, the copy() method for Context objects makes a shallow copy of
> the context.  To get a deepcopy, you have to use
> copy.deepcopy(thecontext).
>
> Though, copies are usually shallow and deepcopies are deep, I'm now
> having misgivings about the copy method.  I would think that a deep copy
> is almost invariably what you would want and expect, (i.e. the copy
> doesn't share traps and flags with the original).

Hmm.  In my head (which I'm sure doesn't match reality, else you
wouldn't be saying this), sets of trap-enabled and it-happened sticky
flags are just integers with meanings assigned to the bits, as they
are in hardware and most APIs for controlling and querying FPU status.
 Then there's no difference between shallow and deep copying.

> Do you guys think it should be left alone (shallow) or changed (made
> deep)?

A user-visible copy certainly should not share mutable state with the
original.  It's possible that really argues for a simpler internal
representation of such state, though.

> The wrinkle in all this is that internally the module makes many uses of
> copy and expects it to be shallow (or least, tests fail in droves when
> copy is made deep).  So I would have to rename the current copy to
> _shallow_copy and replace the original with a deep version for the
> public API.
>
> Any thoughts?

Whatever works <wink>.
From python at rcn.com  Sat Aug  7 17:24:03 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sun Aug  8 05:26:41 2004
Subject: [Python-Dev] decimal.Context.copy() shallow or deep?
In-Reply-To: <1f7befae040807195334227984@mail.gmail.com>
Message-ID: <000401c47c92$b38c43c0$1619c797@oemcomputer>

> > Do you guys think it should be left alone (shallow) or changed (made
> > deep)?
> 
> A user-visible copy certainly should not share mutable state with the
> original. 

I was afraid of that.  What a PITA.


> It's possible that really argues for a simpler internal
> representation of such state, though.

No doubt that would be the right thing to do (doubly so because that is
how you would do it in C).  Unfortunately, there a ton of code that has
to change to get to an integer and bitmask approach.  Also, you have to
expose constants for masking.  Then, you still have to deal with the
underlying code liking the shallow copies.


> > The wrinkle in all this is that internally the module makes many
uses of
> > copy and expects it to be shallow (or least, tests fail in droves
when
> > copy is made deep).  So I would have to rename the current copy to
> > _shallow_copy and replace the original with a deep version for the
> > public API.
> >
> > Any thoughts?
> 
> Whatever works <wink>.


Done fast, right, and cheap -- pick any two.


Raymond

From martin at v.loewis.de  Sun Aug  8 09:02:21 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug  8 09:02:21 2004
Subject: [Python-Dev] Re: Recommended way to tell platform
In-Reply-To: <200408071637.i77Gb0w10415@guido.python.org>
References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de>
	<200408071637.i77Gb0w10415@guido.python.org>
Message-ID: <4115CFFD.1000309@v.loewis.de>

Guido van Rossum wrote:
> I still prefer hasattr(<module>, <attribute>) whenever applicable,
> e.g. preferring hasattr(os, 'fork') over os.name=='posix' (or
> os.name!='nt' :-), but sometimes that's not possible.
> 
> What should be the preferred way?  (It may be impossible to say
> because there are different use cases, but probably one of the most
> important cases is simply distinguishing Windows from the rest -- how
> should that be done?)

Precisely that: impossible to say. In most cases, you should not test
for the platform, but just use the platform functionality, and fall back
to something else if it isn't present.

However, to reliably distinguish Windows from the rest, check whether
os.platform is "win32".

Regards,
Martin
From martin at v.loewis.de  Sun Aug  8 09:22:32 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug  8 09:22:32 2004
Subject: [Python-Dev] Recommended way to tell platform
In-Reply-To: <1f7befae04080711351466da96@mail.gmail.com>
References: <20040807142327.GA2529@rogue.amk.ca>
	<4114E753.2050303@v.loewis.de>	<200408071637.i77Gb0w10415@guido.python.org>
	<1f7befae04080711351466da96@mail.gmail.com>
Message-ID: <4115D4B8.90405@v.loewis.de>

Tim Peters wrote:
> I dislike using os.name == 'nt', mostly because if we asked all
> Windows Python users which OS they use, only a tiny percentage would
> answer "NT".

A similar point holds for Linux, btw. os.name is "posix"; I doubt
many Linux (or, Solaris) users would say that the name of their OS
is "posix". It has the advantage over "nt" that "posix" is not dying.

Regards,
Martin
From eppstein at ics.uci.edu  Sun Aug  8 09:22:43 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Sun Aug  8 09:22:47 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <cf1ulk$75r$1@sea.gmane.org>
	<004501c47cdb$355743d0$6602a8c0@arkdesktop>
Message-ID: <eppstein-E36B6F.00224308082004@sea.gmane.org>

In article <004501c47cdb$355743d0$6602a8c0@arkdesktop>,
 "Andrew Koenig" <ark-mlist@att.net> wrote:

> I like the following variation even better:
> 
> 	def foo() |
> 		classmethod |
> 		accepts(int, int) |
> 		returns(float):
> 
> Alternatively,
> 
> 	def foo() | classmethod | accepts(int, int) | returns(float):
> 
> Yes, I understand that the first of these would require allowing a newline
> after the | without ending the statement.  If it were up to me, I would
> allow a newline to follow any operator, but if that's too radical, then
> allowing newlines between the def and the following : would be fine.

Did you omit the full wink?  Or did you forget that | is already a binop?
Strictly speaking I guess it's unambiguous if one doesn't allow 
arbitrary expressions as decorators, but...

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From martin at v.loewis.de  Sun Aug  8 09:28:42 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug  8 09:28:41 2004
Subject: [Python-Dev] hi I am new in the list
In-Reply-To: <20040807183610.77951.qmail@web41410.mail.yahoo.com>
References: <20040807183610.77951.qmail@web41410.mail.yahoo.com>
Message-ID: <4115D62A.4040204@v.loewis.de>

> I think could help in these topics: Translations (my
> native language is spanish and know a little of
> english), Python programming, C/C++ programming.

Hi Rafael,

Thanks for the offer. There is one area where we really could
use some help: patches and bugs. Please go to
sf.net/projects/python, and see whether you can review some
patches, and whether you can offer fixes for some of the bugs.
See

http://www.python.org/dev/dev_intro.html

for some procedural hints.

Regards,
Martin
From anthony at interlink.com.au  Sun Aug  8 09:59:22 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sun Aug  8 09:59:48 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <1091914025.1064.48.camel@anthem.wooz.org>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
Message-ID: <4115DD5A.2070108@interlink.com.au>

Barry Warsaw wrote:
> I'll probably remove those.  We're bumping the email package's own
> version number to 3.

The AddlistClass in email is marked as deprecated, but is deprecated
in favour of a class in rfc822!


>>* Make mimetools, mimify, Mimewrite raise DeprecationWarnings.  (This 
>>  is the part I'm least sure about; comments from people who know about
>>  these modules would be appreciated.  Should they raise Pending instead?)
> 
> 
> DeprecationWarnings on mimify and MIMEWriter.  Not sure atm, what to do
> about mimetools, but it's clearly destined for death.

mimetools is used in a number of other modules - including urllib
httplib cgi and urllib2. Worse, they use mimetools.Message, which
is a subclass of rfc822.Message. urllib/urllib2's addinfourl.info()
is documented as returning a mimetools.Message().

I mentioned this on IRC yesterday - I do not think we should put in
a PendingDeprecationWarning (or a DeprecationWarning) for anything
until we've stripped it's usage from the stdlib. If we can't be
bothered to update our code to no longer use the code we want to
dump, it hardly seems fair to expect our users to do so. I'd go
so far as to say that this should be explicitly stated in the PEP.

I also think that a "final" release should not issue PDWs from
the test suite (with the obvious exception of code that is explicitly
testing the PDWed feature.

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From scav at blueyonder.co.uk  Sun Aug  8 12:04:25 2004
From: scav at blueyonder.co.uk (Peter Harris)
Date: Sun Aug  8 11:50:29 2004
Subject: [Python-Dev] Re: Python-Dev Digest, Vol 13, Issue 77
In-Reply-To: <20040807132358.975AA1E4015@bag.python.org>
References: <20040807132358.975AA1E4015@bag.python.org>
Message-ID: <4115FAA9.7010707@blueyonder.co.uk>

Andrew Durdin wrote:

    Message: 6
    Date: Sat, 7 Aug 2004 22:36:59 +1000
    From: Andrew Durdin <adurdin@gmail.com>
    Subject: [Python-Dev] Decorators with arguments are curries!
    To: python-dev@python.org
    Message-ID: <59e9fd3a04080705361b6314be@mail.gmail.com>
    Content-Type: text/plain; charset=US-ASCII

    Having waded through all the decorator syntax discussions recently,
    and some of the historical ones, I haven't found this point mentioned
    before, so I'll bring it up now:

    Consider the following code:

    def foo(bar):
    return 1

    a = foo
    a = foo(bar)

    The first assignment to a is binding a reference to a function; the
    second is calling the function. This is a very significant difference
    in python, and I'm concerned that all the current proposed decorator
    syntaxes[*] are liable to cause confusion on this point. For example:

    def foo_decorator(func):
    print "no params to this"
    return func

    def bar_decorator(func, param):
    print param
    return func

    @foo_decorator
    @bar_decorator("one param here")
    def decorated_func():
    pass

    Here the first decorator statement is bare, while the second one
    includes parentheses and an argument; the first one looks like a
    function reference, while the second looks like a function call.
    I find this confusing; semantically, this appears to equivalent to
    (assuming expressions are allowed in decorator statements):

    # using curry as proposed in PEP 309 [**]
    @foo_decorator
    @curry(bar_decorator, "one param here")
    def decorated_func():
    pass

    Most of my concern here is that this aspect of decorator syntax
    appears to be implicitly introducing a currying syntax in one special
    circumstance, which is then *not* transferable to currying functions
    in normal situations, as it would conflict with function calling. And
    if a currying syntax (or built-in) was introduced down the track, then
    these decorators would be inconsistent with them.

    Has anyone else noticed this issue?

    Andrew Durdin

    [*] Except those which don't allow for arguments
    [**] For a nice coincidence, PEP 309 suggested using '@' to mark curries

Hmm.  I sort of see what you're on about, but please note:

PEP309 no longer proposes (and didn't for long) using '@'.

The callable class in PEP309 is called partial() not curry() because
it's not technically a curried function in the narrow computer-science
sense, and if we mis-named it the Haskell folks would point and laugh.

You could use partial() on decorators as well as functions and classes,
and though it would melt my brain, I'm sure someone out there would see
a beautifully elegant use for it.

PEP309 is accepted but not checked in;  @ decorators are checked in but
tentative.

On topic:   @ is OK but I prefer the recently suggested |.
  pie, pipe = (+0.5, +1)

Peter Harris

From rnd at onego.ru  Sun Aug  8 12:09:25 2004
From: rnd at onego.ru (Roman Suzi)
Date: Sun Aug  8 12:08:38 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <4114583C.4070500@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com>
	<4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com>
	<4114583C.4070500@v.loewis.de>
Message-ID: <Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>


On Sat, 7 Aug 2004, [ISO-8859-1] "Martin v. L?wis" wrote:
>
>> But must it be "@", such an eye-catching, ugly piece
>> of flie-dirt?
>
>If you don't like the current proposal, try to find objective
>reasons, in addition to the subjective ones. Also, try to come
>up with counter-proposals that are well thought-out, instead of
>being created in a rush. In particular, if all you are concerned
>with is the specific choice of operator, propose a different one.

BTW, hat is wrong with my %-proposal:

def foo(self) % decor():
   body

and

def foo(self) % (decor(), decor1(args), ...):
   body


- it seems to satisfy the needs.

This form could also be added (for consistency):

def foo(self):   # old way
   body

foo %= decor()

or

foo %= (decor(), decor1(args), ...)

or

foo = foo % (decor())


+ and this doesnt require to add new symbol (@),
+ it is semantically consistant with % meaning as a formatting
("decorating") operation, although in case of def the part
being decorated is on LHS, while in string formatting arguments
are being "decorated" according to the format string.

I want to hear the critics of this as I see this a better syntax
to the decoration needs!

>Regards,
>Martin

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From rnd at onego.ru  Sun Aug  8 12:16:28 2004
From: rnd at onego.ru (Roman Suzi)
Date: Sun Aug  8 12:15:53 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <004501c47cdb$355743d0$6602a8c0@arkdesktop>
References: <004501c47cdb$355743d0$6602a8c0@arkdesktop>
Message-ID: <Pine.LNX.4.58.0408081410420.6199@rnd.onego.ru>

On Sat, 7 Aug 2004, Andrew Koenig wrote:

>> > def foo():
>> >    | classmethod
>> >    | accepts(int,int)
>> >    | returns(float)
>> >
>>
>>
>> Thats it!  I love it.
>
>I like the following variation even better:
>
>	def foo() |
>		classmethod |
>		accepts(int, int) |
>		returns(float):
>
>Alternatively,
>
>	def foo() | classmethod | accepts(int, int) | returns(float):
>
>Yes, I understand that the first of these would require allowing a newline
>after the | without ending the statement.  If it were up to me, I would
>allow a newline to follow any operator, but if that's too radical, then
>allowing newlines between the def and the following : would be fine.

I just anted to rewrite those "real-life-looking" examples with my proposed
'%' syntax:

def foo() % (classmethod, accepts(int,int), returns(float),):

or formatted:

def foo() % (
   classmethod,
   accepts(int,int),
   returns(float),):

Or it can be done the this way:

def foo():
   ...

foo %= (classmethod, accepts(int,int), returns(float),)

The operation could be called "decorative apply", if the LHS
has __call__ attribute. Just a new form of % operation with
special syntax for "def" operator.


Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From martin at v.loewis.de  Sun Aug  8 12:23:49 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug  8 12:23:49 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de>
	<41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
Message-ID: <4115FF35.4030608@v.loewis.de>

Roman Suzi wrote:
>>If you don't like the current proposal, try to find objective
>>reasons, in addition to the subjective ones. Also, try to come
>>up with counter-proposals that are well thought-out, instead of
>>being created in a rush. In particular, if all you are concerned
>>with is the specific choice of operator, propose a different one.
> 
> 
> BTW, hat is wrong with my %-proposal:

Wrt our discussion: it does not just replace the choice of
character; it also replaces the position where the decorators
are placed.

Taken on its own, it is similar to C2 and C3 in

http://www.python.org/moin/PythonDecorators

and as such, it shares the same problems. Feel free to add
it as C4 if you care.

> I want to hear the critics of this as I see this a better syntax
> to the decoration needs!

Read the Wiki.

Regards,
Martin
From marco at bubke.de  Sun Aug  8 08:52:10 2004
From: marco at bubke.de (Marco Bubke)
Date: Sun Aug  8 12:40:38 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <cf4ibo$p1n$1@sea.gmane.org>

Edward Loper wrote:

> The new "vertical bar" syntax was proposed recently, and has gotten
> almost unanimously positive feedback.  In particular, of 17 responses
> to the syntax, only one has been negative (see table, below).
> 
> I'm including both of the following variants:
> 
>      |classmethod
>      |accepts(int, int)
>      |returns(float)
>      def foo(arg1, arg2):
>          ...
> 
>      def foo(arg1, arg2):
>          |classmethod
>          |accepts(int, int)
>          |returns(float)
>          ...

I really like the second one maybe with '@'. For me a decorator is a
property of the function like the attributes. I think this is function foo
with two arguments, its a classmethod etc.. The function name and the
arguments are much more important for me than the decorators. So they
should be go after the doc string.

Marco


From rnd at onego.ru  Sun Aug  8 13:07:11 2004
From: rnd at onego.ru (Roman Suzi)
Date: Sun Aug  8 13:06:20 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <4115FF35.4030608@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com>
	<4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com>
	<4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
	<4115FF35.4030608@v.loewis.de>
Message-ID: <Pine.LNX.4.58.0408081431050.6199@rnd.onego.ru>

On Sun, 8 Aug 2004, [ISO-8859-1] "Martin v. L?wis" wrote:

>Roman Suzi wrote:
>>
>> BTW, hat is wrong with my %-proposal:
>
>Wrt our discussion: it does not just replace the choice of
>character; it also replaces the position where the decorators
>are placed.
>
>Taken on its own, it is similar to C2 and C3 in
>
>http://www.python.org/moin/PythonDecorators
>
>and as such, it shares the same problems. Feel free to add
>it as C4 if you care.

Yes, this is like C3 when '%' is used instead of "using" (pseudo)keyword.
As for the problems, I do not agree that decorators visually mix with
arguments: it is not a problem with +-s and *-s in an expression or other
similar cases. Also: heavily decorated function/method probably tells
that something is wrong with it's design.

BTW, IMHO, "A" case has more drawbacks than "C" cases. I also like
%-syntax being used in |-syntax style (E2).

>> I want to hear the critics of this as I see this a better syntax
>> to the decoration needs!
>
>Read the Wiki.

Done.

>Regards,
>Martin
>

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From gregory.lielens at fft.be  Sun Aug  8 13:08:30 2004
From: gregory.lielens at fft.be (Gregory Lielens)
Date: Sun Aug  8 13:08:55 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <Pine.LNX.4.58.0408081410420.6199@rnd.onego.ru>
Message-ID: <000001c47d38$0ce96740$1000000a@chikubi>

> I just anted to rewrite those "real-life-looking" examples 
> with my proposed '%' syntax:
> 
> def foo() % (classmethod, accepts(int,int), returns(float),):
> 
> or formatted:
> 
> def foo() % (
>    classmethod,
>    accepts(int,int),
>    returns(float),):
> 
> Or it can be done the this way:
> 
> def foo():
>    ...
> 
> foo %= (classmethod, accepts(int,int), returns(float),)
> 
> The operation could be called "decorative apply", if the LHS 
> has __call__ attribute. Just a new form of % operation with 
> special syntax for "def" operator.
> 
> 
> Sincerely yours, Roman Suzi

Yes, you should add this proposal in the wiki
(http://www.python.org/cgi-bin/moinmoin/PythonDecorators) just after
"C3. tuple-after-def syntax with a (pseudo-)keyword".
Your idea is basically the same with a % instead of a pseudo-keyword...
For me, +1 for these two (and your justification for using % is
convincing for me, especially as it is difficult to agree on a best
choice for the pseudo-keyword to use in C2, C3 so you are my current
favorite ;-))

Having followed the thread, my own summary (that didn't change much
these last days) is that all before-def variants are prefered because it
is more aesthetically pleasing (I do not agree ;-) ) and it is more
related to the "exterior" behavior of the function/method (i.e. seen
from the caller perspective) than to it's internal behavior (i.e. seen
from the implementer of the function, or when examining function
algorithm).
This last argument is nice and made me re-evaluate my gut feeling, but
in the end it was not sufficient for changing my preference for the
various C proposal (with C3 and yours beeing my favorites). 
Indeed, adding decorators (or any modifier/metadata) in front of a
declaration is a valid way to proceed of course, but feel not in line
with the rest of python syntax, and that is what bother me the most, it
feel too much like an ad-hoc special casing...And anyway,
function/method name and arguments are at least as important as
decorator from a caller perspective, which for me mitigate a lot the
argument that decorators should be in front: In a C++ like syntax, yes,
sure, but with Python syntax it does not fit well in the general picture
imho...

Maybe I'll change my mind after a while, if before-def syntax is
adopted, but for this it would help if this type of construct could be
used not only for decorators (and with | instead of @ - pleaaase ;-) ),
so that "special case" does not flash into my head each time I see it
;-)

Regards,

Greg.


From gregory.lielens at fft.be  Sun Aug  8 13:24:22 2004
From: gregory.lielens at fft.be (Gregory Lielens)
Date: Sun Aug  8 13:24:30 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <Pine.LNX.4.58.0408081410420.6199@rnd.onego.ru>
Message-ID: <000101c47d3a$44400d00$1000000a@chikubi>

Oups, saw you just added it, a C4 proposal :-)

One remark though, I would far prefer if the implicit linebreak is not
done, it is a special casing again, for little readability benefit
imho...
I appreciate consistency for syntax, as special cases often causes small
syntax errors for me, that have to be corrected and remove part of the
fun in programming: I really like to have a Python that reads like
pseudo-code and have a nice coherent syntax easy to remember, sure it is
maybe trivial compared to other things that makes a good programming
langage but this is what stand out for me first time I saw a python
script...

Regards,

Greg.


From martin at v.loewis.de  Sun Aug  8 13:39:39 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug  8 13:39:38 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <Pine.LNX.4.58.0408081431050.6199@rnd.onego.ru>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de>
	<41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
	<4115FF35.4030608@v.loewis.de>
	<Pine.LNX.4.58.0408081431050.6199@rnd.onego.ru>
Message-ID: <411610FB.9040201@v.loewis.de>

Roman Suzi wrote:

> As for the problems, I do not agree that decorators visually mix with
> arguments: it is not a problem with +-s and *-s in an expression or other
> similar cases. Also: heavily decorated function/method probably tells
> that something is wrong with it's design.

There is another drawback of course: it does not have an implementation.

Regards,
Martin
From anthony at interlink.com.au  Sun Aug  8 13:52:49 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sun Aug  8 13:53:17 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>	<200408061557.i76FvGo07481@guido.python.org>	<4113BC98.3020407@stackless.com>	<4113CD18.8080803@v.loewis.de>
	<41140E76.2050601@stackless.com>	<4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
Message-ID: <41161411.2060804@interlink.com.au>

Roman Suzi wrote:
> On Sat, 7 Aug 2004, [ISO-8859-1] "Martin v. L?wis" wrote:
> 
>>>But must it be "@", such an eye-catching, ugly piece
>>>of flie-dirt?
>>
>>If you don't like the current proposal, try to find objective
>>reasons, in addition to the subjective ones. Also, try to come
>>up with counter-proposals that are well thought-out, instead of
>>being created in a rush. In particular, if all you are concerned
>>with is the specific choice of operator, propose a different one.
> 
> 
> BTW, hat is wrong with my %-proposal:
> 
> def foo(self) % decor():
>    body

Guido has already ruled out the decorators-after-function-arguments
form. See the last week's archives for python-dev - thread subject
was "def fn (args) [dec,dec]:"


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From jim at zope.com  Sun Aug  8 14:18:40 2004
From: jim at zope.com (Jim Fulton)
Date: Sun Aug  8 14:18:44 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu>
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <41161A20.1080505@zope.com>

Edward Loper wrote:
> The new "vertical bar" syntax was proposed recently, and has gotten
> almost unanimously positive feedback. 

OK, I'll try to fix that. ;)

-1

But, to me, all of the variations on "special character + decorator"
feel like a foreign language.   It reminds me of languages like
PHP that let you mix HTML and some scripting language. It looks like
someone is trying to use 2 separate languages.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From rnd at onego.ru  Sun Aug  8 15:03:14 2004
From: rnd at onego.ru (Roman Suzi)
Date: Sun Aug  8 15:02:25 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <41161411.2060804@interlink.com.au>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com>
	<4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com>
	<4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
	<41161411.2060804@interlink.com.au>
Message-ID: <Pine.LNX.4.58.0408081701080.7067@rnd.onego.ru>

On Sun, 8 Aug 2004, Anthony Baxter wrote:

>> BTW, hat is wrong with my %-proposal:
>>
>> def foo(self) % decor():
>>    body
>
>Guido has already ruled out the decorators-after-function-arguments
>form. See the last week's archives for python-dev - thread subject
>was "def fn (args) [dec,dec]:"

Hmmm... Maybe it's time to introduce better lambdas? ;)

foo = decor(
  def foo(self): ...
    ...
)


Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From rnd at onego.ru  Sun Aug  8 15:11:37 2004
From: rnd at onego.ru (Roman Suzi)
Date: Sun Aug  8 15:10:44 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <411610FB.9040201@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com>
	<4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com>
	<4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
	<4115FF35.4030608@v.loewis.de>
	<Pine.LNX.4.58.0408081431050.6199@rnd.onego.ru>
	<411610FB.9040201@v.loewis.de>
Message-ID: <Pine.LNX.4.58.0408081703430.7067@rnd.onego.ru>

On Sun, 8 Aug 2004, [ISO-8859-1] "Martin v. L?wis" wrote:

>Roman Suzi wrote:
>
>> As for the problems, I do not agree that decorators visually mix with
>> arguments: it is not a problem with +-s and *-s in an expression or other
>> similar cases. Also: heavily decorated function/method probably tells
>> that something is wrong with it's design.
>
>There is another drawback of course: it does not have an implementation.

Well, GvR convinced me decorators-before-def are needed, thus
leaving my proposal void.

However, I am not happy with those @s.

>Regards,
>Martin

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From jim at zope.com  Sun Aug  8 15:53:46 2004
From: jim at zope.com (Jim Fulton)
Date: Sun Aug  8 15:53:51 2004
Subject: [Python-Dev] Thoughts on validation, events, rules
Message-ID: <4116306A.1000904@zope.com>

I'm a bit unsatisfied with container constraints:

   http://dev.zope.org/Zope3/ContainmentConstraints

While I think this was a step forward conceptually, it has turned out
to be a bit unwieldy in practice:

- Container constraints are difficult to express.  I've written
   convenience API's to make the basic machinery of expressing
   constraints easier, but there is a problem that's not so easy to
   fix.  Often you want to express a symmetric constraint between two
   interfaces.  For example, you want to say that objects implementing
   I1 can only contain objects implementing I2 and that objects
   implementing I2 can only be contained in objects implementing I1.
   If the constraint is expressed in the interfaces themselves, you can
   only express it by either defining the interfaces in a single
   operation or by mutating the interfaces.  I don't like the idea of
   mutating interfaces, but perhaps I just need to get over that.

- Container constraints are cumbersome to check.

- Container constraints force the contained objects to implement
   IContained. This is fine when the contained objects will actually
   access their containers.  Often, however, we want to constrain an
   object's container for external reasons.  For example, we might want
   to only allow indexes to be added to catalogs, even though indexes
   are passive and never invoke operations on their containers. In this
   case, the constraint is not part of the index's contract.

I've been thinking of other ways to solve this problem.

In the context of schema validation, we've been thinking of using
subscription adapters:

   http://dev.zope.org/Zope3/NoMoreSchemaBinding

   http://mail.zope.org/pipermail/zope3-dev/2004-June/011315.html

The advantage of subscription adapters is that they provide the
ability to express constraints in a highly flexible and decentralized
manner.  Another advantage of using subscription adapters is that
doing so takes advantage of a highly optimized mechanism and should be
fairly efficient.

It occurred to me today that subscription adapters provide a form of
rule-based processing.  Any number of rules (subscribers) can fire as
the result of some event, where an event could take the form of a
query, such as "validate adding this object to this container" or
"validate setting this attribute to this value".

It's interesting to note that validating adding an object to a
container and validating assigning an attribute are both special cases
of validating establishing a relationship between two objects.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jim at zope.com  Sun Aug  8 15:58:32 2004
From: jim at zope.com (Jim Fulton)
Date: Sun Aug  8 15:58:35 2004
Subject: [Python-Dev] Re: Thoughts on validation, events, rules
In-Reply-To: <4116306A.1000904@zope.com>
References: <4116306A.1000904@zope.com>
Message-ID: <41163188.4080407@zope.com>

Sorry, I sent this to the wrong list.  I've been typing python-dev
a lot lately. :)

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From adurdin at gmail.com  Sun Aug  8 16:17:23 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Sun Aug  8 16:17:26 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <Pine.LNX.4.58.0408081703430.7067@rnd.onego.ru>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com>
	<4113CD18.8080803@v.loewis.de> <41140E76.2050601@stackless.com>
	<4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
	<4115FF35.4030608@v.loewis.de>
	<Pine.LNX.4.58.0408081431050.6199@rnd.onego.ru>
	<411610FB.9040201@v.loewis.de>
	<Pine.LNX.4.58.0408081703430.7067@rnd.onego.ru>
Message-ID: <59e9fd3a04080807177dd446af@mail.gmail.com>

Having just gone through the wiki page again, I think the last example
under A (quoted below) shows up very plainly the only really serious
problem with the before-def decorator syntax:

class TestDecorators(unittest.TestCase):

    ...

    def test_dotted(self):
        decorators = MiscDecorators()
        @decorators.author('Cleese')
        def foo(): return 42
        self.assertEqual(foo(), 42)
        self.assertEqual(foo.author, 'Cleese')

Suppose that you were not familiar enough with python to have heard of
decorators: looking at this code, would you have any idea that the @
statement (potentially) fundamentally affects the behaviour of the
foo() function? Is there anything except their proximity to suggest
that they are interdependent? The only implication that they *might*
be is the second assertEqual call, and that is by no means obvious.

On the other hand, the after-def (but before-colon) variants clearly
imply that the decorators are related to the function definition --
the reader might not understand their purpose, but they will not fail
to connect them with the function.

And here is a response to Guido's comments on the list-after-def
syntax in another thread that were then incorporated into the wiki
page.

Guido> - it hides crucial information (e.g. that it is a static method)
  after the signature, where it is easily missed

Not at all! The list of decorators would *be* a part of the signature.
The decorators will be no more easily missed than the name of the last
argument, which is arguably just as crucial.

Guido> - it's easy to miss the transition between a long argument list and a
  long decorator list

True. This is the main reason why having the list set off from the
argument list with a keyword or symbol would be preferred; but
suitable formatting would minimise the risk anyway -- see the example
further below.

Guido> - it's cumbersome to cut and paste a decorator list for reuse, because
  it starts and ends in the middle of a line

This is (IMHO) a very weak argument. A decorator list with one or two
decorators will probably be just as quick to type as to copy and paste
with the before-def syntax, when the time to scroll between lines is
taken into account (unless you're using a mouse, in which case
highlighting within a line is not slow at all). Moreover, in more
complicated decorator lists, the lists would likely be different
(reducing the advantage of copy+paste); or if not, if you regularly
use the same long decorator list, you're much better off defining your
own decorator which applies those from the list, and then using that
one repeatedly -- this will avoid errors and make maintenance easier.

Guido> Given that the whole point of adding decorator syntax is to move the
decorator from the end ("foo = staticmethod(foo)" after a 100-line
body) to the front, where it is more in-your-face, it should IMO be
moved all the way to the front.

I disagree. Having the decorator at the end, after the function body,
is obviously far from ideal, as it is extremely easy to miss; but the
decorators are not the most important part of the function definition.
The most important part (which should be most prominent, and thus
first) is the "def" keyword introducing the function definition, and
the function name. Whether the decorators are more or less important
than the arguments (and thus which should come first) is less certain.

The example of a long decorated function declaration that Guido gives was:

def longMethodNameForEffect(longArgumentOne=None,
                            longArgumentTwo=42) [
    staticmethod,
    funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
              status="experimental", author="BDFL")
    ]:

I don't think this is as clear as it could be (and an argument from
the "ugliness" of syntax should be using the clearest possible
formatting) -- breaking the line before the decorator list, and
indenting the decorator list one more level (to set it off from the
docstring/body) makes the whole much clearer, and prevents the
decorator list from being confused with the arguments:

def longMethodNameForEffect(longArgumentOne=None,
                            longArgumentTwo=42) \
        [staticmethod,
        funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
                  status="experimental", author="BDFL")]:

(Having the [ and ]: on their own lines at the same indentation level
may be clearer yet)

-- Andrew Durdin.
From jhylton at gmail.com  Sun Aug  8 16:26:33 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Sun Aug  8 16:26:35 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <1f7befae04080717357814d502@mail.gmail.com>
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
	<1f7befae0408071253544d2c31@mail.gmail.com>
	<1f7befae04080712597b9e7e0b@mail.gmail.com>
	<2misbu3cbp.fsf@starship.python.net>
	<20040807222357.GA27723@vicky.ecs.soton.ac.uk>
	<1f7befae04080717357814d502@mail.gmail.com>
Message-ID: <e8bf7a53040808072658e6b779@mail.gmail.com>

On Sat, 7 Aug 2004 20:35:44 -0400, Tim Peters <tim.peters@gmail.com> wrote:
> [Armin Rigo]
> > ...
> > This is now fixed by skipping the optimization if we got an interned string.
> > While messing with ob_refcnt is always a bit fragile I am still convinced that
> > the += optimization is safe again.
> 
> Ya, this bug wasn't scary (an unlikely but clear-in-hindsight problem,
> specifically due to a subsystem lying about true refcounts -- not much
> of that goes on, thank God).
> 
> Alas, test_compiler still fails (but in an ordinary "doesn't work"
> way, no longer in an internal-error way), so testing with -uall is
> still a bit broken.

I won't be able to fix it today, but I'll get on that asap.

Jeremy
From jason at tishler.net  Sun Aug  8 16:54:42 2004
From: jason at tishler.net (Jason Tishler)
Date: Sun Aug  8 16:47:51 2004
Subject: [Python-Dev] Cygwin and sockets
In-Reply-To: <4110AABA.80407@v.loewis.de>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E86C541C@its-xchg4.massey.ac.nz>
	<4110AABA.80407@v.loewis.de>
Message-ID: <20040808145442.GF55644@tishler.net>

Martin,

On Wed, Aug 04, 2004 at 11:22:02AM +0200, "Martin v. L?wis" wrote:
> Tony Meyer wrote:
> >By "anybody", did you mean anybody that you know & trust? <0.5 wink>.
> >
> >I can duplicate the process outlined in the bug report and get the
> >same failure to compile (I've added a note to it).

The following:

    http://sf.net/tracker/?group_id=5470&atid=105470&func=detail&aid=728330

broke CVS for Cygwin too.

> This is good enough for the moment, although I would have preferred an
> answer like "I know, and I have a fix sitting on my harddisk" :-)

I know, and I have a fix sitting on my harddisk. :,)

See attach for the details.  OK to apply?

BTW, the socket module builds automatically for me.  This may be because
I have not upgraded to the latest Cygwin binutils due to the following
issue:

    http://cygwin.com/ml/cygwin/2004-07/msg00997.html

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6
-------------- next part --------------
Index: Modules/socketmodule.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/socketmodule.c,v
retrieving revision 1.298
diff -u -p -r1.298 socketmodule.c
--- Modules/socketmodule.c	3 Aug 2004 08:52:45 -0000	1.298
+++ Modules/socketmodule.c	8 Aug 2004 14:29:33 -0000
@@ -285,6 +285,10 @@ int h_errno; /* not used */
 #include "addrinfo.h"
 #endif
 
+#if defined(__CYGWIN__)
+#include "addrinfo.h"
+#endif
+
 #ifndef HAVE_INET_PTON
 int inet_pton(int af, const char *src, void *dst);
 const char *inet_ntop(int af, const void *src, char *dst, socklen_t size);
From guido at python.org  Sun Aug  8 17:25:19 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Aug  8 17:25:24 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: Your message of "Mon, 09 Aug 2004 00:17:23 +1000."
	<59e9fd3a04080807177dd446af@mail.gmail.com> 
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de>
	<41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
	<4115FF35.4030608@v.loewis.de>
	<Pine.LNX.4.58.0408081431050.6199@rnd.onego.ru>
	<411610FB.9040201@v.loewis.de>
	<Pine.LNX.4.58.0408081703430.7067@rnd.onego.ru> 
	<59e9fd3a04080807177dd446af@mail.gmail.com> 
Message-ID: <200408081525.i78FPJm20862@guido.python.org>

> Having just gone through the wiki page again, I think the last example
> under A (quoted below) shows up very plainly the only really serious
> problem with the before-def decorator syntax:
> 
> class TestDecorators(unittest.TestCase):
> 
>     ...
> 
>     def test_dotted(self):
>         decorators = MiscDecorators()
>         @decorators.author('Cleese')
>         def foo(): return 42
>         self.assertEqual(foo(), 42)
>         self.assertEqual(foo.author, 'Cleese')
> 
> Suppose that you were not familiar enough with python to have heard of
> decorators: looking at this code, would you have any idea that the @
> statement (potentially) fundamentally affects the behaviour of the
> foo() function? Is there anything except their proximity to suggest
> that they are interdependent? The only implication that they *might*
> be is the second assertEqual call, and that is by no means obvious.

This an example of bad formatting.  Tests often do that because their
purpose is to test odd corners of functionality, not serve as sample
code.  Given that in typical example code there will always be a blank
line separating the decorator from anything previous (and separating
the body from what follows!) the connection between the decorator and
the def is pretty clear.

Also note that arguments based on partial knowledge of Python can be
used as an argument against pretty much anything (just tweak the
assumptions a bit), and hence have little value.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From mark at prothon.org  Sun Aug  8 00:55:53 2004
From: mark at prothon.org (Mark Hahn)
Date: Sun Aug  8 18:20:39 2004
Subject: [Python-Dev] Decimal type question [Prothon]
Message-ID: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>

When I first announced Prothon, Tim Peters told me to feel free to post on
Python lists because he considered Prothon a possible sandbox to learn
from.  So far I have interpreted this to mean c.l.p but I have been
dissapointed in the quality of technical responses.  Several people have
suggested I post here instead.  Let me know if you think I should go back
to c.l.p.

Prothon is considering using a floating Decimal type to fix the confusion
around binary floating point mentioned in some of the Python pitfalls,
warts, & gotchas.  We are not considering total replacement of the binary
Float because it would be a waste of the floating point hardware
performance, and because the decimal implementation we are considering
would be inferior in range.

Our proposal it to add a new Decimal type "between" Int and Float. It would
have less precision than the infinite precision of Int (Prothon uses Longs
for Int), but more than Float.  The Decimal would have less range than
either Ints or Floats though (although 56 orders of magnitude is more range
than is usually used in Ints).

The Decimal type in .Net, which will be the VM for the first Prothon
release, has a precision of 28 decimal digits and has a range of 7e-28 to
7e28.  Decimal calculations that overflowed the range of 7e28 to 7e-28
(like 0.0000_0000_0000_0000_0000_0000_0000_7 / 2) would throw an exception
instead of automatically converting to Float.

Numeric constants containing only a decimal point would become a Decimal
type: 1.354, 3452.7862.  Any constant with the letter e would be a normal
binary Float type:  1e-3, 1.23e62.  Int divide "i/j" would always create a
Decimal.  Decimal would only support the basic ops: +, -, *, /, and simple
funcs like min, max, and abs.  The math module would automatically coerce
Decimal arguments to Float for things like sin and log, just as it does
with Int arguments.

This would still solve the gotcha:  0.1 + 0.1 would equal 0.2 exactly.  For
that matter, any constant or expression (not including a divide) that
didn't overflow/underflow the 28 digit precision would have an exact
representation.

Can anyone see any problem with this scheme?


From ark-mlist at att.net  Sun Aug  8 18:21:35 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Sun Aug  8 18:21:26 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <eppstein-E36B6F.00224308082004@sea.gmane.org>
Message-ID: <003101c47d63$c67b7330$6602a8c0@arkdesktop>

> Did you omit the full wink?  Or did you forget that | is already a binop?
> Strictly speaking I guess it's unambiguous if one doesn't allow
> arbitrary expressions as decorators, but...

Neither, actually.  It didn't occur to me that there might be an ambiguity
any more than it occurs to me that f(3, 4) could be ambiguous in C because
the comma might be an operator.  I imagined the following rule:  If a
statement begins with "def", then every | outside parentheses delimits a
decorator.  If you insist on using an expression with | in it as a
decorator, you have to parenthesize it.

I can see now that even though this suggestion isn't really syntactically
fuzzy, it might be considered too fussy.  Still, I like it better than the
prefix @ idea.

I even realize why I don't like the @ syntax.  It's not so much reserving @
as it is that it's a prefix syntax.  It is ingrained in me that "def" starts
a function definition, and now it doesn't any more.  So, for example, if I
use my editor to search for "def", I have to back up to see whether I've
missed something before it.

From pf_moore at yahoo.co.uk  Sun Aug  8 18:35:16 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sun Aug  8 18:35:05 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<200408071617.i77GHgs10281@guido.python.org>
	<cf3vg5$1m5$1@sea.gmane.org>
Message-ID: <vfft1tbv.fsf@yahoo.co.uk>

Fernando Perez <fperez528@yahoo.com> writes:

>>>      def foo(arg1, arg2):
>
> Ok, a new function foo() goes in, it takes 2 args, arg1 and arg2.
> Make mental bucket labeled 'foo'. Anything that comes now, until we
> de-indent, goes into box 'foo'. No need to keep random bits of
> information up in the air until I know where they're supposed to go.

Add a double-take here, because this is a method whose first argument
isn't "self". We can't be sure it's not right yet, because of the
staticmethod decorator, but store that for later thinking about...

>
>>>          |classmethod
>
> Store 'classmethod' in bucket 'foo'.

Oops. If we recall, foo doesn't have a "self" first argument, but
neither does it have "cls". probably an error somewhere here, but we
have to wait until the decorators are over to be sure.

>
>>>          |accepts(int, int)
>
> Store 'accepts(int, int)' in bucket 'foo'.
>
>>>          |returns(float)
>
> Store 'returns(float)' in bucket 'foo'.
>
> Done. All information is stored in mental bucket 'foo', where it
> belongs.

But we have a couple of inconsistencies to backtrack and resolve. (And
the accepts/returns stuff is also inconsistent, but I'll stop picking
holes in what is clearly just an example...)

I'm not saying that the same issues mightn't exist with the
decorator-before-def form, but I don't think the mental model for the
decorator-after-def form is quite as clean as you'd like either...

It's still subjective :-)

Paul
-- 
The major difference between a thing that might go wrong and a thing
that cannot possibly go wrong is that when a thing that cannot
possibly go wrong goes wrong it usually turns out to be impossible to
get at or repair. -- Douglas Adams

From tim.peters at gmail.com  Sun Aug  8 18:46:09 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Aug  8 18:46:12 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <e8bf7a53040808072658e6b779@mail.gmail.com>
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
	<1f7befae0408071253544d2c31@mail.gmail.com>
	<1f7befae04080712597b9e7e0b@mail.gmail.com>
	<2misbu3cbp.fsf@starship.python.net>
	<20040807222357.GA27723@vicky.ecs.soton.ac.uk>
	<1f7befae04080717357814d502@mail.gmail.com>
	<e8bf7a53040808072658e6b779@mail.gmail.com>
Message-ID: <1f7befae0408080946767c0a1c@mail.gmail.com>

[Tim]
>> Alas, test_compiler still fails (but in an ordinary "doesn't work"
>> way, no longer in an internal-error way), so testing with -uall is
>> still a bit broken.

[Jeremy]
> I won't be able to fix it today, but I'll get on that asap.

That's OK.  I agree it's valuable to run this, so I fixed it. 
test_compiler passes on Windows now.  But I have a gigabyte of RAM on
this box to hold the stack <wink>.
From tismer at stackless.com  Sun Aug  8 18:48:56 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sun Aug  8 18:46:24 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <41145A7E.8010500@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<1f7befae040806090644be9c04@mail.gmail.com>	<cf0gd1$smv$1@sea.gmane.org>	<1091818207.7529.34.camel@localhost>	<4113D5E1.6040006@interlink.com.au>
	<41141255.5010408@stackless.com> <41145A7E.8010500@v.loewis.de>
Message-ID: <41165978.3010107@stackless.com>

Martin v. L?wis wrote:

> Christian Tismer wrote:
> 
>> Ok, I dislike special prefix chars at all, in a language that
>> doesn't have this concept elsewhere (despite strings of course,
>> but their prefixes are just regular chars),
> 
> 
> Lists, tuples, and dictionaries are also introduced with non-letter
> characters. Furthermore, nested expressions use the same prefix
> char that lists use. # introduces comments.

I agree on the "#".
Not so for lists, tuples and dicts. These are constructs in
their own right, containing something, which is again made
of normal Python objects.

But what we here have is an escape symbol, which expresses that
the token following it is special and has special semantics,
and the reach of this special symbol then even extends over its
own line, but modifies how the next def statement is understood.
This is remarkably different from being usual.

I should not have talked about prefix chars, but escape sequences,
which fits much better.

As somebody already mentioned (Jim Fulton I think) this feels
like mixing two different languages into each other.

all the best -- chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Sun Aug  8 19:04:35 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sun Aug  8 19:02:03 2004
Subject: [Python-Dev] capturing RETURN_VALUE
Message-ID: <41165D23.4070202@stackless.com>

Dear list,

just by chance, I discovered a probably very unknown
feature of the block stack machinery of Python's frames:

--  It is possible to abandon a return statement.  --

My question: Is this an artifact, or can I rely on this
feature to persist for a while (while while >= 1 year) ?

Example code:
I have a code body which either computes and returns a value,
or it raises an exception. Surrounding it, I have a while
true loop, embracing a try..finally block. In case of an
exception, it simply lets the exception pass. In case of a
return value, it intercepts it and can do anything (in the
example, it just decorates the value and returns).

def return_capture(n, do_raise=False):
	_have_value = False
	# outer while loop, just there to get cancelled
	while True:
		try:
			# function body, returning a value or raising
			if do_raise:
				raise ValueError, "was told to raise"
			# signal valid value
			_have_value = True
			retval = n*n
			return
		finally:
			if _have_value:
				break
	# here we end after a captured return, and can do postprocessing
	retval = ("here the captured return value", retval)
	return retval

Sample output:
 >>> return_capture(5)
('here the captured return value', 25)
 >>> return_capture(5, True)
Traceback (most recent call last):
   File "<interactive input>", line 1, in ?
   File "<interactive input>", line 8, in return_capture
ValueError: was told to raise
 >>>


Again the question: Artifact, or a reliable feature?

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From tismer at stackless.com  Sun Aug  8 20:30:30 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sun Aug  8 20:28:00 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <4114583C.4070500@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de>
	<41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de>
Message-ID: <41167146.4070401@stackless.com>

Martin v. L?wis wrote:

> Christian Tismer wrote:

[dropping things that I trashed while implementing]

>> To save *some* typing (laziness was also a driver for this
>> whole decorator nightmare), we could also remove the assignment
>> and just have a function call like
>>
>> classmethod(myfunc)
> 
> 
> This breaks backward compatibility. This is already legal
> syntax, and should raise a NameError if myfunc is not
> defined (actually, this holds for the first proposal, too).

This is correct. I failed to understand that, initially,
but when trying an implementation, I also found out that
I need something to make this unambiguous. Here it is.

What I added was yet another __stuff__ entry for new-style
classes. It may also appear on module level, like the
__metaclass__ feature works.

So what I implemented was special treatment for decorators
appearing this way:

class MyClass(object):

     __decorators__ = classmethod, staticmethod, otherdeco


     classmethod(mymethod)

     def mymethod(self, *args, **kwds):
         ...

The implementation intercepts all top-level occurances
of objects mentioned in the __decorators__ sequence
and treats the according statements as comments.
They are all collected through the whole class definition,
and put into an extra code object which is executed
after the class code has finished.
The generated post-code is of the form

     mymethod = classmethod(mymethod)

and for nested cases

     mymethod = otherdeco(classmethod(mymethod))

...

> Your proposals are all by no means little. They are significant,
> counter-intuitive changes of the language.

What I depicted above doesn't seem so counter-intuitive to me.
I don't claim it is *the* solution for the future, but it
is a compromize which gives me the desired effect, without
introducing new syntax, just a new __special_entry__. This
makes obvious that this is a temporary feature which should
be replaced by a better construct at some time, like __metaclass__
and __slots__ will probably, too.
But it provides a solution without enforcing us to decide
about new syntax right now, which appears to be too early
for a group that can't find a common denominator, yet.

> Also, try to come
> up with counter-proposals that are well thought-out, instead of
> being created in a rush.

No rush, just incomplete.
A side effect of this idea is that the decorator(funcname) items
can appear wherever you like them in the class definition.
I don't know if this is good. It might get used to advantage
if we use decorators to define interface-like stuff, for instance.
Then, a series of interface decorator "calls" might be put at
the top of the class.

My "implementation" is just a proof of concept. I didn't patch
the code generator yet or hacked bytecode, but did a little
source code transformation with he desired effect.
This is enough to get a feeling for proof-of-concept.
A drawback of this implementation is that the decorators
are taken literally. It won't work if you rename them
during the class definition.

For my purposes it works fine, but I don't claim that it fulfils
all other desired use cases. At least it is compatible, simple,
complete, implemented in a way, and works for my applications.

kind regards -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From edreamleo at charter.net  Sun Aug  8 22:58:24 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Sun Aug  8 22:58:32 2004
Subject: [Python-Dev] Attribute strings?
Message-ID: <001501c47d8a$72851660$6700a8c0@computer>

Has anyone suggested the following?

Create a new kind of string, _not_ a docstring, to hold annotations, etc.

"""@annotate whatever""" or even ""@" whatever"""

Pros: Open ended possibilities, might quiet dissent, could be placed before
function or inside it.

Biggest drawback: something that looks like a string will affect generated
code.

thinking-inside-the-string'ly yours,

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From guido at python.org  Sun Aug  8 23:04:11 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Aug  8 23:04:17 2004
Subject: [Python-Dev] Attribute strings?
In-Reply-To: Your message of "Sun, 08 Aug 2004 15:58:24 CDT."
	<001501c47d8a$72851660$6700a8c0@computer> 
References: <001501c47d8a$72851660$6700a8c0@computer> 
Message-ID: <200408082104.i78L4Bb21243@guido.python.org>

> Has anyone suggested the following?

Not AFAIK.

> Create a new kind of string, _not_ a docstring, to hold annotations, etc.
> 
> """@annotate whatever""" or even ""@" whatever"""

The first is a valid string literal now, and thus gets the same fate
as [decorators] -- lots of people get very uncomfortable at the
ambiguity of assigning a new meaning to something that was legal
syntax before (even if it was meaningless).

The second is not a valid string literal now -- doesn't that cause the
same problems for Leo as @decorators?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From bac at OCF.Berkeley.EDU  Sun Aug  8 23:31:14 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Aug  8 23:31:19 2004
Subject: [Python-Dev] Tuple/list assignment question
In-Reply-To: <000e01c47a2d$84f3be60$e841fea9@oemcomputer>
References: <000e01c47a2d$84f3be60$e841fea9@oemcomputer>
Message-ID: <41169BA2.9030108@ocf.berkeley.edu>

Raymond Hettinger wrote:
 >[Guido]
>>... I appreciate the idea (which BTW has been proposed before)
> 
> 
> IIRC, the conversation for the original proposal fizzled out without
> reaching a conclusion.  So, resuming the discussion was probably a good
> idea.  But now it has been officially zapped.
> 
>

Actually, it was shot down by Guido last time as well: 
http://www.python.org/dev/summary/2002-11-16_2002-11-30.html#half-baked-proposal-and-in-assignments

History has spoken!  The power of the python-dev Summary and Google's 
'site' functionality strikes again! =)

-Brett
From edreamleo at charter.net  Sun Aug  8 23:20:21 2004
From: edreamleo at charter.net (Edward K. Ream)
Date: Sun Aug  8 23:40:03 2004
Subject: [Python-Dev] re: Attribute Strings
Message-ID: <000701c47d8d$8391aec0$6700a8c0@computer>

> The second is not a valid string literal now -- 

That's why I suggested it.  ""@" would be a new token, so the statement
about '@' in the delims section would remain essentially unchanged.

> doesn't that cause the same problems for Leo as @decorators?

No.  At most leading whitespace may precede Leo's directives.

Edward
--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------



From bac at OCF.Berkeley.EDU  Sun Aug  8 23:46:37 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Aug  8 23:46:42 2004
Subject: [Python-Dev] Optimized string concatenation
In-Reply-To: <410FE549.1030407@iinet.net.au>
References: <003201c47904$5cd353c0$df30cb97@oemcomputer>	<200408031527.i73FRQb31100@guido.python.org>	<d7cee4a70408031046ebfbf2b@mail.gmail.com>
	<410FE549.1030407@iinet.net.au>
Message-ID: <41169F3D.7060508@ocf.berkeley.edu>

Nick Coghlan wrote:

> Adam Souzis wrote:
> 
>> As someone relatively new to python, it struck me as a language wart
>> that i had to learn the form '"".join() as the proper way to do string
>> concatenation. It violates the principles of OOWTI and its certainly
>> not the obvious way to do string concatenation. This patch does not
>> cover all the cases so we're still stuck with join(), but as long as
>> it is not a documentated "feature" it will harmlessly improve the
>> performance of countless lines of code where the coder is either
>> untrained or too lazy to use join().  If its documented it'd just
>> muddy the waters vis a vis join(), besides the issues with other
>> Python implementation mentioned here.
> 
> 
> If I understand correctly, you're suggesting that ''.join(strings) 
> continue to be the recommended, portable, non-quadratic method for 
> concatenating strings.
> 

I think the "portable" label is a little misleading.  It isn't like 
using string concatenation now is non-portable, it's just slow.

The other labels are correct, though.

-Brett
From nbastin at opnet.com  Sun Aug  8 23:54:38 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Sun Aug  8 23:54:44 2004
Subject: [Python-Dev] re: Attribute Strings
In-Reply-To: <000701c47d8d$8391aec0$6700a8c0@computer>
References: <000701c47d8d$8391aec0$6700a8c0@computer>
Message-ID: <8B2986D7-E985-11D8-A534-000D932927FE@opnet.com>


On Aug 8, 2004, at 5:20 PM, Edward K. Ream wrote:

>> The second is not a valid string literal now --
>
> That's why I suggested it.  ""@" would be a new token, so the statement
> about '@' in the delims section would remain essentially unchanged.

I don't know if anyone is actually seriously considering this, but I'm 
-1.  We're venturing into an area where the syntax makes people really 
not want to use the feature (and ""@" probably doesn't please the 
people who don't like the aesthetics of '@', so all we're doing is 
fixing the problem for IPython and a few other corner people).

My impression is that <something>-before-def is going to be The 
Syntax(tm), but Guido is content to let <something> be something other 
than '@'.  If that's the case, then I'm not sure what all the alternate 
placement syntaxes are being proposed for.  If that's not the case, 
then I'll start lobbying harder for <something>-after-def (although not 
""@")... ;-)

--
Nick

From guido at python.org  Sun Aug  8 23:59:54 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 00:00:00 2004
Subject: [Python-Dev] Re: Attribute Strings
In-Reply-To: Your message of "Sun, 08 Aug 2004 16:20:21 CDT."
	<000701c47d8d$8391aec0$6700a8c0@computer> 
References: <000701c47d8d$8391aec0$6700a8c0@computer> 
Message-ID: <200408082159.i78LxsO21394@guido.python.org>

> > The second is not a valid string literal now -- 
> 
> That's why I suggested it.  ""@" would be a new token, so the statement
> about '@' in the delims section would remain essentially unchanged.

Hm.  You might as well propose @#$% then. :-)

Anyway, the problem with anything inside "string literals" (real or
perceived) is that then the parser has to recursively parse the
expression in the string literal, because it has to generate code for
it.  I consider this too perverse to bother.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Mon Aug  9 01:09:41 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  9 01:09:44 2004
Subject: [Python-Dev] capturing RETURN_VALUE
In-Reply-To: <41165D23.4070202@stackless.com>
References: <41165D23.4070202@stackless.com>
Message-ID: <1f7befae040808160924e4d8a0@mail.gmail.com>

[Christian Tismer]
> just by chance, I discovered a probably very unknown
> feature of the block stack machinery of Python's frames:
> 
> --  It is possible to abandon a return statement.  --

[if the return tickles an enclosing finally block, and the finally block, e.g.,
 does a different return, or breaks out of a loop, or raises an exception]

> My question: Is this an artifact, or can I rely on this
> feature to persist for a while (while while >= 1 year) ?

Yes, it's reliable.  The Java spec is clearer about it, but Python
acts the same way.

If Python allowed continue statements in finally blocks, you'd also
find that a finally block could "abandon" a break statement (if the
break tickled an enclosing finally block in the same loop, and the
finally block did a continue).

> I have a code body which either computes and returns a value,
> or it raises an exception. Surrounding it, I have a while
> true loop, embracing a try..finally block. In case of an
> exception, it simply lets the exception pass.

It did in your specific example, but not necessarily.  In your
specific example, the finally block didn't execute "break" if an
exception was raised, it only did "break" if the finally block was
entered because of a return.  If it had done "break" in the exception
case too, it would have "abandoned" the exception too:

def return_capture(n, do_raise=False):
   _have_value = False
   retval = 666
   while True:
       try:
           if do_raise:
               raise ValueError, "was told to raise"
           _have_value = True
           retval = n*n
           return
       finally:
           break
   retval = ("here the captured return value", retval)
   return retval

>>> return_capture(12)
('here the captured return value', 144)
>>> return_capture(12, True)
('here the captured return value', 666)
>>>

So there's really no difference between how returns and exceptions work here.
From tismer at stackless.com  Mon Aug  9 01:18:23 2004
From: tismer at stackless.com (Christian Tismer)
Date: Mon Aug  9 01:15:51 2004
Subject: [Python-Dev] capturing RETURN_VALUE
In-Reply-To: <1f7befae040808160924e4d8a0@mail.gmail.com>
References: <41165D23.4070202@stackless.com>
	<1f7befae040808160924e4d8a0@mail.gmail.com>
Message-ID: <4116B4BF.5030409@stackless.com>

Tim Peters wrote:

...

> It did in your specific example, but not necessarily.  In your
> specific example, the finally block didn't execute "break" if an
> exception was raised, it only did "break" if the finally block was
> entered because of a return.  If it had done "break" in the exception
> case too, it would have "abandoned" the exception too:

Sure, that was by design.
I'm trying to inline functions into each other, with
smallest possible changes to the bytecodes of the
inlined functions. Exceptions shall pass through,
returns shall be captured and *not* cause the embracing
function to return, but to continue with the "returned"
value.

thanks for this answer -- chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tim.peters at gmail.com  Mon Aug  9 01:32:00 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  9 01:32:04 2004
Subject: [Python-Dev] capturing RETURN_VALUE
In-Reply-To: <4116B4BF.5030409@stackless.com>
References: <41165D23.4070202@stackless.com>
	<1f7befae040808160924e4d8a0@mail.gmail.com>
	<4116B4BF.5030409@stackless.com>
Message-ID: <1f7befae04080816325f3fa2f2@mail.gmail.com>

[Tim Peters]
>> It did in your specific example, but not necessarily.  In your
>> specific example, the finally block didn't execute "break" if an
>> exception was raised, it only did "break" if the finally block was
>> entered because of a return.  If it had done "break" in the exception
>> case too, it would have "abandoned" the exception too:

[Christian Tismer]
> Sure, that was by design.
> I'm trying to inline functions into each other, with
> smallest possible changes to the bytecodes of the
> inlined functions. Exceptions shall pass through,
> returns shall be captured and *not* cause the embracing
> function to return, but to continue with the "returned"
> value.

The point was that exceptions and returns act the same way: 
RETURN_VALUE isn't unique here, it's just another instance of leaving
a try-block's suite, the same in this respect as an exception or a
break statement.  That's why it's reliable over time:  it's a general
mechanism at work, not an arbitrary hack specific to RETURN_VALUE.
From greg at cosc.canterbury.ac.nz  Mon Aug  9 01:58:12 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  9 01:58:26 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <e443ad0e04080522023e4029e6@mail.gmail.com>
Message-ID: <200408082358.i78NwCPf005658@cosc353.cosc.canterbury.ac.nz>

> I realize that when Greg said ,"...see these arguments as being
> rational ones about usability and not irrational ones about
> aesthetics" he meant that arguments about aesthetics are usually
> subjective.

Partly that, but also that I don't think all of the arguments
about @ vs. [...] are purely subjective. It seems a fairly
objective fact to me that [...] carries more meaning in
terms of pre-existing connotations.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Mon Aug  9 02:01:47 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  9 02:01:52 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <f8ef9ae40408052303384b55c@mail.gmail.com>
Message-ID: <200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz>

IxokaI <ixokai@gmail.com>:

> '"define methy1 at staticmethod" is wrong, but "at
> staticmethod define methy2" is right.

Pardon? They *both* sound like utter nonsense to me.

That's the other main objection I have to '@'. I just
can't help pronouncing it in my head as "at".

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From kamikaze at kuoi.asui.uidaho.edu  Mon Aug  9 02:25:56 2004
From: kamikaze at kuoi.asui.uidaho.edu (Mark 'Kamikaze' Hughes)
Date: Mon Aug  9 02:26:01 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu>
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <20040809002556.GA75629@kuoi.asui.uidaho.edu>

Sat, Aug 07, 2004 at 12:35:17AM -0400 in <41145C05.1020501@gradient.cis.upenn.edu>,
Edward Loper <edloper@gradient.cis.upenn.edu> spake:
>The new "vertical bar" syntax was proposed recently, and has gotten
>almost unanimously positive feedback.  In particular, of 17 responses
>to the syntax, only one has been negative (see table, below).
>I'm including both of the following variants:
>    |classmethod
>    |accepts(int, int)
>    |returns(float)
>    def foo(arg1, arg2):
>        ...
>    def foo(arg1, arg2):
>        |classmethod
>        |accepts(int, int)
>        |returns(float)
>        ...
>(I prefer the second, but would be happy with either.)
>The one negative comment was that "|" can look similar to "I" or "l",
>depending on the font; but it's only an issue for some fonts, and it
>should be largely mitigated by syntax-highlighting.
>So I wanted to see if anyone has anything *negative* to say about this
>syntax (or one of its variants).  If possible, please respond by
>updating the wiki, rather than by email.  This syntax is listed under
>H (pre-def) and E2 (post-def).  You will need to sign in before you
>can edit the wiki.
>    Wiki: <http://www.python.org/cgi-bin/moinmoin/PythonDecorators>

  -1.  I think vbar-before is an extremely hard syntax to read, and
vbar-after is extremely misleading (but I don't like where docstrings
are, either).  Wiki updated.

-- 
 <a href="http://kuoi.asui.uidaho.edu/~kamikaze/"> Mark Hughes </a>
"Doing the impossible makes us mighty." -Captain Malcolm Reynolds, Firefly
From tim.peters at gmail.com  Mon Aug  9 02:42:57 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  9 02:42:59 2004
Subject: [Python-Dev] PyRun_ with file name
In-Reply-To: <4113EF69.2020102@v.loewis.de>
References: <4113EF69.2020102@v.loewis.de>
Message-ID: <1f7befae040808174233fb20db@mail.gmail.com>

[Martin v. L?wis]
> There is a constant issue of mixing CRTs on Windows. Patch 1003535
> proposes to work around them by adding yet another PyRun_ function.
> I'm in favour, except that:
> - it should be called PyRun_Filename
> - it should have the maximum current parameter set, i.e. including
>   CompilerFlags*

The patch appears to include a whole bunch of irrelevant changes too
(which I've noted on the tracker).

> I'm doubtful as to whether specifying an opening mode is necessary.

Or useful <0.5 wink>.

> What do you think?

If you understand the problem, and believe it will fix it, +1 (I don't
think I fully understand the problem, but it gets points from me for
being a small change).
From greg at cosc.canterbury.ac.nz  Mon Aug  9 03:05:48 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  9 03:05:55 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <200408061445.i76EjqT07026@guido.python.org>
Message-ID: <200408090105.i7915mfS005734@cosc353.cosc.canterbury.ac.nz>

Guido:

> Just look at it when either the list of decorators or the argument
> list doesn't fit on one line:

You don't *have* to format it quite that badly.

    def longMethodNameForEffect(longArgumentOne=None, 
                                longArgumentTwo=42) \
        [staticmethod,
         funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
                   status="experimental",
                   author="BDFL")]:

If you can arrange for the '[' to appear immediately before the first
decorator, I think it makes it a lot easier to recognise the construct
for what it is.

I also think that's a rather untypical decorator argument you've used
there, being full of spaces, brackets and other confounding
characters. That sort of thing is going to look confusing in any
context, whether function decorators are involved or not.

Perhaps someone can post some real-life use cases written with this
syntax, so we can see what it would *really* look like in typical use,
as opposed to made-up worst-case examples?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Mon Aug  9 03:07:55 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  9 03:08:05 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <e8bf7a5304080607524990942d@mail.gmail.com>
Message-ID: <200408090107.i7917tOl005740@cosc353.cosc.canterbury.ac.nz>

Jeremy Hylton <jhylton@gmail.com>:

> I thought it was obvious that this option was not acceptable.

Trouble is, it's equally "obvious" to the opponents of "@"
that *it's* not acceptable...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From kamikaze at kuoi.asui.uidaho.edu  Mon Aug  9 03:11:00 2004
From: kamikaze at kuoi.asui.uidaho.edu (Mark 'Kamikaze' Hughes)
Date: Mon Aug  9 03:11:03 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz>
References: <f8ef9ae40408052303384b55c@mail.gmail.com>
	<200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040809011100.GB75629@kuoi.asui.uidaho.edu>

Mon, Aug 09, 2004 at 12:01:47PM +1200 in <200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz>,
Greg Ewing <greg@cosc.canterbury.ac.nz> spake:
>IxokaI <ixokai@gmail.com>:
>> '"define methy1 at staticmethod" is wrong, but "at
>> staticmethod define methy2" is right.
>Pardon? They *both* sound like utter nonsense to me.
>That's the other main objection I have to '@'. I just
>can't help pronouncing it in my head as "at".

  Try pronouncing it as "attribute" (n.) or "attribute" (v.).  Either
one makes sense in context.

-- 
 <a href="http://kuoi.asui.uidaho.edu/~kamikaze/"> Mark Hughes </a>
"Doing the impossible makes us mighty." -Captain Malcolm Reynolds, Firefly
From bob at redivi.com  Mon Aug  9 03:25:20 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug  9 03:25:30 2004
Subject: [Python-Dev] @decorators, the PEP and the "options" out there?
In-Reply-To: <20040809011100.GB75629@kuoi.asui.uidaho.edu>
References: <f8ef9ae40408052303384b55c@mail.gmail.com>
	<200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz>
	<20040809011100.GB75629@kuoi.asui.uidaho.edu>
Message-ID: <FA9413C2-E9A2-11D8-A616-000A95686CD8@redivi.com>


On Aug 8, 2004, at 9:11 PM, Mark 'Kamikaze' Hughes wrote:

> Mon, Aug 09, 2004 at 12:01:47PM +1200 in 
> <200408090001.i7901lsx005666@cosc353.cosc.canterbury.ac.nz>,
> Greg Ewing <greg@cosc.canterbury.ac.nz> spake:
>> IxokaI <ixokai@gmail.com>:
>>> '"define methy1 at staticmethod" is wrong, but "at
>>> staticmethod define methy2" is right.
>> Pardon? They *both* sound like utter nonsense to me.
>> That's the other main objection I have to '@'. I just
>> can't help pronouncing it in my head as "at".
>
>   Try pronouncing it as "attribute" (n.) or "attribute" (v.).  Either
> one makes sense in context.

Not really.  It's actually a transformation, not an attribute.  A more 
appropriate verb would be "make" or "is", I guess.

@staticmethod
def foo(...):
	pass

-bob

From janssen at parc.com  Mon Aug  9 03:31:49 2004
From: janssen at parc.com (Bill Janssen)
Date: Mon Aug  9 03:32:24 2004
Subject: [Python-Dev] Decorators: vertical bar syntax 
In-Reply-To: Your message of "Sat, 07 Aug 2004 09:17:42 PDT."
	<200408071617.i77GHgs10281@guido.python.org> 
Message-ID: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>

> In the discussion on decorators months ago, solutions involving
> special syntax inside the block were ruled out early on.  Basically,
> you shouldn't have to peek inside the block to find out important
> external properties of the function.

Guido, could you expand on this principle a bit?  Just stated as it
is, it sounds as pointless (to my untutored mind, at least :-), as the
arguments about "@ is ugly, so it shouldn't be used".  After all,
isn't "what the function does" an important external property of the
function, and don't you have to peek inside the block to determine
that?  For instance, how do you know whether a function is a generator
unless you look for yield statements inside it?

Bill
From greg at cosc.canterbury.ac.nz  Mon Aug  9 04:02:39 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  9 04:02:51 2004
Subject: [SPAM: 3.611] [Python-Dev] IPython, @, and option E from the wiki
In-Reply-To: <cf17ic$83o$1@sea.gmane.org>
Message-ID: <200408090202.i7922diB005813@cosc353.cosc.canterbury.ac.nz>

3. In this kind of setup, using | instead of @ would be ok as well, I guess:
Fernando Perez <fperez528@yahoo.com>:

> def bar(low,high):
>    |accepts(int,int)
>    |returns(float)

How about

  def bar(low,high):
    ...accepts(int,int)
    ...returns(float)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Mon Aug  9 04:12:49 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  9 04:13:00 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <cf1nvj$uk0$1@sea.gmane.org>
Message-ID: <200408090212.i792CnP9005824@cosc353.cosc.canterbury.ac.nz>

Terry Reedy <tjreedy@udel.edu>:

> Is a space allowed after the prefic character, whatever it might be?  With
> 
> def foo():
>    | classmethod
>    | accepts(int,int)
>    | returns(float)

It certainly looks much better with the space. But I'm
not sure if it should be required -- it would be a strange
kind of rule, inconsistent with the lexical style of the
rest of the language.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Mon Aug  9 04:16:45 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  9 04:16:50 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <041501c47c45$54e3f630$e615a88d@neil>
Message-ID: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz>

> For example, if there is later a need or desire to annotate function
> arguments individually as well as functions, or individual
> statements or expressions, then @ is a better choice. For example
> 
> @RemoteCall
> def Ignite(engine @Remote, environment @Copy, thrust @Float):

Nooooo.....

If @decorator stays in, I fervently hope it won't be taken as a
precedent for using @ for other things all over the place.  Otherwise
we really *will* end up with a language that resembles Perl.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Mon Aug  9 04:29:07 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  9 04:29:20 2004
Subject: [Python-Dev] Suggesting '.' decorators (PEP318)
In-Reply-To: <200408071128.56011.heikowu@ceosg.de>
Message-ID: <200408090229.i792T7QI005872@cosc353.cosc.canterbury.ac.nz>

Heiko Wundram <heikowu@ceosg.de>:

> Same objection from me as to the proposal of decorating with | in
> the function: it simply takes away the actual meaning from the
> decorator (which actually is to make it very clear to see that the
> function is special in some way).

I think a problem we have here is that how important it is to know
that a function is being decorated depends on the nature of the
decorator. For example, it can be important for the user of a function
to know that it's a classmethod or staticmethod. On the other hand,
the Objective-C signature of a PyObjC method is something that's only
of interest to the implementor of the method.

So there isn't going to be a one-size-fits-all approach that will give
all decorators exactly the right amount of prominence.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From jepler at unpythonic.net  Mon Aug  9 04:29:34 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Mon Aug  9 04:30:07 2004
Subject: [Python-Dev] The date in the generated documentation
Message-ID: <20040809022934.GB6582@unpythonic.net>

This is probably the strangest question I've ever asked python-dev.

For the Python bug day I updated my CVS, and later on typed "make" in
the Doc directory.

I'm now viewing the generated html/lib/lib.html file, and it's displayed
in that weird font that Japanese web pages always use, and the top of
the document says
    Release 2.4a2
    ??16?8?7?
which is, as nearly as I can tell, a (correct?) Japanese "Emperor Date".
http://www.i18nguy.com/l10n/emperor-date.html gives a little explanation
for us westerners, the first two characters match the Kanji he gives
for the current era (beginning in 1989), and the Western calendar year
is now 1989 + 16 - 1.

Also, the strings "Context", "Index", and "About this document" are
written in Japanese characters.

This appears to affect the other documents (ref/ref.html, etc) generated
as well.

Nothing on my system should be particularly Japanese (my locale is
en_US.UTF-8, for example).  While I installed a lot of fonts with this
system, I haven't done anything else that would lead me to expect this.

The OS is Fedora Core 2 upgraded from FC1 if that helps ring anybody's
bell.

The paper-letter/*.pdf files that "make pdf" generates have the
proper "August 8, 2004" date in them.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040808/c5869781/attachment.pgp
From greg at cosc.canterbury.ac.nz  Mon Aug  9 04:31:00 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug  9 04:31:09 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <59e9fd3a04080705067633a790@mail.gmail.com>
Message-ID: <200408090231.i792V0Y0005878@cosc353.cosc.canterbury.ac.nz>

> Two, and more importantly: the pipe is in the shifted position of the
> backslash key, which on different keyboards is in different locations,
> many of them awkward to reach while typing at speed.

But realistically, how many function decorators do you
expect to be typing per second? :-)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From nhodgson at bigpond.net.au  Mon Aug  9 04:37:05 2004
From: nhodgson at bigpond.net.au (Neil Hodgson)
Date: Mon Aug  9 04:37:10 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz>
Message-ID: <019301c47db9$c269ec80$a9498890@neil>

Greg Ewing:
> > @RemoteCall
> > def Ignite(engine @Remote, environment @Copy, thrust @Float):
>
> Nooooo.....
>
> If @decorator stays in, I fervently hope it won't be taken as a
> precedent for using @ for other things all over the place.  Otherwise
> we really *will* end up with a language that resembles Perl.

   A motivation for this was that one of the examples

@accepts(int,int)
@returns(float)
def bar(low,high):

   which separates related information too much for me. If decorators need
to decorate parameters, they are more likely to be understood correctly by a
reader if the decorations are next to the parameters.

   Neil

From jepler at unpythonic.net  Mon Aug  9 04:43:36 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Mon Aug  9 04:44:05 2004
Subject: [Python-Dev] The date in the generated documentation
In-Reply-To: <20040809022934.GB6582@unpythonic.net>
References: <20040809022934.GB6582@unpythonic.net>
Message-ID: <20040809024336.GC6582@unpythonic.net>

I had to edit the lines in the file /usr/share/latex2html/l2hconf.pm
which mentioned japanese (by default!  I'm certain I never changed them)
and change them back to english.

The generated html documentation is now much improved.

Sorry to bother everyone for what turned out to be a fedora bug /
configuration problem.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040808/3a7fb980/attachment.pgp
From aahz at pythoncraft.com  Mon Aug  9 05:35:54 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Aug  9 05:35:56 2004
Subject: [Python-Dev] pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
In-Reply-To: <lkdwu0bcz7r.fsf@panix2.panix.com>
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<lkdwu0bcz7r.fsf@panix2.panix.com>
Message-ID: <20040809033554.GA21340@panix.com>

On Sat, Aug 07, 2004, Mike Coleman wrote:
>
> Motivation
> ==========

This section needs fleshing out to explain why re.structmatch works
better for some purposes than forcing people to switch to a full-blown
parser.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From aahz at pythoncraft.com  Mon Aug  9 05:40:36 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Aug  9 05:40:38 2004
Subject: [Python-Dev] re.split on empty patterns
In-Reply-To: <20040807145142.GB2529@rogue.amk.ca>
References: <20040807145142.GB2529@rogue.amk.ca>
Message-ID: <20040809034036.GB21340@panix.com>

On Sat, Aug 07, 2004, A.M. Kuchling wrote:
>
> Question: do we want to make this option the new default?  

Agreed with Tim: -1 until Python 3
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From aahz at pythoncraft.com  Mon Aug  9 05:51:38 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Aug  9 05:51:46 2004
Subject: [Python-Dev] Decimal type question [Prothon]
In-Reply-To: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
Message-ID: <20040809035138.GC21340@panix.com>

On Sat, Aug 07, 2004, Mark Hahn wrote:
>
> Can anyone see any problem with this scheme?

Yes: it doesn't follow the decimal floating point standard.  Read PEP327
for an exhaustive discussion of the subject WRT Python (which is nowhere
near as exhausting as reading the discussion in the python-dev archives).


Side note: I haven't bothered checking recently, but since you were
complaining about the quality of technical responses on c.l.py, I've
been mostly ignoring you because of your habit of top-posting.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From tim.peters at gmail.com  Mon Aug  9 06:01:17 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  9 06:01:21 2004
Subject: [Python-Dev] Re: MSI Installer issues
In-Reply-To: <4114A02F.7060906@v.loewis.de>
References: <411231C8.3020308@interlink.com.au>
	<411241F5.3080602@v.loewis.de>	<1f7befae040805103911610582@mail.gmail.com>	<411274C0.4070300@v.loewis.de>
	<1f7befae04080623276089908e@mail.gmail.com>
	<4114A02F.7060906@v.loewis.de>
Message-ID: <1f7befae040808210137f08a9e@mail.gmail.com>

[Martin v. L?wis]
>  ...
> While I have your attention: you have commented that the installer logo
> needs attribution, during the installation process. Is this still the
> case, and what is the text I should use? Any other attribution that
> needs to be made?

Its creator (Erik van Blokland) asked for attribution, in return for
doing the work.  All I know is here:

     http://mail.python.org/pipermail/python-dev/2004-July/046317.html

He seemed happy with the attribution quoted there.

> A colleague has commented that logo is too low-res, is ugly, and does
> not feature reptiles. Should we look for an update to the logo?

I'm surprised this didn't trigger a flood of responses dwarfing the
decorators thread!

I'm out of any logo debate.  "low-res" may be a feature, as some
people have very low screen resolution settings and very large fonts. 
Etc, etc.  If you want the quest to end, don't ask Guido to approve a
new logo <0.9 wink>.
From dima at trit.org  Mon Aug  9 06:31:05 2004
From: dima at trit.org (Dima Dorfman)
Date: Mon Aug  9 06:33:28 2004
Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2,
	and @decorators)
In-Reply-To: <410EF963.40704@interlink.com.au>
References: <410DDFE3.7010006@interlink.com.au>
	<200408021850.i72IoCg28650@guido.python.org>
	<410EF963.40704@interlink.com.au>
Message-ID: <20040809043104.GA82909@trit.org>

Anthony Baxter <anthonyatinterlink.com.au> wrote:
> Guido van Rossum wrote:
> > Did the relative import syntax that was approved ever get checked in?
> > I thought it was on the list of things to land in 2.4?
> 
> Nope. The 2.4 release schedule PEP now lists this as lacking someone
> to drive it. I won't/can't spend time on it. So, if one of the people
> who wants it could step forward, that'd be excellent. Even if it's
> just to do the multi-line import syntax of
> 
>   from foo import ( bar, baz, bar2,
>                     baz2, bar3, baz3 )

I took a stab at implementing this (the easy part of PEP 328--parens
around import lists). Are the exact semantics of what's allowed
documented somewhere? PEP 328 mostly talks about relative and absolute
imports, and it doesn't specify the exact semantics of where
parentheses should be allowed. My patch (attached) accepts

  import (os, sys)
  from sys import (stdin, stdout, stderr)
  import (os)
  from sys import (*)

but rejects

  from (sys) import stdin
  import (os), (sys)
  import (os,)

Should any of those be allowed? Anything that I missed?

The patch is incomplete in other ways; the docs haven't been updated
and neither have the parser module and compile package. If it's
decided that it would be okay to include this separately from
relative/absolute import support (they're different features, really),
I'll complete the patch.

Dima.
-------------- next part --------------
Index: Python/compile.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/compile.c,v
retrieving revision 2.314
diff -u -c -r2.314 compile.c
*** Python/compile.c	6 Aug 2004 19:46:34 -0000	2.314
--- Python/compile.c	9 Aug 2004 04:01:38 -0000
***************
*** 3333,3374 ****
  static void
  com_import_stmt(struct compiling *c, node *n)
  {
  	int i;
  	REQ(n, import_stmt);
! 	/* 'import' dotted_name (',' dotted_name)* |
! 	   'from' dotted_name 'import' ('*' | NAME (',' NAME)*) */
! 	if (STR(CHILD(n, 0))[0] == 'f') {
  		PyObject *tup;
- 		/* 'from' dotted_name 'import' ... */
  		REQ(CHILD(n, 1), dotted_name);
! 		
! 		if (TYPE(CHILD(n, 3)) == STAR) {
  			tup = Py_BuildValue("(s)", "*");
! 		} else {
! 			tup = PyTuple_New((NCH(n) - 2)/2);
! 			for (i = 3; i < NCH(n); i += 2) {
! 				PyTuple_SET_ITEM(tup, (i-3)/2, 
  					PyString_FromString(STR(
! 						CHILD(CHILD(n, i), 0))));
  			}
  		}
  		com_addoparg(c, LOAD_CONST, com_addconst(c, tup));
  		Py_DECREF(tup);
  		com_push(c, 1);
  		com_addopname(c, IMPORT_NAME, CHILD(n, 1));
! 		if (TYPE(CHILD(n, 3)) == STAR) 
  			com_addbyte(c, IMPORT_STAR);
  		else {
! 			for (i = 3; i < NCH(n); i += 2) 
! 				com_from_import(c, CHILD(n, i));
  			com_addbyte(c, POP_TOP);
  		}
  		com_pop(c, 1);
  	}
  	else {
! 		/* 'import' ... */
! 		for (i = 1; i < NCH(n); i += 2) {
! 			node *subn = CHILD(n, i);
  			REQ(subn, dotted_as_name);
  			com_addoparg(c, LOAD_CONST, com_addconst(c, Py_None));
  			com_push(c, 1);
--- 3333,3379 ----
  static void
  com_import_stmt(struct compiling *c, node *n)
  {
+ 	node *nn;
  	int i;
  	REQ(n, import_stmt);
! 	n = CHILD(n, 0);
! 	/* import_stmt: just_import | from_import */
! 	if (TYPE(n) == from_import) {
! 		/* from_import: 'from' dotted_name 'import' ('*' | '(' '*' ')'
! 		     | '(' import_as_names ')' | import_as_names) */
  		PyObject *tup;
  		REQ(CHILD(n, 1), dotted_name);
! 		nn = CHILD(n, 3 + (TYPE(CHILD(n, 3)) == LPAR));
! 		if (TYPE(nn) == STAR)
  			tup = Py_BuildValue("(s)", "*");
! 		else {
! 			REQ(nn, import_as_names);
! 			tup = PyTuple_New((NCH(nn) + 1) / 2);
! 			for (i = 0; i < NCH(nn); i += 2) {
! 				PyTuple_SET_ITEM(tup, i / 2,
  					PyString_FromString(STR(
! 						CHILD(CHILD(nn, i), 0))));
  			}
  		}
  		com_addoparg(c, LOAD_CONST, com_addconst(c, tup));
  		Py_DECREF(tup);
  		com_push(c, 1);
  		com_addopname(c, IMPORT_NAME, CHILD(n, 1));
! 		if (TYPE(nn) == STAR)
  			com_addbyte(c, IMPORT_STAR);
  		else {
! 			for (i = 0; i < NCH(nn); i += 2)
! 				com_from_import(c, CHILD(nn, i));
  			com_addbyte(c, POP_TOP);
  		}
  		com_pop(c, 1);
  	}
  	else {
! 		/* 'import' ('(' dotted_as_names ')' | dotted_as_names)*/
! 		nn = CHILD(n, 1 + (TYPE(CHILD(n, 1)) == LPAR));
! 		REQ(nn, dotted_as_names);
! 		for (i = 0; i < NCH(nn); i += 2) {
! 			node *subn = CHILD(nn, i);
  			REQ(subn, dotted_as_name);
  			com_addoparg(c, LOAD_CONST, com_addconst(c, Py_None));
  			com_push(c, 1);
***************
*** 6131,6144 ****
  static void
  symtable_import(struct symtable *st, node *n)
  {
  	int i;
! 	/* import_stmt: 'import' dotted_as_name (',' dotted_as_name)* 
!               | 'from' dotted_name 'import' 
!                                 ('*' | import_as_name (',' import_as_name)*)
! 	   import_as_name: NAME [NAME NAME]
! 	*/
! 	if (STR(CHILD(n, 0))[0] == 'f') {  /* from */
  		node *dotname = CHILD(n, 1);
  		if (strcmp(STR(CHILD(dotname, 0)), "__future__") == 0) {
  			/* check for bogus imports */
  			if (n->n_lineno >= st->st_future->ff_last_lineno) {
--- 6136,6150 ----
  static void
  symtable_import(struct symtable *st, node *n)
  {
+ 	node *nn;
  	int i;
! 	/* import_stmt: just_import | from_import */
! 	n = CHILD(n, 0);
! 	if (TYPE(n) == from_import) {
! 		/* from_import: 'from' dotted_name 'import' ('*' | '(' '*' ')'
! 		     | '(' import_as_names ')' | import_as_names) */
  		node *dotname = CHILD(n, 1);
+ 		REQ(dotname, dotted_name);
  		if (strcmp(STR(CHILD(dotname, 0)), "__future__") == 0) {
  			/* check for bogus imports */
  			if (n->n_lineno >= st->st_future->ff_last_lineno) {
***************
*** 6148,6154 ****
  				return;
  			}
  		}
! 		if (TYPE(CHILD(n, 3)) == STAR) {
  			if (st->st_cur->ste_type != TYPE_MODULE) {
  				if (symtable_warn(st,
  				  "import * only allowed at module level") < 0)
--- 6154,6161 ----
  				return;
  			}
  		}
! 		nn = CHILD(n, 3 + (TYPE(CHILD(n, 3)) == LPAR));
! 		if (TYPE(nn) == STAR) {
  			if (st->st_cur->ste_type != TYPE_MODULE) {
  				if (symtable_warn(st,
  				  "import * only allowed at module level") < 0)
***************
*** 6157,6164 ****
  			st->st_cur->ste_optimized |= OPT_IMPORT_STAR;
  			st->st_cur->ste_opt_lineno = n->n_lineno;
  		} else {
! 			for (i = 3; i < NCH(n); i += 2) {
! 				node *c = CHILD(n, i);
  				if (NCH(c) > 1) /* import as */
  					symtable_assign(st, CHILD(c, 2),
  							DEF_IMPORT);
--- 6164,6172 ----
  			st->st_cur->ste_optimized |= OPT_IMPORT_STAR;
  			st->st_cur->ste_opt_lineno = n->n_lineno;
  		} else {
! 			REQ(nn, import_as_names);
! 			for (i = 0; i < NCH(nn); i += 2) {
! 				node *c = CHILD(nn, i);
  				if (NCH(c) > 1) /* import as */
  					symtable_assign(st, CHILD(c, 2),
  							DEF_IMPORT);
***************
*** 6167,6176 ****
  							DEF_IMPORT);
  			}
  		}
! 	} else { 
! 		for (i = 1; i < NCH(n); i += 2) {
! 			symtable_assign(st, CHILD(n, i), DEF_IMPORT);
! 		}
  	}
  }
  
--- 6175,6186 ----
  							DEF_IMPORT);
  			}
  		}
! 	} else {
! 		/* 'import' ('(' dotted_as_names ')' | dotted_as_names)*/
! 		nn = CHILD(n, 1 + (TYPE(CHILD(n, 1)) == LPAR));
! 		REQ(nn, dotted_as_names);
! 		for (i = 0; i < NCH(nn); i += 2)
! 			symtable_assign(st, CHILD(nn, i), DEF_IMPORT);
  	}
  }
  
Index: Python/future.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/future.c,v
retrieving revision 2.13
diff -u -c -r2.13 future.c
*** Python/future.c	11 Dec 2002 14:04:59 -0000	2.13
--- Python/future.c	9 Aug 2004 04:01:38 -0000
***************
*** 18,35 ****
  {
  	int i;
  	char *feature;
! 	node *ch;
  
! 	REQ(n, import_stmt); /* must by from __future__ import ... */
! 
! 	for (i = 3; i < NCH(n); i += 2) {
! 		ch = CHILD(n, i);
! 		if (TYPE(ch) == STAR) {
! 			PyErr_SetString(PyExc_SyntaxError,
! 					FUTURE_IMPORT_STAR);
! 			PyErr_SyntaxLocation(filename, ch->n_lineno);
! 			return -1;
! 		}
  		REQ(ch, import_as_name);
  		feature = STR(CHILD(ch, 0));
  		if (strcmp(feature, FUTURE_NESTED_SCOPES) == 0) {
--- 18,35 ----
  {
  	int i;
  	char *feature;
! 	node *ch, *nn;
  
! 	REQ(n, from_import);
! 	nn = CHILD(n, 3 + (TYPE(CHILD(n, 3)) == LPAR));
! 	if (TYPE(nn) == STAR) {
! 		PyErr_SetString(PyExc_SyntaxError, FUTURE_IMPORT_STAR);
! 		PyErr_SyntaxLocation(filename, nn->n_lineno);
! 		return -1;
! 	}
! 	REQ(nn, import_as_names);
! 	for (i = 0; i < NCH(nn); i += 2) {
! 		ch = CHILD(nn, i);
  		REQ(ch, import_as_name);
  		feature = STR(CHILD(ch, 0));
  		if (strcmp(feature, FUTURE_NESTED_SCOPES) == 0) {
***************
*** 188,194 ****
  	case import_stmt: {
  		node *name;
  
! 		if (STR(CHILD(n, 0))[0] != 'f') { /* from */
  			ff->ff_last_lineno = n->n_lineno;
  			return 0;
  		}
--- 188,195 ----
  	case import_stmt: {
  		node *name;
  
! 		n = CHILD(n, 0);
! 		if (TYPE(n) != from_import) {
  			ff->ff_last_lineno = n->n_lineno;
  			return 0;
  		}
Index: Grammar/Grammar
===================================================================
RCS file: /cvsroot/python/python/dist/src/Grammar/Grammar,v
retrieving revision 1.50
diff -u -c -r1.50 Grammar
*** Grammar/Grammar	2 Aug 2004 06:09:53 -0000	1.50
--- Grammar/Grammar	9 Aug 2004 04:01:38 -0000
***************
*** 51,59 ****
  return_stmt: 'return' [testlist]
  yield_stmt: 'yield' testlist
  raise_stmt: 'raise' [test [',' test [',' test]]]
! import_stmt: 'import' dotted_as_name (',' dotted_as_name)* | 'from' dotted_name 'import' ('*' | import_as_name (',' import_as_name)*)
  import_as_name: NAME [NAME NAME]
  dotted_as_name: dotted_name [NAME NAME]
  dotted_name: NAME ('.' NAME)*
  global_stmt: 'global' NAME (',' NAME)*
  exec_stmt: 'exec' expr ['in' test [',' test]]
--- 51,63 ----
  return_stmt: 'return' [testlist]
  yield_stmt: 'yield' testlist
  raise_stmt: 'raise' [test [',' test [',' test]]]
! import_stmt: just_import | from_import
! just_import: 'import' ('(' dotted_as_names ')' | dotted_as_names)
! from_import: 'from' dotted_name 'import' ('*' | '(' '*' ')' | '(' import_as_names ')' | import_as_names)
  import_as_name: NAME [NAME NAME]
  dotted_as_name: dotted_name [NAME NAME]
+ import_as_names: import_as_name (',' import_as_name)*
+ dotted_as_names: dotted_as_name (',' dotted_as_name)*
  dotted_name: NAME ('.' NAME)*
  global_stmt: 'global' NAME (',' NAME)*
  exec_stmt: 'exec' expr ['in' test [',' test]]
Index: Lib/test/test_compile.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_compile.py,v
retrieving revision 1.22
diff -u -c -r1.22 test_compile.py
*** Lib/test/test_compile.py	2 Aug 2004 08:30:07 -0000	1.22
--- Lib/test/test_compile.py	9 Aug 2004 04:01:39 -0000
***************
*** 211,216 ****
--- 211,263 ----
              self.assertRaises(SyntaxError, compile, stmt, 'tmp', 'single')
              self.assertRaises(SyntaxError, compile, stmt, 'tmp', 'exec')
  
+     def test_import(self):
+         # These should pass
+         exec '''
+ from __future__ import (nested_scopes, generators)
+ import sys
+ import (os,
+   sys)
+ from sys import (stdin, stderr,
+   stdout)
+ from keyword import (*)'''
+         # These should fail
+         try:
+             exec 'from (sys) import stdin'
+             self.fail("parens not allowed around single dotted_name")
+         except SyntaxError:
+             pass
+         try:
+             exec 'import (os), (sys)'
+             self.fail("entire name list must be enclosed in parens")
+         except SyntaxError:
+             pass
+         try:
+             exec 'import ((os), (sys))'
+             self.fail("subparens not allowed")
+         except SyntaxError:
+             pass
+         try:
+             exec 'import sys)'
+             self.fail("mismatched parens")
+         except SyntaxError:
+             pass
+         try:
+             exec 'from sys import stdin)'
+             self.fail("mismatched parens")
+         except SyntaxError:
+             pass
+         try:
+             exec 'import (os,)'
+             self.fail("not a tuple")
+         except SyntaxError:
+             pass
+         try:
+             exec 'from __future__ import nested_scopes)'
+             self.fail("mismatched parens in __future__");
+         except SyntaxError:
+             pass
+ 
  def test_main():
      test_support.run_unittest(TestSpecifics)
  
Index: Lib/test/test_grammar.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_grammar.py,v
retrieving revision 1.50
diff -u -c -r1.50 test_grammar.py
*** Lib/test/test_grammar.py	19 May 2004 08:20:09 -0000	1.50
--- Lib/test/test_grammar.py	9 Aug 2004 04:01:42 -0000
***************
*** 417,428 ****
  try: raise KeyboardInterrupt
  except KeyboardInterrupt: pass
  
! print 'import_stmt' # 'import' NAME (',' NAME)* | 'from' NAME 'import' ('*' | NAME (',' NAME)*)
  import sys
  import time, sys
  from time import time
  from sys import *
  from sys import path, argv
  
  print 'global_stmt' # 'global' NAME (',' NAME)*
  def f():
--- 417,434 ----
  try: raise KeyboardInterrupt
  except KeyboardInterrupt: pass
  
! print 'just_import' # 'import' ('(' dotted_as_names ')' | dotted_as_names)
  import sys
+ import (sys)
  import time, sys
+ import (time, sys)
+ print 'from_import' # 'from' dotted_name 'import' ('*' | '(' '*' ')' | '(' import_as_names ')' | import_as_names)
  from time import time
+ from time import (time)
  from sys import *
+ from sys import (*)
  from sys import path, argv
+ from sys import (path, argv)
  
  print 'global_stmt' # 'global' NAME (',' NAME)*
  def f():
Index: Lib/test/output/test_grammar
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/output/test_grammar,v
retrieving revision 1.20
diff -u -c -r1.20 test_grammar
*** Lib/test/output/test_grammar	21 May 2003 17:34:49 -0000	1.20
--- Lib/test/output/test_grammar	9 Aug 2004 04:01:42 -0000
***************
*** 35,41 ****
  testing continue and break in try/except in loop
  return_stmt
  raise_stmt
! import_stmt
  global_stmt
  exec_stmt
  assert_stmt
--- 35,42 ----
  testing continue and break in try/except in loop
  return_stmt
  raise_stmt
! just_import
! from_import
  global_stmt
  exec_stmt
  assert_stmt
From stephen at xemacs.org  Mon Aug  9 06:58:12 2004
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon Aug  9 06:58:19 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <59e9fd3a04080705067633a790@mail.gmail.com> (Andrew Durdin's
	message of "Sat, 7 Aug 2004 22:06:36 +1000")
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<opsccs4qc55c87rg@news.gmane.org> <4114A171.3050809@v.loewis.de>
	<59e9fd3a04080705067633a790@mail.gmail.com>
Message-ID: <87d620lxgb.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Andrew" == Andrew Durdin <adurdin@gmail.com> writes:

    Andrew> @ at least is consistently on the 2 key (except, of
    Andrew> course, for all those European keyboards. I guess their
    Andrew> differences are enough to knock down any argument about
    Andrew> typing convenience :-)

The other exception is Japanese keyboards (unshifted to the immediate
right of "P").


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.
From mark at prothon.org  Mon Aug  9 07:13:20 2004
From: mark at prothon.org (Mark Hahn)
Date: Mon Aug  9 07:13:39 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<20040809035138.GC21340@panix.com>
Message-ID: <1qygyzxyh5qnx.1r2aymzvv6xlw$.dlg@40tude.net>

On Sun, 8 Aug 2004 23:51:38 -0400, Aahz wrote:

> On Sat, Aug 07, 2004, Mark Hahn wrote:
> >
> > Can anyone see any problem with this scheme?
> 
> Yes: it doesn't follow the decimal floating point standard.
> Read PEP327 for an exhaustive discussion of the subject WRT 
> Python (which is nowhere near as exhausting as reading the 
> discussion in the python-dev archives).

I don't care too much about it being a standard but I do care about it
sucking.  Tim Peters filled me in on what's wrong with the MS type.

I am more interested now in hearing about people's feelings on the idea of
having the Decimal type "in-between" Int and Float.  Assume a decent
Decimal implementation is used.
 
> Side note: I haven't bothered checking recently, but since
> you were complaining about the quality of technical responses 
> on c.l.py, I've been mostly ignoring you because of your 
> habit of top-posting.

FWIW, I learned how to post in the c.l.p style back in early April which
which was four months ago.  I'm sorry about my earlier postings.  It was
hard to overcome an emailing style I had used in the business world for 25
years.

From adurdin at gmail.com  Mon Aug  9 07:28:52 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Mon Aug  9 07:28:58 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <200408081525.i78FPJm20862@guido.python.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de>
	<41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
	<4115FF35.4030608@v.loewis.de>
	<Pine.LNX.4.58.0408081431050.6199@rnd.onego.ru>
	<411610FB.9040201@v.loewis.de>
	<Pine.LNX.4.58.0408081703430.7067@rnd.onego.ru> 
	<59e9fd3a04080807177dd446af@mail.gmail.com>
	<200408081525.i78FPJm20862@guido.python.org>
Message-ID: <59e9fd3a04080822284be45e94@mail.gmail.com>

On Sun, 08 Aug 2004 08:25:19 -0700, Guido van Rossum <guido@python.org> wrote:
> 
> This an example of bad formatting.  Tests often do that because their
> purpose is to test odd corners of functionality, not serve as sample
> code.  Given that in typical example code there will always be a blank
> line separating the decorator from anything previous (and separating
> the body from what follows!) the connection between the decorator and
> the def is pretty clear.

In such a case, it would be clear that the author of the code
considered the decorator and the def to be connected as regards to
their function within the program; that the decorator can
fundamentally alter the meaning of the def statement is not at all
clear -- and it ought to be! By contrast, those proposed syntaxes
which have the decorators in the def statement (somewhere between the
def and the colon) do not suffer this opacity.

 > Also note that arguments based on partial knowledge of Python can be
> used as an argument against pretty much anything (just tweak the
> assumptions a bit), and hence have little value.

In the general case, this is probably true; but this argument of mine
is valid for any level of Python user familiar with versions up to
2.3, who has *only* not seen of or heard of the new decorator syntax.
This is because the @decorator syntax has a hidden side-effect, when
compared with normal Python statements. It is not at all consistent
with *any* other valid Python syntax that I know.

Consistency may not always be desirable; but I would have thought it
generally agreed that introducing a severe inconsistency for the sake
of only one feature is not a good design choice. If you believe that
introducing 'new-style' statements/expressions that implicitly affect
what follows them is a good idea *in general*, and is well-suited to
many features other than decorator expressions, then by all means,
introduce it first for decorators, and subsequently for these other
features--but at least let us know *why* these 'new-style'
statements/expressions are a good idea in general (not just for
decorators). Because frankly, and with respect, I haven't seen any
compelling arguments that introducing new statements/expressions with
implicit, non-obvious effects is a good thing for Python. To me, such
a thing reminds me of the horrible implicitly-set $_ and $? (and other
such creatures) in Perl.

Relevant quotes from the Zen of Python (not for argument but contemplation :-)
"Explicit is better than implicit."
"Special cases aren't special enough to break the rules."
"Although practicality beats purity."
"Now is better than never.
Although never is often better than *right* now."

I know you've said you hate the list-after-def syntax. I still hope
that you can change you mind there, or at least reduce "hate" to
"objective reasons for dislike" :-)   The latter half of my previous
post was an attempt at this by addressing your stated reasons for
disliking it.

Andrew
From martin at v.loewis.de  Mon Aug  9 08:14:22 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug  9 08:14:22 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <41167146.4070401@stackless.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<200408061557.i76FvGo07481@guido.python.org>	<4113BC98.3020407@stackless.com>
	<4113CD18.8080803@v.loewis.de>	<41140E76.2050601@stackless.com>
	<4114583C.4070500@v.loewis.de> <41167146.4070401@stackless.com>
Message-ID: <4117163E.7050203@v.loewis.de>

Christian Tismer wrote:
> What I added was yet another __stuff__ entry for new-style
> classes. It may also appear on module level, like the
> __metaclass__ feature works.

While this "works", I still think it is a bad idea, for two
reasons:
1. It only helps the parser, A human reader would have to
    scroll to the top of the class (or worse, to the top of
    the module) to find out.
2. For the author, it is redundant typing - the author *knows*
    this is a decorator, so having to type this again in full
    is redundant.
    1 and 2 together make this a write-only syntax.
3. It means that the parser has to recognize __decorators__
    This is not often the case; I believe the only other
    exception is __future__ (which I also dislike, despite
    its cuteness).
[As any good list of 2 reasons, this list of course has 3]

> But it provides a solution without enforcing us to decide
> about new syntax right now, which appears to be too early
> for a group that can't find a common denominator, yet.

There is a solution already: put the function wrappers after
the declaration. The goal of this change is specifically to
come up with a syntax. If no syntax could be find, it would
be best not to change anything.

I'm not worried that the group cannot agree on syntax:
people can *never* agree on syntax, not just in the case
of Python. If you wait for agreement, you wait forever.

> For my purposes it works fine, but I don't claim that it fulfils
> all other desired use cases. At least it is compatible, simple,
> complete, implemented in a way, and works for my applications.

It's not any more compatbible than the 2.4a2 approach:
- old code (which does not use the feature) continues to work
   correctly (*)
- new code (which uses the feature) won't work on old
   implementations.
(*) Actually, there is a slight incompatibility in your approach:
Code that used to run and used to raise an exception would, under
your change, now suddenly succeed without exception.

Regards,
Martin
From lists at hlabs.spb.ru  Mon Aug  9 12:20:32 2004
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Mon Aug  9 08:14:41 2004
Subject: [Python-Dev] Density of pie-decorator syntax
In-Reply-To: <4113AFC4.5010004@janrain.com>
References: <4113AFC4.5010004@janrain.com>
Message-ID: <41174FF0.8020103@hlabs.spb.ru>

Josh Hoyt wrote:
> One problem I have with the pie-decorator syntax is that it is very 
> typographically dense. The @ character runs in with the following 
> function name. If there are multiple lines of decorators, the problem is 
> worsened. For clarity, the decorators will have no whitespace lines 
> between them, creating a dense block.
> 
> After experimenting, I have found that readability is increased if there 
> is one space required after the @. Don't take my word for it:
> 
> Current Python 2.4a2 syntax:
> 
> @framework_stuff(lots, of, args)
> class Quux(object):
> 
>    @check_args(int, str)
>    @counted
>    @staticmethod
>    def frobnicate(foo, bar):
>        pass
> 
> Proposed change requiring a space:
> 
> @ framework_stuff(lots of args)
> class Quux(object):
> 
>    @ check_args(int, str)
>    @ counted
>    @ staticmethod
>    def frobnicate(foo, bar):
>        pass

"@ decorator" working just fine with Python 2.4.a2. Try this... :-)

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru
From martin at v.loewis.de  Mon Aug  9 08:19:30 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug  9 08:19:30 2004
Subject: [Python-Dev] Cygwin and sockets
In-Reply-To: <20040808145442.GF55644@tishler.net>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E86C541C@its-xchg4.massey.ac.nz>	<4110AABA.80407@v.loewis.de>
	<20040808145442.GF55644@tishler.net>
Message-ID: <41171772.2050602@v.loewis.de>

Jason Tishler wrote:
> See attach for the details.  OK to apply?

Yes, go ahead. I'd like to find a better way though to determine whether 
  we need to include the emulated addrinfo.h

> BTW, the socket module builds automatically for me. 

It might be that the bug reporter is mistaken in his claim that building
_socked.pyd was skipped. More likely, it failed, but setup.py proceeded
happily.

Please close the bug report when you are done, indicating that it now
works for you.

Thanks for looking into this,
Martin
From martin at v.loewis.de  Mon Aug  9 08:27:09 2004
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon Aug  9 08:27:09 2004
Subject: [Python-Dev] Decimal type question [Prothon]
In-Reply-To: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
Message-ID: <4117193D.5030001@v.loewis.de>

> Can anyone see any problem with this scheme?

How does that interact with (binary) floating point numbers?

I think the numerical people might dislike that approach, as
all their algorithms, C interfaces, and so on are heavily tied
to IEEE-754. Not being a NumPy user myself, I cannot say how
significant that is, and what could be done about it.

For example, would it be possible to automatically fall back
to binary floating point if the result cannot be represented
exactly (which would be most divide operations)? Would that
help?

Regards,
Martin
From sjoerd at acm.org  Mon Aug  9 08:53:45 2004
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Mon Aug  9 08:53:58 2004
Subject: [Python-Dev] The date in the generated documentation
In-Reply-To: <20040809024336.GC6582@unpythonic.net>
References: <20040809022934.GB6582@unpythonic.net>
	<20040809024336.GC6582@unpythonic.net>
Message-ID: <41171F79.4040601@acm.org>

Jeff Epler wrote:
> I had to edit the lines in the file /usr/share/latex2html/l2hconf.pm
> which mentioned japanese (by default!  I'm certain I never changed them)
> and change them back to english.
> 
> The generated html documentation is now much improved.
> 
> Sorry to bother everyone for what turned out to be a fedora bug /
> configuration problem.

I see this also.  I filed a bug with Fedora.

-- 
Sjoerd Mullender <sjoerd@acm.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 374 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040809/4c37abf8/signature.pgp
From martin at v.loewis.de  Mon Aug  9 09:08:50 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug  9 09:08:50 2004
Subject: [Python-Dev] PyRun_ with file name
In-Reply-To: <1f7befae040808174233fb20db@mail.gmail.com>
References: <4113EF69.2020102@v.loewis.de>
	<1f7befae040808174233fb20db@mail.gmail.com>
Message-ID: <41172302.6090001@v.loewis.de>

Tim Peters wrote:
> If you understand the problem, and believe it will fix it, +1 (I don't
> think I fully understand the problem, but it gets points from me for
> being a small change).

To fully understand it, you need to spend three days in a locked room
together with the VC debugger :-) The submitter claims that VC7 has
changed the CRT ABI, I believe this is incorrect. Instead, the problem
results from having to CRT copies simultaneously, even though they all
have the very same code sice ages. Here is the full story:

- there are two file structures, struct FILE and struct _FILEX, see
   internal.h. _FILEX is

   typedef struct {
         FILE f;
         CRITICAL_SECTION lock;
   }   _FILEX;

- For historical reasons, the lock couldn't be added to the existing
   FILE objects (stdin, stdout, etc) since compiled code *knew* how
   large struct FILE is, and where therefore _iob[2] (say) is.

- To solve this, MS added a second array of CRITICAL_SECTION (see
   mlock.c), and a helper function _lock to lock one of these.

- Now, _lock_file becomes (see _file.c)

   void __cdecl _lock_file (void *pf)
   {
     if((pf >= (void *)_iob) && (pf <= (void *)(&_iob[_IOB_ENTRIES-1])))
           _lock( _STREAM_LOCKS + (int)((FILE *)pf - _iob) );
     else
           EnterCriticalSection( &(((_FILEX *)pf)->lock) );
   }

   So essentially, if it is a builtin FILE, find its lock elsewhere,
   otherwise, it is a _FILEX, and find its lock inside the FILE itself.

- Suppose we have an embedded Python, and the container is compiled
   with VC7 (or 5, doesn't matter). The host application does

     FILE *f = fopen("some file", "r");
     PyRun_SimpleFile(f);

   This allocates a builtin FILE (say, fileno 5) inside the VC7 CRT, and
   passes it to python23.dll. We do fread, and fread (from the VC6 CRT)
   does _lock_file. This checks whether this is a builtin FILE, which it
   isn't, so the VC6 CRT assumes it is a FILEX, and uses the
   CRTITICAL_SECTION. Of course, this is really the storage of fileno 6
   of the VC7 CRT, which is not an initialized CRTICAL_SECTION.

The end result is that the _lock_file operation crashes. As MSDN
explains, you must not pass FILE* across CRTs. The proposed patch avoids
the problem by passing file names instead.

Regards,
Martin


From stephen at xemacs.org  Mon Aug  9 09:19:31 2004
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon Aug  9 09:19:39 2004
Subject: [Python-Dev] pre-PEP [corrected]: Complete, Structured Regular
	Expression Group Matching
In-Reply-To: <lkdwu0bcz7r.fsf@panix2.panix.com> (Mike Coleman's message of
	"07 Aug 2004 01:08:08 -0400")
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<lkdwu0bcz7r.fsf@panix2.panix.com>
Message-ID: <87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Mike" == Mike Coleman <mkc@mathdogs.com> writes:

    Mike> Motivation
    Mike> ==========

    Mike> A notable limitation of the ``re.match`` method is that it
    Mike> fails to capture all group match information for repeatedly
    Mike> matched groups.  For example, in a call like this ::

    Mike>     m0 = re.match(r'([A-Z]+|[a-z]+)*', 'XxxxYzz')

    Mike> one would like to see that the group which matched four
    Mike> times matched the strings ``'X'``, ``'xxx'``, ``'Y'`` and
    Mike> ``'zz'``.

Sure, but regexp syntax is a horrible way to express that.  This
feature would be an attractive nuisance, IMHO.  For example:

    Mike> Parsing ``/etc/group``
    Mike> ----------------------

    Mike> If ``/etc/group`` contains the following lines ::

    Mike>     root:x:0:
    Mike>     daemon:x:1:
    Mike>     bin:x:2:
    Mike>     sys:x:3:

    Mike> then it can be parsed as follows ::

    Mike>     p = r'((?:(?:^|:)([^:\n]*))*\n)*\Z'

This is a _easy_ one, but even it absolutely requires being written
with (?xm) and lots of comments, don't you think?  If you're going to
be writing a multiline, verbose regular expression, why not write a
grammar instead, which (assuming a modicum of library support) will be
shorter and self-documenting?


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.
From lists at hlabs.spb.ru  Mon Aug  9 13:52:54 2004
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Mon Aug  9 09:47:02 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
Message-ID: <41176596.5060209@hlabs.spb.ru>

Is there a plan for implementing a base class for int and long (like 
basestring for str and unicode):

 >>> issubclass(int, baseint) and issubclass(long, baseint)
True

?

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru
From tismer at stackless.com  Mon Aug  9 11:04:05 2004
From: tismer at stackless.com (Christian Tismer)
Date: Mon Aug  9 11:04:02 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <4117163E.7050203@v.loewis.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<200408061557.i76FvGo07481@guido.python.org>	<4113BC98.3020407@stackless.com>
	<4113CD18.8080803@v.loewis.de>	<41140E76.2050601@stackless.com>
	<4114583C.4070500@v.loewis.de> <41167146.4070401@stackless.com>
	<4117163E.7050203@v.loewis.de>
Message-ID: <41173E05.40003@stackless.com>

Martin v. L?wis wrote:

> Christian Tismer wrote:
> 
>> What I added was yet another __stuff__ entry for new-style
>> classes. It may also appear on module level, like the
>> __metaclass__ feature works.
> 
> 
> While this "works", I still think it is a bad idea, for two
> reasons:

Just as a remark: You are shifting topic, now.
We were not discussign what's nice or good or
not, it was about a correct proposal, which
I supplied. I was not intending to discuss
if this is a good approach.

> 1. It only helps the parser, A human reader would have to
>    scroll to the top of the class (or worse, to the top of
>    the module) to find out.

compare __slots__, compare __metaclass__, and I mentioned
already that these are intermediate construct, too.

> 2. For the author, it is redundant typing - the author *knows*
>    this is a decorator, so having to type this again in full
>    is redundant.
>    1 and 2 together make this a write-only syntax.

This argument is a little overdone. To repeat the function
name just once isn't that bad.

> 3. It means that the parser has to recognize __decorators__
>    This is not often the case; I believe the only other
>    exception is __future__ (which I also dislike, despite
>    its cuteness).
> [As any good list of 2 reasons, this list of course has 3]

The parser doesn't have to recognize __decorators__. See
the implementation of __metaclass__. This is a lookup
of a name, like it happens already. But it is true, the
parser then has to generate different code for the members
of the __decorators__ sequence.

>> But it provides a solution without enforcing us to decide
>> about new syntax right now, which appears to be too early
>> for a group that can't find a common denominator, yet.
> 
> There is a solution already: put the function wrappers after
> the declaration. The goal of this change is specifically to
> come up with a syntax. If no syntax could be find, it would
> be best not to change anything.

(**) That's of course a very good argument.

> I'm not worried that the group cannot agree on syntax:
> people can *never* agree on syntax, not just in the case
> of Python. If you wait for agreement, you wait forever.

Ok. My real fear is that we get to a solution which will
spoil Python's appearance in an unrevertable way, just
because the relevant people are getting bored by discussion
and want to stop it by supplying a premature solution.

...

> It's not any more compatbible than the 2.4a2 approach:
> - old code (which does not use the feature) continues to work
>   correctly (*)
> - new code (which uses the feature) won't work on old
>   implementations.
> (*) Actually, there is a slight incompatibility in your approach:
> Code that used to run and used to raise an exception would, under
> your change, now suddenly succeed without exception.

I don't see it as a requirement that code that ran before,
continues to run after a new construct is introduced. By adding
__decorators__ to your class, you decide to use this feature,
you also change its implementation and put the decorators where
you like them. Relying on exceptions which happened before the
change is nothing that I want to be concerned with.
Changing from old-style to new-style classes is also an operation
that will cause small semantic differences, but I never heared
someone arguing against this.

I don't think you need such far-fetched argumentation to
kill this proposal. Stick with (**), it is convincing enough. :-)

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From tismer at stackless.com  Mon Aug  9 11:16:52 2004
From: tismer at stackless.com (Christian Tismer)
Date: Mon Aug  9 11:21:49 2004
Subject: [Python-Dev] capturing RETURN_VALUE
In-Reply-To: <1f7befae04080816325f3fa2f2@mail.gmail.com>
References: <41165D23.4070202@stackless.com>
	<1f7befae040808160924e4d8a0@mail.gmail.com>
	<4116B4BF.5030409@stackless.com>
	<1f7befae04080816325f3fa2f2@mail.gmail.com>
Message-ID: <41174104.2060606@stackless.com>

Tim Peters wrote:

...

> The point was that exceptions and returns act the same way: 
> RETURN_VALUE isn't unique here, it's just another instance of leaving
> a try-block's suite, the same in this respect as an exception or a
> break statement.  That's why it's reliable over time:  it's a general
> mechanism at work, not an arbitrary hack specific to RETURN_VALUE.

Yes, that was special for me: That return is _not_ special.
I should have known because I know the implementation so
very well, but I always felt that return is something final.
Maybe it would make sense to add a few sentences about
this fact somewhere (where?) in the docs; It was quite
remarkable for me.

Actually, there is nothing final in Python: It is not possible
to deduce from an inner code block whether it will be able
to finish the enclosing function, unless you know that your
code block consumes 20 block levels (and the latter is
probably a much less reliable fact. :-)

return-finally-break - ly y'rs -- chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From walter at livinglogic.de  Mon Aug  9 12:38:58 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon Aug  9 12:39:04 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <41141255.5010408@stackless.com>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<1f7befae040806090644be9c04@mail.gmail.com>	<cf0gd1$smv$1@sea.gmane.org>	<1091818207.7529.34.camel@localhost>	<4113D5E1.6040006@interlink.com.au>
	<41141255.5010408@stackless.com>
Message-ID: <41175442.4010902@livinglogic.de>

Christian Tismer wrote:

> Anthony Baxter wrote:
> 
>> Mark Russell wrote:
>>
>>> I too like |.  It is not quite as visible as @, but I don't think that's
>>> a serious problem, and not breaking ipython and other tools is a big
>>> win.  I just tried implementing this change, and all the tests still
>>> pass (not a big surprise, but worth confirming I guess).
>>
>> To jump on the band-wagon - I can live with | as well. I was going to
>> add it to the PEP rewrite, but I'll wait for a decision, to save myself
>> the writing ;)
> 
> Ok, I dislike special prefix chars at all, in a language that
> doesn't have this concept elsewhere (despite strings of course,
> but their prefixes are just regular chars),

appart from * and ** for argument passing.

I can't understand why we can't have a new keyword for decorators.
If I remember correctly the introduction of yield didn't result
in such a public outcry. We'd have to change our programs once
if a variable names collides with the new keyword, but that's better
than having to look at @s for the rest of our Python days.

So how about:

make classmethod
def foo(cls, bar):
    ...

> but the bar "|" appears a lot nicer than "@" to me.
> The bar is small and friendly, a small character for a small
> feature. With "@", I associate a powerful, magical Voodoo
> thing, something that I never expected to see in Python.
> Unless there is a real need, I'd say save "@" for a feature that
> really deserves such a powerful character.

Bye,
    Walter D?rwald

From mwh at python.net  Mon Aug  9 12:45:10 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug  9 12:45:12 2004
Subject: [Python-Dev] new failing test -- test_compiler
In-Reply-To: <1f7befae0408080946767c0a1c@mail.gmail.com> (Tim Peters's
	message of "Sun, 8 Aug 2004 12:46:09 -0400")
References: <e8bf7a53040807122937ba5105@mail.gmail.com>
	<1f7befae0408071253544d2c31@mail.gmail.com>
	<1f7befae04080712597b9e7e0b@mail.gmail.com>
	<2misbu3cbp.fsf@starship.python.net>
	<20040807222357.GA27723@vicky.ecs.soton.ac.uk>
	<1f7befae04080717357814d502@mail.gmail.com>
	<e8bf7a53040808072658e6b779@mail.gmail.com>
	<1f7befae0408080946767c0a1c@mail.gmail.com>
Message-ID: <2mbrhk3809.fsf@starship.python.net>

Tim Peters <tim.peters@gmail.com> writes:

> [Tim]
>>> Alas, test_compiler still fails (but in an ordinary "doesn't work"
>>> way, no longer in an internal-error way), so testing with -uall is
>>> still a bit broken.
>
> [Jeremy]
>> I won't be able to fix it today, but I'll get on that asap.
>
> That's OK.  I agree it's valuable to run this, so I fixed it. 
> test_compiler passes on Windows now.  But I have a gigabyte of RAM on
> this box to hold the stack <wink>.

I actually worked out the fix on Saturday but succumbed to hunger
before the test run finished :-)

Cheers,
mwh

-- 
  I never realized it before, but having looked that over I'm certain
  I'd rather have my eyes burned out by zombies with flaming dung
  sticks than work on a conscientious Unicode regex engine.
                                             -- Tim Peters, 3 Dec 1998
From mwh at python.net  Mon Aug  9 13:05:04 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug  9 13:05:06 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <41176596.5060209@hlabs.spb.ru> (Dmitry Vasiliev's message of
	"Mon, 09 Aug 2004 11:52:54 +0000")
References: <41176596.5060209@hlabs.spb.ru>
Message-ID: <2m7js83733.fsf@starship.python.net>

Dmitry Vasiliev <lists@hlabs.spb.ru> writes:

> Is there a plan for implementing a base class for int and long (like
> basestring for str and unicode):
>
>  >>> issubclass(int, baseint) and issubclass(long, baseint)
> True

Not that I'm aware of.  Personally, I think it's slightly more likely
that we'll have:

>>> int is long
True

but this is a long way off, and could cause upset with people who need
to do bit bashing or interface with C.

Of course,

>>> baseint = (int, long)

makes what you type accurate today :-)

Cheers,
mwh

-- 
  > so python will fork if activestate starts polluting it?
  I find it more relevant to speculate on whether Python would fork
  if the merpeople start invading our cities riding on the backs of 
  giant king crabs.                 -- Brian Quinlan, comp.lang.python
From lists at hlabs.spb.ru  Mon Aug  9 17:38:18 2004
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Mon Aug  9 13:32:26 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <2m7js83733.fsf@starship.python.net>
References: <41176596.5060209@hlabs.spb.ru>
	<2m7js83733.fsf@starship.python.net>
Message-ID: <41179A6A.90905@hlabs.spb.ru>

Michael Hudson wrote:
> Dmitry Vasiliev <lists@hlabs.spb.ru> writes:
> 
>>Is there a plan for implementing a base class for int and long (like
>>basestring for str and unicode):
>>
>> >>> issubclass(int, baseint) and issubclass(long, baseint)
>>True
> 
> Not that I'm aware of.  Personally, I think it's slightly more likely
> that we'll have:
> 
> 
>>>>int is long
> 
> True
> 
> but this is a long way off, and could cause upset with people who need
> to do bit bashing or interface with C.

Quote from PEP-237:

"""
A new type,
integer, may be introduced that is an abstract base type of
which both the int and long implementation types are
subclassed.  This is useful so that programs can check
integer-ness with a single test:

      if isinstance(i, integer): ...
"""

So maybe correct question then: is there a plan for implementing the 
integer type in Python 2.4?

> 
> Of course,
> 
>>>>baseint = (int, long)
> 
> makes what you type accurate today :-)

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru
From anthony at interlink.com.au  Mon Aug  9 14:58:26 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Aug  9 14:59:17 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <41179A6A.90905@hlabs.spb.ru>
References: <41176596.5060209@hlabs.spb.ru>	<2m7js83733.fsf@starship.python.net>
	<41179A6A.90905@hlabs.spb.ru>
Message-ID: <411774F2.7040408@interlink.com.au>

Dmitry Vasiliev wrote:
> 
> Quote from PEP-237:
> 
> """
> A new type,
> integer, may be introduced that is an abstract base type of
> which both the int and long implementation types are
> subclassed.  This is useful so that programs can check
> integer-ness with a single test:
> 
>      if isinstance(i, integer): ...
> """
> 
> So maybe correct question then: is there a plan for implementing the 
> integer type in Python 2.4?

Well, we're a couple of weeks from what will hopefully be the last
alpha. I'd _hope_ anything like this would be in before then (although
we can have things that change behaviour up until b1, I'd prefer that
a3->b1 is just fixing bugs that require changes, or minor changes, not
major changes). OTOH, I'm not sure how deep a change like this would be.

In any case, if it's a change that you (or anyone else) feels strongly
about, a patch on SF would be a first step towards making this happen.

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From s.percivall at chello.se  Mon Aug  9 15:02:42 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Mon Aug  9 15:02:44 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <41175442.4010902@livinglogic.de>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<1f7befae040806090644be9c04@mail.gmail.com>	<cf0gd1$smv$1@sea.gmane.org>	<1091818207.7529.34.camel@localhost>	<4113D5E1.6040006@interlink.com.au>
	<41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de>
Message-ID: <6682D8E6-EA04-11D8-8B30-0003934AD54A@chello.se>

On 2004-08-09, at 12.38, Walter D?rwald wrote:
> I can't understand why we can't have a new keyword for decorators.
> If I remember correctly the introduction of yield didn't result
> in such a public outcry. We'd have to change our programs once
> if a variable names collides with the new keyword, but that's better
> than having to look at @s for the rest of our Python days.
>
> So how about:
>
> make classmethod
> def foo(cls, bar):
>    ...

I think "adorn" would make a better keyword in that case, as in
"adorn with" this decorator the method foo.

//Simon
From bob at redivi.com  Mon Aug  9 15:33:06 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug  9 15:33:16 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <6682D8E6-EA04-11D8-8B30-0003934AD54A@chello.se>
References: <200408051636.i75Gaak03654@guido.python.org>	<20040805174252.GA27820@burma.localdomain>	<20040805112318.B13062@ActiveState.com>	<20040805183444.GA29796@burma.localdomain>	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>	<1091806774.8414.63.camel@localhost>	<1f7befae040806090644be9c04@mail.gmail.com>	<cf0gd1$smv$1@sea.gmane.org>	<1091818207.7529.34.camel@localhost>	<4113D5E1.6040006@interlink.com.au>
	<41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de>
	<6682D8E6-EA04-11D8-8B30-0003934AD54A@chello.se>
Message-ID: <A5C91C16-EA08-11D8-9661-000A95686CD8@redivi.com>


On Aug 9, 2004, at 9:02 AM, Simon Percivall wrote:

> On 2004-08-09, at 12.38, Walter D?rwald wrote:
>> I can't understand why we can't have a new keyword for decorators.
>> If I remember correctly the introduction of yield didn't result
>> in such a public outcry. We'd have to change our programs once
>> if a variable names collides with the new keyword, but that's better
>> than having to look at @s for the rest of our Python days.
>>
>> So how about:
>>
>> make classmethod
>> def foo(cls, bar):
>>    ...
>
> I think "adorn" would make a better keyword in that case, as in
> "adorn with" this decorator the method foo.

I don't think that's appropriate at all.  Most decorators are 
transformations that change the function object into something else 
entirely, most likely by wrapping it but quite possibly not.

I think the idea of adding a new keyword has already been dismissed, 
though.

-bob
From dave at boost-consulting.com  Mon Aug  9 15:34:58 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Mon Aug  9 15:35:34 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <200408071617.i77GHgs10281@guido.python.org>
	<04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <uy8koxwn1.fsf@boost-consulting.com>

Bill Janssen <janssen@parc.com> writes:

>> In the discussion on decorators months ago, solutions involving
>> special syntax inside the block were ruled out early on.  Basically,
>> you shouldn't have to peek inside the block to find out important
>> external properties of the function.
>
> Guido, could you expand on this principle a bit?  Just stated as it
> is, it sounds as pointless (to my untutored mind, at least :-), as the
> arguments about "@ is ugly, so it shouldn't be used".  After all,
> isn't "what the function does" an important external property of the
> function, and don't you have to peek inside the block to determine
> that?  For instance, how do you know whether a function is a generator
> unless you look for yield statements inside it?

I'm not Guido (obviously), but:

  You look at the documentation.  The fact that Python doesn't divorce
  interface declarations from implementation is convenient for
  authors, but in some ways a trap for readers/maintainers (this is
  not a design criticism -- it was probably the right design choice).
  I often have to look inside Python functions to understand them, and
  find that their authors expect me to do so.  I think a style that
  cleanly delimits interface and implementation as much as possible
  should be encouraged. The fact that docstrings sit inside the
  function block make that harder to achieve; that's why I'm glad to
  hear that prefix decorators are being considered for docstrings.

That said, I'm not sure that "immediately following the opening colon"
is such a bad place for interface-y things.  It's nice that you can
find the whole function's declaration by looking forward from "def
..." today [except for the decorators, of course ;-(].

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From dave at boost-consulting.com  Mon Aug  9 15:42:31 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Mon Aug  9 15:50:28 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain> <ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de>
	<41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
	<4115FF35.4030608@v.loewis.de>
	<Pine.LNX.4.58.0408081431050.6199@rnd.onego.ru>
	<411610FB.9040201@v.loewis.de>
	<Pine.LNX.4.58.0408081703430.7067@rnd.onego.ru>
	<59e9fd3a04080807177dd446af@mail.gmail.com>
	<200408081525.i78FPJm20862@guido.python.org>
	<59e9fd3a04080822284be45e94@mail.gmail.com>
Message-ID: <usmawxwag.fsf@boost-consulting.com>

Andrew Durdin <adurdin@gmail.com> writes:

> In the general case, this is probably true; but this argument of mine
> is valid for any level of Python user familiar with versions up to
> 2.3, who has *only* not seen of or heard of the new decorator syntax.
> This is because the @decorator syntax has a hidden side-effect, when
> compared with normal Python statements. It is not at all consistent
> with *any* other valid Python syntax that I know.
>
> Consistency may not always be desirable; but I would have thought it
> generally agreed that introducing a severe inconsistency for the sake
> of only one feature is not a good design choice. If you believe that
> introducing 'new-style' statements/expressions that implicitly affect
> what follows them is a good idea *in general*, and is well-suited to
> many features other than decorator expressions, then by all means,
> introduce it first for decorators, and subsequently for these other
> features--but at least let us know *why* these 'new-style'
> statements/expressions are a good idea in general (not just for
> decorators). Because frankly, and with respect, I haven't seen any
> compelling arguments that introducing new statements/expressions with
> implicit, non-obvious effects is a good thing for Python. To me, such
> a thing reminds me of the horrible implicitly-set $_ and $? (and other
> such creatures) in Perl.

Seeing as how we're now firmly back in the land of opinions, voting,
and trying to influence the BDFL, I'd like to add:

  I agree with Andrew.  Whether or not the implementation uses
  sys.settrace internally, it still feels like it has the same weird,
  stateful semantics of temporarily affecting the next executable
  statement and then evaporating.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From FBatista at uniFON.com.ar  Mon Aug  9 15:48:50 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Aug  9 15:52:56 2004
Subject: [Python-Dev] decimal.Context.copy() shallow or deep?
Message-ID: <A128D751272CD411BC9200508BC2194D053C7785@escpl.tcp.com.ar>

[Raymond Hettinger]

#- > > Do you guys think it should be left alone (shallow) or 
#- changed (made
#- > > deep)?
#- > 
#- > A user-visible copy certainly should not share mutable 
#- state with the
#- > original. 
#- 
#- I was afraid of that.  What a PITA.

My opinion is that the important step here is to fix the behaviour before
alpha-3 or beta (whatever cames first).

So, it's nice your idea to rename context.copy() to context._shallow_copy(),
make context.copy() deep, and then, in all the tests that fail, change copy
for the shallow one.

Later, we'll have time to maybe fix that and don't depend of a shallow copy
(work around that WorkRep would be nice also).

.	Facundo
From bkc at murkworks.com  Mon Aug  9 16:27:05 2004
From: bkc at murkworks.com (Brad Clements)
Date: Mon Aug  9 16:20:58 2004
Subject: [Python-Dev] PyRun_ with file name
In-Reply-To: <41172302.6090001@v.loewis.de>
References: <1f7befae040808174233fb20db@mail.gmail.com>
Message-ID: <41175178.1484.ED32EF69@coal.murkworks.com>

On 9 Aug 2004 at 9:08, Martin v. L?wis wrote:

> The end result is that the _lock_file operation crashes. As MSDN
> explains, you must not pass FILE* across CRTs. The proposed patch avoids
> the problem by passing file names instead.

I vaugely recall having a similar problem embedding Python perhaps two years ago. At that 
time I had contemplated adding a function to python.dll along the lines of 

FILE * Py_OpenFile(char *filename, char *mode) 

The sole purpose of this function was to allow embedding applications to use 
PyRun_SimpleFile and similar python.dll functions that expect a (crt) FILE object. 

I'm sorry I can't remember the exact use case. I'm not proposing Py_OpenFile as a solution 
to this problem. 

I just wanted to speak up and say "me too". 

+1



-- 
Brad Clements,                bkc@murkworks.com   (315)268-1000
http://www.murkworks.com                          (315)268-9812 Fax
http://www.wecanstopspam.org/                   AOL-IM: BKClements

From jason at tishler.net  Mon Aug  9 16:52:42 2004
From: jason at tishler.net (Jason Tishler)
Date: Mon Aug  9 16:48:46 2004
Subject: [Python-Dev] Cygwin and sockets
In-Reply-To: <41171772.2050602@v.loewis.de>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E86C541C@its-xchg4.massey.ac.nz>
	<4110AABA.80407@v.loewis.de> <20040808145442.GF55644@tishler.net>
	<41171772.2050602@v.loewis.de>
Message-ID: <20040809145241.GA67536@tishler.net>

Martin,

On Mon, Aug 09, 2004 at 08:19:30AM +0200, "Martin v. L?wis" wrote:
> Jason Tishler wrote:
> >See attach for the details.  OK to apply?
> 
> Yes, go ahead.

Done.

> I'd like to find a better way though to determine whether we need to
> include the emulated addrinfo.h

Agreed, which was I checked before committing my simplistic patch.

> >BTW, the socket module builds automatically for me. 
> 
> It might be that the bug reporter is mistaken in his claim that
> building _socked.pyd was skipped. More likely, it failed, but setup.py
> proceeded happily.

I thought the above was a likely possibility too.

> Please close the bug report when you are done, indicating that it now
> works for you.

Done.

> Thanks for looking into this,

You are quite welcome -- thanks for the heads up.

BTW, I discovered that the following modules fail to build under Cygwin
too:

    strop
    cPickle
    readline

I will submit a patch to fix these problems shortly.

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6
From guido at python.org  Mon Aug  9 17:04:47 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 17:06:01 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: Your message of "Sun, 08 Aug 2004 18:31:49 PDT."
	<04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> 
References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com> 
Message-ID: <200408091504.i79F4lD22945@guido.python.org>

[Guido]
> > In the discussion on decorators months ago, solutions involving
> > special syntax inside the block were ruled out early on.  Basically,
> > you shouldn't have to peek inside the block to find out important
> > external properties of the function.

[Bill Janssen]
> Guido, could you expand on this principle a bit?  Just stated as it
> is, it sounds as pointless (to my untutored mind, at least :-), as the
> arguments about "@ is ugly, so it shouldn't be used".  After all,
> isn't "what the function does" an important external property of the
> function, and don't you have to peek inside the block to determine
> that?  For instance, how do you know whether a function is a generator
> unless you look for yield statements inside it?

It's probably untenable as a formal principle. :-)

Nevertheless I don't like to put decorators in the method body.  I
think it has something to do with the fact that normally one thinks of
the body contents as relevalt to "call-time" while decorators happen
at "definition-time".

And before you bring up docstrings as a counterexample, yes, they
should ideally have been designed to precede the method, too.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From python at rcn.com  Mon Aug  9 05:03:40 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug  9 17:06:45 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
In-Reply-To: <1qygyzxyh5qnx.1r2aymzvv6xlw$.dlg@40tude.net>
Message-ID: <002601c47dbd$79161dc0$191fc797@oemcomputer>

> I am more interested now in hearing about people's feelings on the
idea of
> having the Decimal type "in-between" Int and Float.  Assume a decent
> Decimal implementation is used.

FWIW, Lua used floating point for everything.  With integers upto 53
bits being represented exactly, everything works out fine.  Plus is has
a nice side effect of simplifying the language and underlying code.



Raymond

From guido at python.org  Mon Aug  9 17:06:49 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 17:07:07 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: Your message of "Mon, 09 Aug 2004 14:16:45 +1200."
	<200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> 
References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408091506.i79F6nc22962@guido.python.org>

> If @decorator stays in, I fervently hope it won't be taken as a
> precedent for using @ for other things all over the place.
> Otherwise we really *will* end up with a language that resembles
> Perl.

That's dangerously close to rhetorical nonsense.  @ will always be
used to indicate meta-information (roughly speaking), while in Perl it
indicates a (certain kind of) variable.  Very different.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From fredrik at pythonware.com  Mon Aug  9 17:11:45 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Aug  9 17:10:40 2004
Subject: [Python-Dev] Re: Multi-line import implementation (was: 2.4a2,
	and @decorators)
References: <410DDFE3.7010006@interlink.com.au><200408021850.i72IoCg28650@guido.python.org><410EF963.40704@interlink.com.au>
	<20040809043104.GA82909@trit.org>
Message-ID: <cf844a$rf8$1@sea.gmane.org>

Dima Dorfman wrote:

> I took a stab at implementing this (the easy part of PEP 328--parens
> around import lists). Are the exact semantics of what's allowed
> documented somewhere?

I think this syntax was originally proposed here:

http://mail.python.org/pipermail/python-dev/2003-December/041072.html

which mentions that since "a, b = c" is already equivalent to "(a, b) = c", so
"
in mind, I'd hand out the following scores (0=don't really care):

>   import (os, sys)

+0

>   from sys import (stdin, stdout, stderr)

+1

>   import (os)

+0

>   from sys import (*)

-0

>   from (sys) import stdin

-0

>   import (os), (sys)

-1

>   import (os,)

-1

</F>



From jhylton at gmail.com  Mon Aug  9 17:13:41 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Mon Aug  9 17:13:46 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <41145C05.1020501@gradient.cis.upenn.edu>
References: <41145C05.1020501@gradient.cis.upenn.edu>
Message-ID: <e8bf7a5304080908135a142da@mail.gmail.com>

On Sat, 07 Aug 2004 00:35:17 -0400, Edward Loper
<edloper@gradient.cis.upenn.edu> wrote:
> The new "vertical bar" syntax was proposed recently, and has gotten
> almost unanimously positive feedback.  In particular, of 17 responses
> to the syntax, only one has been negative (see table, below).

I dislike it.  It's not as friendly or visible as the @.

Jeremy
From guido at python.org  Mon Aug  9 17:16:38 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 17:16:42 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: Your message of "Mon, 09 Aug 2004 12:37:05 +1000."
	<019301c47db9$c269ec80$a9498890@neil> 
References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz>  
	<019301c47db9$c269ec80$a9498890@neil> 
Message-ID: <200408091516.i79FGc722993@guido.python.org>

>    A motivation for this was that one of the examples
> 
> @accepts(int,int)
> @returns(float)
> def bar(low,high):
> 
>    which separates related information too much for me. If
> decorators need to decorate parameters, they are more likely to be
> understood correctly by a reader if the decorations are next to the
> parameters.

Let's make one thing clear; while decorators may be useful for some to
experiment with attaching signatures to methods, mainstream Python
will eventually have optional static typing built in, and it won't use
decorator syntax.  (More likely you would specify argument types by
saying e.g. "def bar(low: int, high: int) -> float: ...")

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Mon Aug  9 17:25:21 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 17:25:26 2004
Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2,
	and @decorators)
In-Reply-To: Your message of "Mon, 09 Aug 2004 04:31:05 -0000."
	<20040809043104.GA82909@trit.org> 
References: <410DDFE3.7010006@interlink.com.au>
	<200408021850.i72IoCg28650@guido.python.org>
	<410EF963.40704@interlink.com.au> 
	<20040809043104.GA82909@trit.org> 
Message-ID: <200408091525.i79FPLM23023@guido.python.org>

> > The 2.4 release schedule PEP now lists this as lacking someone
> > to drive it. I won't/can't spend time on it. So, if one of the people
> > who wants it could step forward, that'd be excellent. Even if it's
> > just to do the multi-line import syntax of
> > 
> >   from foo import ( bar, baz, bar2,
> >                     baz2, bar3, baz3 )
> 
> I took a stab at implementing this (the easy part of PEP 328--parens
> around import lists).

Great!

> Are the exact semantics of what's allowed documented somewhere?

Not unless it's in the PEP -- it usually takes a stab at implementing
something to find what's left unspecified!

> PEP 328 mostly talks about relative and absolute
> imports, and it doesn't specify the exact semantics of where
> parentheses should be allowed. My patch (attached) accepts
> 
>   import (os, sys)
>   from sys import (stdin, stdout, stderr)
>   import (os)
>   from sys import (*)
> 
> but rejects
> 
>   from (sys) import stdin
>   import (os), (sys)
>   import (os,)
> 
> Should any of those be allowed? Anything that I missed?

This suggests (given the recent criticism of how we've been following
the PEP process) that the PEP needs to be updated first.

My own gut feeling is that it's not important to allow parentheses on
"bare" (from-less) imports, since you can easily repeat the 'import'
keyword on a new line; but that it is important on the from...import
form, and then only after the "import" keyword.  So that would accept
your 2nd and 4th example but reject the 1st and 3rd.

I guess one can argue about (*); I don't care about it and suggest
that it be dropped.

> The patch is incomplete in other ways; the docs haven't been updated
> and neither have the parser module and compile package. If it's
> decided that it would be okay to include this separately from
> relative/absolute import support (they're different features,
> really), I'll complete the patch.

Please do, and please use the SF patch manager to upload the patch.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Mon Aug  9 17:41:11 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 17:41:18 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: Your message of "Mon, 09 Aug 2004 08:14:22 +0200."
	<4117163E.7050203@v.loewis.de> 
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de>
	<41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de>
	<41167146.4070401@stackless.com> <4117163E.7050203@v.loewis.de> 
Message-ID: <200408091541.i79FfBl23117@guido.python.org>

> [As any good list of 2 reasons, this list of course has 3]

Cute.  Perhaps there are only two good reasons amongst those three?
:-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Mon Aug  9 17:39:45 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 17:42:25 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: Your message of "Mon, 09 Aug 2004 15:28:52 +1000."
	<59e9fd3a04080822284be45e94@mail.gmail.com> 
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<200408061557.i76FvGo07481@guido.python.org>
	<4113BC98.3020407@stackless.com> <4113CD18.8080803@v.loewis.de>
	<41140E76.2050601@stackless.com> <4114583C.4070500@v.loewis.de>
	<Pine.LNX.4.58.0408081400150.6199@rnd.onego.ru>
	<4115FF35.4030608@v.loewis.de>
	<Pine.LNX.4.58.0408081431050.6199@rnd.onego.ru>
	<411610FB.9040201@v.loewis.de>
	<Pine.LNX.4.58.0408081703430.7067@rnd.onego.ru>
	<59e9fd3a04080807177dd446af@mail.gmail.com>
	<200408081525.i78FPJm20862@guido.python.org> 
	<59e9fd3a04080822284be45e94@mail.gmail.com> 
Message-ID: <200408091539.i79Fdjr23097@guido.python.org>

> that the decorator can fundamentally alter the meaning of the def
> statement is not at all clear -- and it ought to be!  By contrast,
> those proposed syntaxes which have the decorators in the def
> statement (somewhere between the def and the colon) do not suffer
> this opacity.

I think that's too strict a rule.  You're asking for something to be
intuitively clear from looking at the syntax alone, without having
learned it before.  All I'm asking for is that it's easy to remember
once explained.  Very few things beyond 2+2 are intuitively clear,
even in Python; almost everything is easy to explain and remember
though, and I think that @decorators fall in that category, too.

(And of course, to anyone who's programmed in Java 1.5 before,
@decorators *will* be intuitively clear.  But that's not the main
motivation.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Mon Aug  9 17:47:10 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 17:47:17 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: Your message of "Mon, 09 Aug 2004 11:52:54 -0000."
	<41176596.5060209@hlabs.spb.ru> 
References: <41176596.5060209@hlabs.spb.ru> 
Message-ID: <200408091547.i79FlAe23178@guido.python.org>

> Is there a plan for implementing a base class for int and long (like 
> basestring for str and unicode):
> 
>  >>> issubclass(int, baseint) and issubclass(long, baseint)
> True
> 
> ?

I think this would be a good idea; maybe the name should be
baseinteger?

I think it's been suggested before, and can't remember why it wasn't
implemented -- perhaps the use cases aren't as compelling as for
basestring?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From fredrik at pythonware.com  Mon Aug  9 17:55:59 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Aug  9 17:54:21 2004
Subject: [Python-Dev] Re: Multi-line import implementation (was: 2.4a2,
	and @decorators)
References: <410DDFE3.7010006@interlink.com.au><200408021850.i72IoCg28650@guido.python.org><410EF963.40704@interlink.com.au><20040809043104.GA82909@trit.org>
	<cf844a$rf8$1@sea.gmane.org>
Message-ID: <cf86n9$2sa$1@sea.gmane.org>

I wrote:

> which mentions that since "a, b = c" is already equivalent to "(a, b) = c", so
> "
> in mind, I'd hand out the following scores (0=don't really care):

JZ took control of my keyboard.  sorry for that.

the text above should have been:

    which mentions that since "a, b = c" is already equivalent to "(a, b) = c", so
    "from x import a, b" might as well made be equivalent to "from x import (a, b)".
    with that in mind, I'd hand out the following scores (0=don't really care)

</F>



From fredrik at pythonware.com  Mon Aug  9 18:02:28 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Aug  9 18:00:53 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz>
	<019301c47db9$c269ec80$a9498890@neil>
	<200408091516.i79FGc722993@guido.python.org>
Message-ID: <cf873e$3vo$1@sea.gmane.org>

Guido van Rossum wrote:

> Let's make one thing clear; while decorators may be useful for some to
> experiment with attaching signatures to methods, mainstream Python
> will eventually have optional static typing built in, and it won't use
> decorator syntax.  (More likely you would specify argument types by
> saying e.g. "def bar(low: int, high: int) -> float: ...")

that could, of course, be seen as rather strong argument against having
decorators after the def but before the colon; let's reserve that space for
information about the return type (or types).

(more about decorators later)

</F>



From tim.peters at gmail.com  Mon Aug  9 18:05:01 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  9 18:05:13 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
In-Reply-To: <002601c47dbd$79161dc0$191fc797@oemcomputer>
References: <002601c47dbd$79161dc0$191fc797@oemcomputer>
Message-ID: <1f7befae04080909057a876605@mail.gmail.com>

[Raymond Hettinger]
> FWIW, Lua used floating point for everything.  With integers upto 53
> bits being represented exactly, everything works out fine.  Plus is has
> a nice side effect of simplifying the language and underlying code.

Perl also does, although it later added an actual integer type too. 
It doesn't help with 0.1 not being equal to one-tenth, etc.
From tim.peters at gmail.com  Mon Aug  9 18:23:51 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  9 18:23:57 2004
Subject: [Python-Dev] Decimal type question [Prothon]
In-Reply-To: <4117193D.5030001@v.loewis.de>
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<4117193D.5030001@v.loewis.de>
Message-ID: <1f7befae04080909231d5b5bc@mail.gmail.com>

[Martin v. L?wis]
> ...
> For example, would it be possible to automatically fall back
> to binary floating point if the result cannot be represented
> exactly (which would be most divide operations)? Would that
> help?

It's a puzzle.  .NET Decimal is really more a fixed-point type than a
floating-point type.  It consists of a sign bit, a 96-bit binary
integer, and a "scale factor" in 0..28, which is the power of 10 by
which the integer is conceptually divided.  The largest positive
representable value is thus 2**96 == 79228162514264337593543950336. 
The smallest positive non-zero representable value is 1/10**28.

So for something like 1/3, you get about 28 decimal digits of good
result, which is much better than you can get with an IEEE double.

OTOH, something like 1/300000000000000000000 starts to make the
"gradual underflow" nature of Decimal apparent: for numbers with
absolute value less than 1, the number of digits you get decreases the
smaller the absolute value, until at 1e-28 you have only 1 bit of
precision (and, e.g., 1.49999e-28 "rounds to" 1e-28).

So it's a weird arithmetic as you approach its limits.  But binary FP
is too, and so is IBM's decimal spec.  A primary difference is that
binary FP has a much larger dynamic range, so you don't get near the
limits nearly as often; and IBM's decimal has a gigantic dynamic range
(the expectation is that essentially no real app will get anywhere
near its limits, unless the app is buggy).
From foom at fuhm.net  Mon Aug  9 18:38:35 2004
From: foom at fuhm.net (James Y Knight)
Date: Mon Aug  9 18:38:37 2004
Subject: [Python-Dev] Decorators: vertical bar syntax 
In-Reply-To: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>
References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <8EBFDCAD-EA22-11D8-8D12-000A95A50FB2@fuhm.net>


On Aug 8, 2004, at 9:31 PM, Bill Janssen wrote:

> Guido, could you expand on this principle a bit?  Just stated as it
> is, it sounds as pointless (to my untutored mind, at least :-), as the
> arguments about "@ is ugly, so it shouldn't be used".  After all,
> isn't "what the function does" an important external property of the
> function, and don't you have to peek inside the block to determine
> that?  For instance, how do you know whether a function is a generator
> unless you look for yield statements inside it?

A function being a generator only* affects the return value, so not 
knowing what functions are generator functions are really just a 
special case of not knowing the return value of any function. That can 
be solved with documentation, or a decorator.
e.g.
@returns(types.GeneratorType)

James

*: well not really, but to the external caller it's equivalent.

From janssen at parc.com  Mon Aug  9 18:42:24 2004
From: janssen at parc.com (Bill Janssen)
Date: Mon Aug  9 18:42:55 2004
Subject: [Python-Dev] Decorators: vertical bar syntax 
In-Reply-To: Your message of "Mon, 09 Aug 2004 08:04:47 PDT."
	<200408091504.i79F4lD22945@guido.python.org> 
Message-ID: <04Aug9.094230pdt."58612"@synergy1.parc.xerox.com>

GvR writes:
> Nevertheless I don't like to put decorators in the method body.  I
> think it has something to do with the fact that normally one thinks of
> the body contents as relevalt to "call-time" while decorators happen
> at "definition-time".
> 
> And before you bring up docstrings as a counterexample, yes, they
> should ideally have been designed to precede the method, too.

Perhaps another way to think about this is that there are statements
about properties of the function, method, or class which should
precede the implementation of the whatever-it-is?  That makes sense to
me.

But I'm less sure about the relative placement of the name of the
whatever versus statements about the properties of the whatever.  For
example, should the argument list also occur before the name of the
function?  In a perfect world, would Python have the syntax:

  def (param1, param2) funcname:
     ...

or perhaps:

  @args(param1, param2)
  def funcname:
     ...

In thinking about this, I find the argument made a day or two ago
about "cognitive binding" very powerful.  That is, you should state
the name of the thing you are defining before you start stating
properties of it, so that the reader has the opportunity to mentally
construct a tagged "bucket" to put those properties in.  This is
probably a useful thing for non-human tools, as well.

Perhaps that's one reason that arguments to a function almost
universally occur after the mention of the function name, in most
programming languages:

  def foo (arg1, arg2, ...)

So, by this principle, docstrings occur in the "right place", after
the function name but before the implementation.  Perhaps this is why
docstrings are so successful in Python.  Extending the principle to
decorators would mean that they should occur somewhere between the
function name and the function implementation.  All of the C, D, and E
variants in the Wiki seem to support this.  Perhaps one of them is the
right choice?

Bill
From bob at redivi.com  Mon Aug  9 19:02:27 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug  9 19:02:37 2004
Subject: [Python-Dev] Decorators: vertical bar syntax 
In-Reply-To: <04Aug9.094230pdt."58612"@synergy1.parc.xerox.com>
References: <04Aug9.094230pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <E4562872-EA25-11D8-9661-000A95686CD8@redivi.com>


On Aug 9, 2004, at 12:42 PM, Bill Janssen wrote:

> GvR writes:
>> Nevertheless I don't like to put decorators in the method body.  I
>> think it has something to do with the fact that normally one thinks of
>> the body contents as relevalt to "call-time" while decorators happen
>> at "definition-time".
>>
>> And before you bring up docstrings as a counterexample, yes, they
>> should ideally have been designed to precede the method, too.
>
> Perhaps another way to think about this is that there are statements
> about properties of the function, method, or class which should
> precede the implementation of the whatever-it-is?  That makes sense to
> me.
>
> But I'm less sure about the relative placement of the name of the
> whatever versus statements about the properties of the whatever.  For
> example, should the argument list also occur before the name of the
> function?  In a perfect world, would Python have the syntax:
>
>   def (param1, param2) funcname:
>      ...
>
> or perhaps:
>
>   @args(param1, param2)
>   def funcname:
>      ...
>
> In thinking about this, I find the argument made a day or two ago
> about "cognitive binding" very powerful.  That is, you should state
> the name of the thing you are defining before you start stating
> properties of it, so that the reader has the opportunity to mentally
> construct a tagged "bucket" to put those properties in.  This is
> probably a useful thing for non-human tools, as well.
>
> Perhaps that's one reason that arguments to a function almost
> universally occur after the mention of the function name, in most
> programming languages:
>
>   def foo (arg1, arg2, ...)
>
> So, by this principle, docstrings occur in the "right place", after
> the function name but before the implementation.  Perhaps this is why
> docstrings are so successful in Python.  Extending the principle to
> decorators would mean that they should occur somewhere between the
> function name and the function implementation.  All of the C, D, and E
> variants in the Wiki seem to support this.  Perhaps one of them is the
> right choice?

Actually I would say that in a perfect world, functions would be 
expressions.  Something like this:

foo = staticmethod(def(*args, **kwargs)):
	....

-bob
From mark at prothon.org  Mon Aug  9 19:29:59 2004
From: mark at prothon.org (Mark Hahn)
Date: Mon Aug  9 19:30:06 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
Message-ID: <hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>

Forgive me if I'm being a pest, but no one has commented on the real reason
I asked the question.  What does everyone think of the idea of having these
three built-in numeric types?

1) An Int implemented with infinite precision integer (Python Longs) with
the constant/__str__ form of +-NN such as 0, -123, 173_394, etc.

2) A Decimal implemented with the .Net decimal float (or the IBM decimal if
the .Net decimal sucks too much) with the constant/__str__ form of +-NN.NN
such as 0.0, -123.0, 173_394.0, 173394.912786765, etc.

3) A binary Float implemented with the hardware floating point with the
constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3,
2.35e-78, etc.

There would be no automatic conversion except the / operator would convert
from Int to Decimal and the Math module would convert Int and Decimal
values to Float for almost all functions (except simple ones like abs, min,
max).

From fperez528 at yahoo.com  Mon Aug  9 19:33:59 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Mon Aug  9 19:34:06 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<200408071617.i77GHgs10281@guido.python.org>
	<cf3vg5$1m5$1@sea.gmane.org> <vfft1tbv.fsf@yahoo.co.uk>
Message-ID: <cf8ci8$itg$1@sea.gmane.org>

Paul Moore wrote:

> Fernando Perez <fperez528@yahoo.com> writes:
[...]
> But we have a couple of inconsistencies to backtrack and resolve. (And
> the accepts/returns stuff is also inconsistent, but I'll stop picking
> holes in what is clearly just an example...)
> 
> I'm not saying that the same issues mightn't exist with the
> decorator-before-def form, but I don't think the mental model for the
> decorator-after-def form is quite as clean as you'd like either...

Well, I'm not so naive as to think that there is a silver bullet here or a
perfect syntax.  Obviously in all cases the programmer will need to keep some
context in his head.  But this is true always:

x = 1

... many lines of code which can't change x in any way

x.append('foo')  # BUG!

You spot something like this because you have a notion of local context, and
you remember that x, being an integer which hasn't changed, can't be appended
to as a list.

In fact, I'd argue that one of (not the only one) the key qualities of a good
programmer is the ability to maintain a fair amount of local and global
context in his head.  This is what allows you to realize the implications of a
code change, see the potential for fruitful refactorings, etc.

But that doesn't mean that one syntax choice can't make that process _easier_. 
It doesn't remove the need for having one's brain turned on, it is not a
silver bullet, but it enhances the flow of processing this kind of information
into an organized structure.

> It's still subjective :-)

I disagree, and I think I've given detailed arguments trying _precisely_ to
cast this discussion in terms which are potentially more productive than 'its
all subjective', which lead to endless running in circles.  I very much
appreciate a discussion of the merits of my ideas, but hopefully in terms
which allow something which goes beyond "your shirt is blue, and I don't like
blue".  I've tried to reply here in such a vein, hoping to produce useful
arguments for Guido and those ultimately making the decisions.

Yes, there is always a degree of subjectivity in all choices, and in that sense
I immensely trust Guido's intuition, ability and sense of aesthetics.  But
one's subjectivity works best when assisted by other arguments as well.

Best,

f

From aahz at pythoncraft.com  Mon Aug  9 19:34:15 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Aug  9 19:34:18 2004
Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2,
	and @decorators)
In-Reply-To: <200408091525.i79FPLM23023@guido.python.org>
References: <410DDFE3.7010006@interlink.com.au>
	<200408021850.i72IoCg28650@guido.python.org>
	<410EF963.40704@interlink.com.au> <20040809043104.GA82909@trit.org>
	<200408091525.i79FPLM23023@guido.python.org>
Message-ID: <20040809173415.GB28961@panix.com>

On Mon, Aug 09, 2004, Guido van Rossum wrote:
>Dima Dorfman:
>>
>> Are the exact semantics of what's allowed documented somewhere?
> 
> Not unless it's in the PEP -- it usually takes a stab at implementing
> something to find what's left unspecified!

Yup!

>> PEP 328 mostly talks about relative and absolute
>> imports, and it doesn't specify the exact semantics of where
>> parentheses should be allowed. My patch (attached) accepts
>> 
>>   import (os, sys)
>>   from sys import (stdin, stdout, stderr)
>>   import (os)
>>   from sys import (*)
>> 
>> but rejects
>> 
>>   from (sys) import stdin
>>   import (os), (sys)
>>   import (os,)
>> 
>> Should any of those be allowed? Anything that I missed?
> 
> This suggests (given the recent criticism of how we've been following
> the PEP process) that the PEP needs to be updated first.

Enh; as you say, implementation is often needed to clarify, and I don't
think anyone would dispute that this is a small portion of the PEP.

> My own gut feeling is that it's not important to allow parentheses on
> "bare" (from-less) imports, since you can easily repeat the 'import'
> keyword on a new line; but that it is important on the from...import
> form, and then only after the "import" keyword.  So that would accept
> your 2nd and 4th example but reject the 1st and 3rd.
> 
> I guess one can argue about (*); I don't care about it and suggest
> that it be dropped.

I'd suggest that only ``from foo import (...)`` be permitted (where "..."
does not include ``*``).  But I don't care much, and I'll update the PEP
as soon as it's clear which precise versions are allowed.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From fperez528 at yahoo.com  Mon Aug  9 19:38:49 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Mon Aug  9 19:40:24 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz>
	<019301c47db9$c269ec80$a9498890@neil>
	<200408091516.i79FGc722993@guido.python.org>
Message-ID: <cf8cra$jjn$1@sea.gmane.org>

Guido van Rossum wrote:

> Let's make one thing clear; while decorators may be useful for some to
> experiment with attaching signatures to methods, mainstream Python
> will eventually have optional static typing built in, and it won't use

I am immensely happy to hear this.  I have read your old essay on the topic,
but the long silence on this issue had me wondering whether it had been
completely abandoned.  I can't currently contribute to it, so I don't
complain.  But this is great news.  Would you care to venture even a rough
guess for when this might happen (and no, I'm not a lawyer, so you don't need
a disclaimer in front :)?

>From a scientific computing viewpoint, this is probably the single biggest
stumbling point when using python today.  We have scipy.weave, f2py and even
manually written extension modules, which we all know how to use.  But having
optional static typing for the cases where python's dynamic nature is a cost
with no benefit will be an incredible gain.  Other methods always slow down
the otherwise very fluid feel of using python, so having this as part of the
language will be fantastic.

Best,

f

From pf_moore at yahoo.co.uk  Mon Aug  9 19:42:37 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Mon Aug  9 19:42:23 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<e8bf7a5304080908135a142da@mail.gmail.com>
Message-ID: <k6w8mcmq.fsf@yahoo.co.uk>

Jeremy Hylton <jhylton@gmail.com> writes:

> On Sat, 07 Aug 2004 00:35:17 -0400, Edward Loper
> <edloper@gradient.cis.upenn.edu> wrote:
>> The new "vertical bar" syntax was proposed recently, and has gotten
>> almost unanimously positive feedback.  In particular, of 17 responses
>> to the syntax, only one has been negative (see table, below).
>
> I dislike it.  It's not as friendly or visible as the @.

I prefer @ but am willing to accept | as a compromise. I'd be
interested to know if that sentiment (which I think I've expressed
before) was one of those that Edward counted as "non-negative"...

Paul.
-- 
Home computers are being called upon to perform many new functions,
including the consumption of homework formerly eaten by the dog --
Doug Larson

From fperez528 at yahoo.com  Mon Aug  9 19:50:19 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Mon Aug  9 19:50:27 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>
	<200408091504.i79F4lD22945@guido.python.org>
Message-ID: <cf8dgs$l8a$1@sea.gmane.org>

Guido van Rossum wrote:

> [Guido]
>> > In the discussion on decorators months ago, solutions involving
>> > special syntax inside the block were ruled out early on.  Basically,
>> > you shouldn't have to peek inside the block to find out important
>> > external properties of the function.
> 
> [Bill Janssen]
>> Guido, could you expand on this principle a bit?  Just stated as it

> It's probably untenable as a formal principle. :-)
> 
> Nevertheless I don't like to put decorators in the method body.  I
> think it has something to do with the fact that normally one thinks of
> the body contents as relevalt to "call-time" while decorators happen
> at "definition-time".

I realize what your current position on this topic is, but I'd like to
encourage you to at least reconsider it for a moment.  In python we've come to
consider, for better or worse, the section up to the end of the docstring, as
a kind of 'header' or 'interface' zone for functions.  I know you dislike the
docstrings as an example of this, but in fact I don't view it as such a bad
thing: the def indents and clearly separates the function, and for a while you
have 'interface' information: function name, argument list, docstring, and
perhaps now decorators.  This actually feels very clean to me, and seems to
nicely fit the mental model of how python works.

Elsewhere in this thread:
http://mail.python.org/pipermail/python-dev/2004-August/047326.html
and below, I tried to make a reasoned argument in defense of this.  I won't
repeat all that here.

I further worry what will happen to tools like inspect.getsource() with
decorators outside the def:

@foo
@bar
def baz(x,y):
   ....

if I try to get the source for baz, will I see foo/bar?  For introspection
tools, this can lead to pretty misleading results.  But perhaps this is easy
to solve or has even already been taken care of.  I imagine if @foo/@bar were
inside the def, this wouldn't be an issue.

Best,

f

From barry at python.org  Mon Aug  9 20:00:54 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug  9 20:00:43 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <k6w8mcmq.fsf@yahoo.co.uk>
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<e8bf7a5304080908135a142da@mail.gmail.com> <k6w8mcmq.fsf@yahoo.co.uk>
Message-ID: <1092074454.8687.26.camel@localhost>

On Mon, 2004-08-09 at 13:42, Paul Moore wrote:

> I prefer @ but am willing to accept | as a compromise.

I agree with Paul here.  The only reason I suggested | was to make life
easier for 3rd party tools.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040809/b8d87a29/attachment.pgp
From tim.peters at gmail.com  Mon Aug  9 20:20:15 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  9 20:20:18 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
In-Reply-To: <hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
Message-ID: <1f7befae04080911203b46485c@mail.gmail.com>

[Mark Hahn]
> Forgive me if I'm being a pest,

Nope, not at all.

> but no one has commented on the real reason I asked the question.  What
> does everyone think of the idea of having these three built-in numeric types?

It depends on your goals for Prothon, and also on the relative
performance of the various numeric implementations.  Semantically, it
would be nicest if Python used the IBM decimal type for everything by
default (whether spelled 123, 123.0, or 123e0).  That's not gonna
happen, both because of backward compatibility, and because our
implementation of that type is "too slow".

I don't know how fast or slow the .NET Decimal type is on its own, let
alone compared to whatever implementations of bigints and binary
floats you're using.  Speed may or may not matter to your target
audience (although, in reality, you'll have multiple audiences, each
with their own irrational demands <wink>).

> 1) An Int implemented with infinite precision integer (Python Longs) with
> the constant/__str__ form of +-NN such as 0, -123, 173_394, etc.
>
> 2) A Decimal implemented with the .Net decimal float (or the IBM decimal if
> the .Net decimal sucks too much) with the constant/__str__ form of +-NN.NN
> such as 0.0, -123.0, 173_394.0, 173394.912786765, etc.
>
> 3) A binary Float implemented with the hardware floating point with the
> constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3,
> 2.35e-78, etc.

Sounds decent to me, although the IBM flavor of decimal is a full
floating-point type, and some apps will definitely want to use it for
6.02e23 thingies instead of the native binary Float.  Maybe use "d" as
a "decimal float" exponent marker.

> There would be no automatic conversion except the / operator would convert
> from Int to Decimal and the Math module would convert Int and Decimal
> values to Float for almost all functions (except simple ones like abs, min,
> max).

Int * Decimal will be very common in financial apps, and the latter
are a prime use case for Decimal ("OK, they bought 200 copies of
'Programming Prothon' at 39.98 each", etc).  Numeric auto-conversions
that don't lose information are usually helpful.
From FBatista at uniFON.com.ar  Mon Aug  9 20:24:51 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Aug  9 20:28:49 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
Message-ID: <A128D751272CD411BC9200508BC2194D053C7788@escpl.tcp.com.ar>

[Mark Hahn]

#- 2) A Decimal implemented with the .Net decimal float (or the 
#- IBM decimal if
#- the .Net decimal sucks too much) with the constant/__str__ 
#- form of +-NN.NN
#- such as 0.0, -123.0, 173_394.0, 173394.912786765, etc.

I'm ok with all the rest and this but with a particular point: .Net decimal
sucks too much if doesn't implement the IBM standard, ;)

Basically, it'll be real nice if Python and Prothon behaves equally in
decimal float.

.	Facundo
From pf_moore at yahoo.co.uk  Mon Aug  9 20:31:33 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Mon Aug  9 20:31:21 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
Message-ID: <d620mad6.fsf@yahoo.co.uk>

Mark Hahn <mark@prothon.org> writes:

> Forgive me if I'm being a pest, but no one has commented on the real reason
> I asked the question.  What does everyone think of the idea of having these
> three built-in numeric types?
>
> 1) An Int implemented with infinite precision integer (Python Longs) with
> the constant/__str__ form of +-NN such as 0, -123, 173_394, etc.
>
> 2) A Decimal implemented with the .Net decimal float (or the IBM decimal if
> the .Net decimal sucks too much) with the constant/__str__ form of +-NN.NN
> such as 0.0, -123.0, 173_394.0, 173394.912786765, etc.
>
> 3) A binary Float implemented with the hardware floating point with the
> constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3,
> 2.35e-78, etc.
>
> There would be no automatic conversion except the / operator would convert
> from Int to Decimal and the Math module would convert Int and Decimal
> values to Float for almost all functions (except simple ones like abs, min,
> max).

I think that having 2 different fractional types distinguished by
something as subtle as the presence of an exponent (something no other
language does to my knowledge) is an extremely bad idea.

Worse still is the fact that the exponent in the e notation is a
decimal exponent, but the type used is binary!

Paul.
-- 
It is better to be quotable than to be honest -- Tom Stoppard

From eppstein at ics.uci.edu  Mon Aug  9 20:45:06 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Mon Aug  9 20:45:18 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<e8bf7a5304080908135a142da@mail.gmail.com>
	<k6w8mcmq.fsf@yahoo.co.uk> <1092074454.8687.26.camel@localhost>
Message-ID: <eppstein-16B12F.11450609082004@sea.gmane.org>

In article <1092074454.8687.26.camel@localhost>,
 Barry Warsaw <barry@python.org> wrote:

> > I prefer @ but am willing to accept | as a compromise.
> 
> I agree with Paul here.  The only reason I suggested | was to make life
> easier for 3rd party tools.

But does it?

With @, a third party tool has an unambiguous indication of whether a 
line is a decorator.  With |, it is not possible to recognize decorators 
with a regular expression, instead you have to do some context-free 
parsing to determine whether some previous line has an unclosed paren 
(in which case the | is a binop rather than a decorator).

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From mark at prothon.org  Mon Aug  9 20:45:29 2004
From: mark at prothon.org (Mark Hahn)
Date: Mon Aug  9 20:45:34 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
	<1f7befae04080911203b46485c@mail.gmail.com>
Message-ID: <um9olp66c1at.qpsv216awprc.dlg@40tude.net>

On Mon, 9 Aug 2004 14:20:15 -0400, Tim Peters wrote:

> [Mark Hahn]
>> Forgive me if I'm being a pest,
> 
> Nope, not at all.
> 
>> but no one has commented on the real reason I asked the question.  What
>> does everyone think of the idea of having these three built-in numeric types?
> 
> It depends on your goals for Prothon, and also on the relative
> performance of the various numeric implementations.  Semantically, it
> would be nicest if Python used the IBM decimal type for everything by
> default (whether spelled 123, 123.0, or 123e0).  

Are you sure you mean that?  I have been assuming that there is a
fundamental need for an integer type that has no fraction.  There are many
places like indexing where you semantically want integers.  I think if
speed were no issue I'd still have integers and decimal floats.

> That's not gonna
> happen, both because of backward compatibility, and because our
> implementation of that type is "too slow".
> 
> I don't know how fast or slow the .NET Decimal type is on its own, let
> alone compared to whatever implementations of bigints and binary
> floats you're using.  Speed may or may not matter to your target
> audience (although, in reality, you'll have multiple audiences, each
> with their own irrational demands <wink>).

I'm already planning on using Jim's IronPython implementation of your
BigInts so I might as well port your Decimal code over for the Prothon
Decimal also.  The only thing I've ever stolen from Python for Prothon was
your bigint code so I might as well be consistent in my thievery. :-)

>> 1) An Int implemented with infinite precision integer (Python Longs) with
>> the constant/__str__ form of +-NN such as 0, -123, 173_394, etc.
>>
>> 2) A Decimal implemented with the .Net decimal float (or the IBM decimal if
>> the .Net decimal sucks too much) with the constant/__str__ form of +-NN.NN
>> such as 0.0, -123.0, 173_394.0, 173394.912786765, etc.
>>
>> 3) A binary Float implemented with the hardware floating point with the
>> constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3,
>> 2.35e-78, etc.
> 
> Sounds decent to me, although the IBM flavor of decimal is a full
> floating-point type, and some apps will definitely want to use it for
> 6.02e23 thingies instead of the native binary Float.  Maybe use "d" as
> a "decimal float" exponent marker.

I would hate to mess up my "pure" scheme but it makes sense.  Will do.

>> There would be no automatic conversion except the / operator would convert
>> from Int to Decimal and the Math module would convert Int and Decimal
>> values to Float for almost all functions (except simple ones like abs, min,
>> max).
> 
> Int * Decimal will be very common in financial apps, and the latter
> are a prime use case for Decimal ("OK, they bought 200 copies of
> 'Programming Prothon' at 39.98 each", etc).  Numeric auto-conversions
> that don't lose information are usually helpful.

When I wrote that I was thinking of no automatic conversion from Decimal to
Float.  Conversion from Int to Decimal would make sense as you suggest.

Thanks very much.  I think I have a handle on the issues now.  To
summarize, .Net sucks (no big surprise). :-)

From chris.cavalaria at free.fr  Mon Aug  9 20:43:05 2004
From: chris.cavalaria at free.fr (Christophe Cavalaria)
Date: Mon Aug  9 20:50:15 2004
Subject: [Python-Dev] Re: Re: def fn (args) [dec,dec]:
References: <200408061445.i76EjqT07026@guido.python.org>
	<200408090105.i7915mfS005734@cosc353.cosc.canterbury.ac.nz>
Message-ID: <cf8gjq$teo$1@sea.gmane.org>

Greg Ewing wrote:

> Guido:
> 
>> Just look at it when either the list of decorators or the argument
>> list doesn't fit on one line:
> 
> You don't *have* to format it quite that badly.
> 
>     def longMethodNameForEffect(longArgumentOne=None,
>                                 longArgumentTwo=42) \
>         [staticmethod,
>          funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
>                    status="experimental",
>                    author="BDFL")]:

Am I the only one who finds that it's :
- ugly
- incredibly error prone due to a lot of special cases on each line of code
- a back door for the "how to indent C code" flamewars in Python

I guess it depends on the amount of decorators you will place on each
function. If you place 3-4 decorators per function ( it could easily happen
once we get a good and easy to use syntax ), we'll favor a syntax that
makes multiline decorator declaration clear and easy to type. The [...]
doesn't apply.

On a readability point of view, the @ syntax scales much better with a big
number of decorators than the [...] syntax.

Also, the decorators don't have that much effet on what happens between the
def and the return. For that reason I find it a mistake to place any
decorator information after the def statement.


From mark at prothon.org  Mon Aug  9 20:45:51 2004
From: mark at prothon.org (Mark Hahn)
Date: Mon Aug  9 20:50:54 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
References: <A128D751272CD411BC9200508BC2194D053C7788@escpl.tcp.com.ar>
Message-ID: <f6kfqweisqx5$.yffjvr6zqyp1.dlg@40tude.net>

On Mon, 9 Aug 2004 15:24:51 -0300, Batista, Facundo wrote:

> [Mark Hahn]
> 
> #- 2) A Decimal implemented with the .Net decimal float (or the 
> #- IBM decimal if
> #- the .Net decimal sucks too much) with the constant/__str__ 
> #- form of +-NN.NN
> #- such as 0.0, -123.0, 173_394.0, 173394.912786765, etc.
> 
> I'm ok with all the rest and this but with a particular point: .Net decimal
> sucks too much if doesn't implement the IBM standard, ;)
> 
> Basically, it'll be real nice if Python and Prothon behaves equally in
> decimal float.

OK, will do.

From jack at performancedrivers.com  Mon Aug  9 21:05:31 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Mon Aug  9 21:05:36 2004
Subject: [Python-Dev] A usability argument for list-after-def
In-Reply-To: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
References: <5.1.1.6.0.20040805160715.01f0c060@mail.telecommunity.com>
Message-ID: <20040809190531.GA23725@performancedrivers.com>

On Thu, Aug 05, 2004 at 04:48:09PM -0400, Phillip J. Eby wrote:
> Brackets are normally used in English to indicate an annotation or "aside" 
> of some sort; such as a commentary *about* the surrounding material.

+1, glad this one is back from the dead.

-Jack
From mark at prothon.org  Mon Aug  9 21:09:29 2004
From: mark at prothon.org (Mark Hahn)
Date: Mon Aug  9 21:09:34 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
	<d620mad6.fsf@yahoo.co.uk>
Message-ID: <6rdhjlf9tfol$.d6mi30mbjt80.dlg@40tude.net>

On Mon, 09 Aug 2004 19:31:33 +0100, Paul Moore wrote:

> I think that having 2 different fractional types distinguished by
> something as subtle as the presence of an exponent (something no other
> language does to my knowledge) is an extremely bad idea.
> 
> Worse still is the fact that the exponent in the e notation is a
> decimal exponent, but the type used is binary!

Good point.  Since Tim talked me into allowing the decimal type to use the
floating point form anyway, I might as well drop that idea for
differentiation.  

I've learned what I needed here.  I've learned that the .Net decimal type
sucks and that having an "in-between" decimal type isn't necessarily a bad
idea.

We can hammer out the syntax back on the Prothon mailing list.  Of course
I'd like to encourage any and all to help me out back there.  The mailing
list has light traffic since we have no pesky users. :-)  Discussion is at: 
http://lists.prothon.org/mailman/listinfo/prothon-user

Thanks everyone...  Have a good day.

From s.percivall at chello.se  Mon Aug  9 21:20:35 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Mon Aug  9 21:20:38 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <cf8dgs$l8a$1@sea.gmane.org>
References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>
	<200408091504.i79F4lD22945@guido.python.org>
	<cf8dgs$l8a$1@sea.gmane.org>
Message-ID: <30C8BD17-EA39-11D8-8B30-0003934AD54A@chello.se>

On 2004-08-09, at 19.50, Fernando Perez wrote:
> I further worry what will happen to tools like inspect.getsource() with
> decorators outside the def:
>
> @foo
> @bar
> def baz(x,y):
>    ....
>
> if I try to get the source for baz, will I see foo/bar?  For 
> introspection
> tools, this can lead to pretty misleading results.  But perhaps this 
> is easy
> to solve or has even already been taken care of.  I imagine if 
> @foo/@bar were
> inside the def, this wouldn't be an issue.

Right now, you'll see the source of the function/method defined above
the decorators or, if none, the top line/lines of the source file. Yes,
it's pretty broken.

From bsder at mail.allcaps.org  Mon Aug  9 21:34:31 2004
From: bsder at mail.allcaps.org (Andrew P. Lentvorski, Jr.)
Date: Mon Aug  9 21:34:37 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
In-Reply-To: <d620mad6.fsf@yahoo.co.uk>
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
	<d620mad6.fsf@yahoo.co.uk>
Message-ID: <22FD7E0D-EA3B-11D8-B417-000A95C874EE@mail.allcaps.org>


On Aug 9, 2004, at 11:31 AM, Paul Moore wrote:

> I think that having 2 different fractional types distinguished by
> something as subtle as the presence of an exponent (something no other
> language does to my knowledge) is an extremely bad idea.

Doesn't Common Lisp use 1.0e+10 to signify singles and 1.0d+10 to
signify doubles?

That doesn't make it a good idea, but there is precedent.

-a

From python at rcn.com  Sun Aug  8 21:42:38 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug  9 21:42:44 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
In-Reply-To: <um9olp66c1at.qpsv216awprc.dlg@40tude.net>
Message-ID: <004801c47d7f$dcedb120$191fc797@oemcomputer>

>  I have been assuming that there is a
> fundamental need for an integer type that has no fraction.

The IBM spec covers almost all needs except speed and space (and even
those aren't horrible in a good C implementation).  It is a superset of
754/854, of fixed point, and of high precision integer arithmetic.  If
you were bold, you would have this as the sole numeric type.  This would
spare you decimal representation issues, inadequate range/precision
problems, type conversion issues, and compatibility challenges.  Plus,
it is darned nice to have something as thoroughly thought out and
tested.

Of course, if you want to roll your own from scratch, I hear that it is
trivially easy and that everything always works out in the end, NOT!


Raymond

From bsder at mail.allcaps.org  Mon Aug  9 21:46:41 2004
From: bsder at mail.allcaps.org (Andrew P. Lentvorski, Jr.)
Date: Mon Aug  9 21:46:45 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
In-Reply-To: <um9olp66c1at.qpsv216awprc.dlg@40tude.net>
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
	<1f7befae04080911203b46485c@mail.gmail.com>
	<um9olp66c1at.qpsv216awprc.dlg@40tude.net>
Message-ID: <D5C50964-EA3C-11D8-B417-000A95C874EE@mail.allcaps.org>


On Aug 9, 2004, at 11:45 AM, Mark Hahn wrote:

> On Mon, 9 Aug 2004 14:20:15 -0400, Tim Peters wrote:
>>
>> It depends on your goals for Prothon, and also on the relative
>> performance of the various numeric implementations.  Semantically, it
>> would be nicest if Python used the IBM decimal type for everything by
>> default (whether spelled 123, 123.0, or 123e0).
>
> Are you sure you mean that?  I have been assuming that there is a
> fundamental need for an integer type that has no fraction.

Sure, but both binary floating point and Decimal happily inhabit
that space.  The issue in Days Gone By(tm) was the fact that
a single precision float tended to be the default type and
only had about 24 bits of precision (somewhere around 8-9
decimal digits).  Consequently, people would often get surprised
when accumulating small integers ceased to work around 10^8 or 10^9.

With binary FP doubles, this is less of an issue as it has about 54 bits
of precision (about 18 decimal digits or so).  However, it still places
an artificial restriction on maximum precision.

Besides, the only reason programmers write loops as integers is
because accumulating floats "doesn't work".  With Decimal, that
is no longer the case.

-a

From bsder at mail.allcaps.org  Mon Aug  9 21:49:12 2004
From: bsder at mail.allcaps.org (Andrew P. Lentvorski, Jr.)
Date: Mon Aug  9 21:49:16 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
In-Reply-To: <hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
Message-ID: <3030F1DE-EA3D-11D8-B417-000A95C874EE@mail.allcaps.org>


On Aug 9, 2004, at 10:29 AM, Mark Hahn wrote:

> 3) A binary Float implemented with the hardware floating point with the
> constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3,
> 2.35e-78, etc.

Be careful defining string representations of numbers.  It is a 
non-trivial
problem to adhere to the correct internationalization standards.

> There would be no automatic conversion except the / operator would 
> convert
> from Int to Decimal

Int goes to the default type of float (which is currently binary FP) 
already
upon using the / operator.

>  and the Math module would convert Int and Decimal
> values to Float for almost all functions (except simple ones like abs, 
> min,
> max).

This is not necessarily a good idea.  Exponentials are very useful in
financial situations and want to be decimals, not binary floats, in 
those
cases.

-a

From barry at python.org  Mon Aug  9 21:53:10 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug  9 21:53:00 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <eppstein-16B12F.11450609082004@sea.gmane.org>
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<e8bf7a5304080908135a142da@mail.gmail.com> <k6w8mcmq.fsf@yahoo.co.uk>
	<1092074454.8687.26.camel@localhost>
	<eppstein-16B12F.11450609082004@sea.gmane.org>
Message-ID: <1092081190.8687.40.camel@localhost>

On Mon, 2004-08-09 at 14:45, David Eppstein wrote:
> > I agree with Paul here.  The only reason I suggested | was to make life
> > easier for 3rd party tools.
> 
> But does it?
> 
> With @, a third party tool has an unambiguous indication of whether a 
> line is a decorator.  With |, it is not possible to recognize decorators 
> with a regular expression, instead you have to do some context-free 
> parsing to determine whether some previous line has an unclosed paren 
> (in which case the | is a binop rather than a decorator).

Oh ick.  I hadn't thought of that, but if that's going to cause a
problem given the current grammar (and I can't test that atm), then I'm
going to have to retract my support for |.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040809/265fc754/attachment.pgp
From python at rcn.com  Sun Aug  8 21:57:17 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug  9 21:57:22 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <1092081190.8687.40.camel@localhost>
Message-ID: <004f01c47d81$e861e740$191fc797@oemcomputer>

> > > I agree with Paul here.  The only reason I suggested | was to make
> life
> > > easier for 3rd party tools.
> >
> > But does it?
> >
> > With @, a third party tool has an unambiguous indication of whether
a
> > line is a decorator.  With |, it is not possible to recognize
decorators
> > with a regular expression, instead you have to do some context-free
> > parsing to determine whether some previous line has an unclosed
paren
> > (in which case the | is a binop rather than a decorator).
> 
> Oh ick.  I hadn't thought of that, but if that's going to cause a
> problem given the current grammar (and I can't test that atm), then
I'm
> going to have to retract my support for |.

An exclamation point would be better in this regard.


Raymond

From fredrik at pythonware.com  Mon Aug  9 22:00:19 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Aug  9 21:58:42 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu><e8bf7a5304080908135a142da@mail.gmail.com><k6w8mcmq.fsf@yahoo.co.uk>
	<1092074454.8687.26.camel@localhost>
	<eppstein-16B12F.11450609082004@sea.gmane.org>
Message-ID: <cf8l1e$9rk$1@sea.gmane.org>

David Eppstein wrote:

> With @, a third party tool has an unambiguous indication of whether a
> line is a decorator.  With |, it is not possible to recognize decorators
> with a regular expression, instead you have to do some context-free
> parsing to determine whether some previous line has an unclosed paren
> (in which case the | is a binop rather than a decorator).

"\
@decorator #"

"""
@decorator
"""

(etc)

</F>



From chris.cavalaria at free.fr  Mon Aug  9 22:04:41 2004
From: chris.cavalaria at free.fr (Christophe Cavalaria)
Date: Mon Aug  9 22:04:45 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu><e8bf7a5304080908135a142da@mail.gmail.com><k6w8mcmq.fsf@yahoo.co.uk>
	<1092074454.8687.26.camel@localhost>
	<eppstein-16B12F.11450609082004@sea.gmane.org>
	<cf8l1e$9rk$1@sea.gmane.org>
Message-ID: <cf8lcp$ak1$1@sea.gmane.org>

Fredrik Lundh wrote:

> David Eppstein wrote:
> 
>> With @, a third party tool has an unambiguous indication of whether a
>> line is a decorator.  With |, it is not possible to recognize decorators
>> with a regular expression, instead you have to do some context-free
>> parsing to determine whether some previous line has an unclosed paren
>> (in which case the | is a binop rather than a decorator).
> 
> "\
> @decorator #"
> 
> """
> @decorator
> """
> 
> (etc)

Let me continue the (etc) for you :
"""
def foo():
    return 0
"""


From s.percivall at chello.se  Mon Aug  9 22:10:40 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Mon Aug  9 22:10:43 2004
Subject: [Python-Dev] tokenize.py (generate_tokens yielding @ as OP)
Message-ID: <2FE44090-EA40-11D8-8B30-0003934AD54A@chello.se>

This is a question regarding the tokenize.generate_tokens function.

As it is in CVS HEAD, generate_tokens yields the '@' token as
having type OP, not type AT. Is this how it should be?

//Simon

From guido at python.org  Mon Aug  9 22:14:42 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 22:14:49 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: Your message of "Mon, 09 Aug 2004 11:50:19 MDT."
	<cf8dgs$l8a$1@sea.gmane.org> 
References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>
	<200408091504.i79F4lD22945@guido.python.org> 
	<cf8dgs$l8a$1@sea.gmane.org> 
Message-ID: <200408092014.i79KEh123856@guido.python.org>

> I realize what your current position on this topic is, but I'd like
> to encourage you to at least reconsider it for a moment.

Please remember the procedure.  The community is to come up with *one*
alternative that they universally prefer over @decorators; otherwise,
@decorators-before-def will stay.  (I'll even give you *two* if the
community can agree that both are acceptable and really wishes to let
me pick one.)

Plenty of people agree with me on this particular issue (decorators
inside or outside the def); you have to convince them first before
pleading with me.

I don't want to have to argue every single detail.  As it stands now,
it looks like the community is having a hard time agreeing on
anything, which suggests that I might as well stick with
@decorators-before-def.

I try to limit my posts to (a) procedural adjustments; (b)
clarifications of my previously stated position; (c) clarifications of
other factual misunderstandings; (d) pointing out issues that nobody
else has brought up yet.  I try not to be pulled into the debate; I
simply can't afford the time.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Mon Aug  9 22:17:46 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 22:17:52 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: Your message of "Mon, 09 Aug 2004 11:38:49 MDT."
	<cf8cra$jjn$1@sea.gmane.org> 
References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz>
	<019301c47db9$c269ec80$a9498890@neil>
	<200408091516.i79FGc722993@guido.python.org> 
	<cf8cra$jjn$1@sea.gmane.org> 
Message-ID: <200408092017.i79KHkW23877@guido.python.org>

> Guido van Rossum wrote:
> > Let's make one thing clear; while decorators may be useful for
> > some to experiment with attaching signatures to methods,
> > mainstream Python will eventually have optional static typing
> > built in, and it won't use
> 
> I am immensely happy to hear this.  I have read your old essay on
> the topic, but the long silence on this issue had me wondering
> whether it had been completely abandoned.  I can't currently
> contribute to it, so I don't complain.  But this is great news.
> Would you care to venture even a rough guess for when this might
> happen (and no, I'm not a lawyer, so you don't need a disclaimer in
> front :)?

Probably not before Python 3000, which is awaiting my retirement (or
at least a sabbatical year), unless the community revisits the issue
and comes up with a solid design that I can agree with.  My retirement
or sabbatical is at least a few years off... :-(

> From a scientific computing viewpoint, this is probably the single
> biggest stumbling point when using python today.  We have
> scipy.weave, f2py and even manually written extension modules, which
> we all know how to use.  But having optional static typing for the
> cases where python's dynamic nature is a cost with no benefit will
> be an incredible gain.  Other methods always slow down the otherwise
> very fluid feel of using python, so having this as part of the
> language will be fantastic.

I'm afraid that if you want something *now* in order to interface to
3rd party tools, an interim solution using decorators may be your best
bet for the next few years (assuming Python 2.4 has some form of
decorators).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Mon Aug  9 22:19:25 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 22:19:32 2004
Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2,
	and @decorators)
In-Reply-To: Your message of "Mon, 09 Aug 2004 13:34:15 EDT."
	<20040809173415.GB28961@panix.com> 
References: <410DDFE3.7010006@interlink.com.au>
	<200408021850.i72IoCg28650@guido.python.org>
	<410EF963.40704@interlink.com.au> <20040809043104.GA82909@trit.org>
	<200408091525.i79FPLM23023@guido.python.org> 
	<20040809173415.GB28961@panix.com> 
Message-ID: <200408092019.i79KJPw23893@guido.python.org>

> I'd suggest that only ``from foo import (...)`` be permitted (where
> "..."  does not include ``*``).  But I don't care much, and I'll
> update the PEP as soon as it's clear which precise versions are
> allowed.

This seems to be the consensus so far, so why don't you add all that
detail to the PEP.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Mon Aug  9 22:21:57 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug  9 22:22:04 2004
Subject: [Python-Dev] tokenize.py (generate_tokens yielding @ as OP)
In-Reply-To: Your message of "Mon, 09 Aug 2004 22:10:40 +0200."
	<2FE44090-EA40-11D8-8B30-0003934AD54A@chello.se> 
References: <2FE44090-EA40-11D8-8B30-0003934AD54A@chello.se> 
Message-ID: <200408092021.i79KLv823929@guido.python.org>

> This is a question regarding the tokenize.generate_tokens function.
> 
> As it is in CVS HEAD, generate_tokens yields the '@' token as
> having type OP, not type AT. Is this how it should be?

Without looking at the code, I believe tokenize treats all operators
as OP, for a rather liberal definition of "operator".

(A more serious problem that I just encountered in a different
context: when you feed it input containing unsupported characters, the
spaces before those unsupported characters are also returned as
tokens.  I think that's a bug.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From amk at amk.ca  Mon Aug  9 22:42:28 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Aug  9 22:42:53 2004
Subject: [Python-Dev] Re: Recommended way to tell platform
In-Reply-To: <4115CFFD.1000309@v.loewis.de>
References: <20040807142327.GA2529@rogue.amk.ca> <4114E753.2050303@v.loewis.de>
	<200408071637.i77Gb0w10415@guido.python.org>
	<4115CFFD.1000309@v.loewis.de>
Message-ID: <20040809204228.GA11199@rogue.amk.ca>

On Sun, Aug 08, 2004 at 09:02:21AM +0200, "Martin v. L?wis" wrote:
> Precisely that: impossible to say. In most cases, you should not test
> for the platform, but just use the platform functionality, and fall back
> to something else if it isn't present.

Returning to the asyncore case that started all this: should I follow
this principle and check if SO_EXCLUSIVEADDRUSE exists, and if it does
use that constant instead of SO_REUSEADDR?

--amk

From eppstein at ics.uci.edu  Mon Aug  9 22:49:20 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Mon Aug  9 22:49:28 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<1092074454.8687.26.camel@localhost>
	<eppstein-16B12F.11450609082004@sea.gmane.org>
	<cf8l1e$9rk$1@sea.gmane.org>
Message-ID: <eppstein-5CF885.13492009082004@sea.gmane.org>

In article <cf8l1e$9rk$1@sea.gmane.org>,
 "Fredrik Lundh" <fredrik@pythonware.com> wrote:

> David Eppstein wrote:
> 
> > With @, a third party tool has an unambiguous indication of whether a
> > line is a decorator.  With |, it is not possible to recognize decorators
> > with a regular expression, instead you have to do some context-free
> > parsing to determine whether some previous line has an unclosed paren
> > (in which case the | is a binop rather than a decorator).
> 
> "\
> @decorator #"
> 
> """
> @decorator
> """
> 
> (etc)
> 
> </F>

I think quoted string recognition in Python is still regular, not 
context-free.  Anyway, the syntax highlighters in editors that I've used 
don't seem to have a problem with it.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From tim.peters at gmail.com  Mon Aug  9 22:54:00 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  9 22:54:02 2004
Subject: [Python-Dev] Another test_compiler mystery
Message-ID: <1f7befae0408091354605dc58b@mail.gmail.com>

I've noticed several times now, in both debug and release builds, that
if I run regrtest.py with -uall, *sometimes* it just stops after
running test_compiler:

$ python_d  regrtest.py -uall
test_grammar
test_opcodes
...
test_compare
test_compile
test_compiler

$

There's no indication of error, it just ends.  It's not consistent.

Happened once when I was running with -v, and test_compiler's output ended here:

...
compiling C:\Code\python\lib\test\test_operator.py
compiling C:\Code\python\lib\test\test_optparse.py
compiling C:\Code\python\lib\test\test_os.py
compiling C:\Code\python\lib\test\test_ossaudiodev.py
compiling C:\Code\python\lib\test\test_parser.py

In particular, there's no

    Ran M tests in Ns

output, so it doesn't look like unittest (let alone regrtest) ever got
control back.

Hmm.  os.listdir() is in sorted order on NTFS, so test_compiler should
be chewing over a lot more files after test_parser.py.

*This* I could blame on a blown C stack -- although I'd expect a much
nastier symptom then than just premature termination.

Anyone else?
From amk at amk.ca  Mon Aug  9 23:15:17 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Aug  9 23:15:43 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <4115DD5A.2070108@interlink.com.au>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>
Message-ID: <20040809211517.GB11199@rogue.amk.ca>

On Sun, Aug 08, 2004 at 05:59:22PM +1000, Anthony Baxter wrote:
> The AddlistClass in email is marked as deprecated, but is deprecated
> in favour of a class in rfc822!

Maybe deprecating rfc822 simply isn't a worthwhile goal?

Anyway, here's a revised action list:

* Remove TERMIOS, mpz, statcache, xreadlines, rotor.
  (Unchanged from my earlier proposal.)

  These modules have raised DeprecationWarning for a while, but only
  rotor is listed in PEP 4.  I'll add them to the PEP in any event; 
  is not being listed sufficient grounds for delaying their removal to
  2.5?

  (I would say we don't need to wait until 2.5; they've been raising
  DeprecationWarnings for a long time, so users should be well aware 
  the modules' days are numbered.)

* Leave rfc822, mimetools alone; the stdlib will have to avoid their use 
  first, and I'm not going to do that right now.

* Make mimify, MimeWriter raise DeprecationWarnings; nothing in the
  stdlib uses them.

--amk



From aahz at pythoncraft.com  Mon Aug  9 23:23:46 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Aug  9 23:23:48 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <20040809211517.GB11199@rogue.amk.ca>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>
	<20040809211517.GB11199@rogue.amk.ca>
Message-ID: <20040809212346.GA2699@panix.com>

On Mon, Aug 09, 2004, A.M. Kuchling wrote:
> 
> Maybe deprecating rfc822 simply isn't a worthwhile goal?
> 
> Anyway, here's a revised action list:
> 
> * Remove TERMIOS, mpz, statcache, xreadlines, rotor.
>   (Unchanged from my earlier proposal.)
> 
>   These modules have raised DeprecationWarning for a while, but only
>   rotor is listed in PEP 4.  I'll add them to the PEP in any event; 
>   is not being listed sufficient grounds for delaying their removal to
>   2.5?
> 
>   (I would say we don't need to wait until 2.5; they've been raising
>   DeprecationWarnings for a long time, so users should be well aware 
>   the modules' days are numbered.)
> 
> * Leave rfc822, mimetools alone; the stdlib will have to avoid their use 
>   first, and I'm not going to do that right now.
> 
> * Make mimify, MimeWriter raise DeprecationWarnings; nothing in the
>   stdlib uses them.

What about DeprecationWarning for whrandom?  (Not that I care
personally.)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From s.percivall at chello.se  Mon Aug  9 23:34:50 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Mon Aug  9 23:34:52 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <30C8BD17-EA39-11D8-8B30-0003934AD54A@chello.se>
References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>
	<200408091504.i79F4lD22945@guido.python.org>
	<cf8dgs$l8a$1@sea.gmane.org>
	<30C8BD17-EA39-11D8-8B30-0003934AD54A@chello.se>
Message-ID: <F1929FAF-EA4B-11D8-8B30-0003934AD54A@chello.se>

On 2004-08-09, at 21.20, Simon Percivall wrote:
> On 2004-08-09, at 19.50, Fernando Perez wrote:
>> I further worry what will happen to tools like inspect.getsource() 
>> with
>> decorators outside the def:
>>
>> @foo
>> @bar
>> def baz(x,y):
>>    ....
>>
>> if I try to get the source for baz, will I see foo/bar?  For 
>> introspection
>> tools, this can lead to pretty misleading results.  But perhaps this 
>> is easy
>> to solve or has even already been taken care of.  I imagine if 
>> @foo/@bar were
>> inside the def, this wouldn't be an issue.
>
> Right now, you'll see the source of the function/method defined above
> the decorators or, if none, the top line/lines of the source file. Yes,
> it's pretty broken.

I uploaded a quick patch for this broken behaviour to SF, patch 
1006219, if
anyone cares.

//Simon

From tim.peters at gmail.com  Mon Aug  9 23:35:14 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug  9 23:35:16 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <20040809212346.GA2699@panix.com>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>
	<20040809211517.GB11199@rogue.amk.ca> <20040809212346.GA2699@panix.com>
Message-ID: <1f7befae04080914352cf5085d@mail.gmail.com>

[Aahz]
> What about DeprecationWarning for whrandom?  (Not that I care
> personally.)

That's already been done.
From amk at amk.ca  Mon Aug  9 23:38:20 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Aug  9 23:38:46 2004
Subject: [Python-Dev] Third Bug Day outcome
Message-ID: <20040809213820.GC11199@rogue.amk.ca>

The bug day on Saturday went well, as usual.  19 bugs and 12 patches
were closed.  That's not as good as the first bug day (30 bugs), but
is competitive with the second (18 bugs, 21 patches).  Lots of the
easy bugs have been cleaned out, I expect, so each remaining bug takes
more time to fix.

The composition of the turnout was the surprising thing. My plaintive
bleating on python-dev resulted in much higher participation by people
with CVS committers, but there weren't many non-committer people
around (Seo Sanghyeon and Mike Coleman were the two non-developers I
noticed).  This is a pity, because bug days seem like a good way to
gently introduce more people to hacking on the source.

During the day, Armin Rigo wrote a nifty IRC bot that takes an SF
bug/patch ID and returns the title; taking an example from the transcript:

20:53:56 <jlgijsbers> Looking at #777659.
20:53:57 <sf_number> * sf_number bug 777659 - Uninitialized variable used in Tools/faqwiz/faqwiz.py

I won't be running a bug day in September; if someone else wants to
arrange it, please feel free.  Otherwise, they'll resume in October.

--amk

From python at rcn.com  Sun Aug  8 23:41:48 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug  9 23:41:54 2004
Subject: [Python-Dev] Third Bug Day outcome
In-Reply-To: <20040809213820.GC11199@rogue.amk.ca>
Message-ID: <006101c47d90$8228efa0$191fc797@oemcomputer>

> A.M. Kuchling
> I won't be running a bug day in September; if someone else wants to
> arrange it, please feel free.  Otherwise, they'll resume in October.

What is involved beyond the Wiki page and announcing a day?

learned-to-ask-questions-before-volunteering-ly yours,


Raymond

From fperez528 at yahoo.com  Mon Aug  9 23:50:00 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Mon Aug  9 23:50:09 2004
Subject: [Python-Dev] Re: Re: Decorators: vertical bar syntax
References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>
	<200408091504.i79F4lD22945@guido.python.org>
	<cf8dgs$l8a$1@sea.gmane.org>
	<200408092014.i79KEh123856@guido.python.org>
Message-ID: <cf8ri9$kcc$1@sea.gmane.org>

Guido van Rossum wrote:

>> I realize what your current position on this topic is, but I'd like
>> to encourage you to at least reconsider it for a moment.
> 
> Please remember the procedure.  The community is to come up with *one*
> alternative that they universally prefer over @decorators; otherwise,
> @decorators-before-def will stay.  (I'll even give you *two* if the
> community can agree that both are acceptable and really wishes to let
> me pick one.)

I was simply trying to outline, in a hopefully clear and useful form, an
argument specifically pointing in a direction you'd personally called 'closed'
from your side.

I fully realize your bandwith is limited, and probably already well saturated
by this debate.  

I also have the strong suspicion, given how difficult it is proving to reach
significant consensus on almost anything on this, that it will probably come
down to you making a final 'hard' decision.  That's ok, and we can all live
with it (and most of us trust your judgement because you've shown its
qualities before).  But if it comes to that, at least I figured it was worth
arguing better this particular point, which I felt had been discussed from
other angles but not this one.

I feel like I've said as much as is reasonable on this topic, so I'll probably
shut up and wait for something to happen.  I'll then see if IPython needs
fixing to live in a post-decorators world or not, and work accordingly.

Best,

f

From bob at redivi.com  Tue Aug 10 00:10:35 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Aug 10 00:10:42 2004
Subject: [Python-Dev] Re: Re: Decorators: vertical bar syntax
In-Reply-To: <cf8ri9$kcc$1@sea.gmane.org>
References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>
	<200408091504.i79F4lD22945@guido.python.org>
	<cf8dgs$l8a$1@sea.gmane.org>
	<200408092014.i79KEh123856@guido.python.org>
	<cf8ri9$kcc$1@sea.gmane.org>
Message-ID: <F03D42C1-EA50-11D8-9D40-000A95686CD8@redivi.com>


On Aug 9, 2004, at 5:50 PM, Fernando Perez wrote:

> I feel like I've said as much as is reasonable on this topic, so I'll 
> probably
> shut up and wait for something to happen.  I'll then see if IPython 
> needs
> fixing to live in a post-decorators world or not, and work accordingly.

Wouldn't it be reasonable to just change the magic character that 
IPython (and maybe even Leo) uses from '@' to ':', '=', or '/'?   As 
far as I know, it's practically[1] never valid Python to start a line 
with any of these symbols (this list of not definitive, of course).  
'/' has the advantage, for IPython, that it's easy to type (no shift 
involved, at least on my US keyboard) and should be familiar to many 
people as a "command" character from IRC clients, searching in vim and 
less, etc.

[1] It can be done with a backslash line continuation or in a 
multi-line docstring, but I wouldn't really consider that the beginning 
of a line in the practical sense...

-bob
From foom at fuhm.net  Tue Aug 10 00:14:12 2004
From: foom at fuhm.net (James Y Knight)
Date: Tue Aug 10 00:14:14 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but
	aren't
Message-ID: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>

There's a fair number of classes that claim they are defined in 
__builtin__, but do not actually appear there. For example:

 >>> def qual(clazz):
...     return clazz.__module__ + '.' + clazz.__name__
...
 >>> qual(types.GeneratorType)
'__builtin__.generator'
 >>> qual(types.FunctionType)
'__builtin__.function'
 >>> qual(types.MethodType)
'__builtin__.instancemethod'
 >>> qual(types.NoneType)
'__builtin__.NoneType'
 >>> qual(types.GeneratorType)
'__builtin__.generator'
 >>> __builtin__.generator
AttributeError: 'module' object has no attribute 'generator'
[[[etc.]]]

IMO classes ought to actually appear in __builtin__ if they claim they 
are defined there. Doing otherwise breaks reflection, as you have to 
add a special case for these?class names to use the appropriate object 
from the types module instead. Thoughts? If it isn't desirable to have 
these names appear in __builtin__, perhaps their '__module__' should be 
changed to another module where they are defined?

James
From jim at zope.com  Tue Aug 10 00:16:53 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Aug 10 00:16:56 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__
	but	aren't
In-Reply-To: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
Message-ID: <4117F7D5.3090907@zope.com>

James Y Knight wrote:
> There's a fair number of classes that claim they are defined in 
> __builtin__, but do not actually appear there. For example:
> 
>  >>> def qual(clazz):
> ...     return clazz.__module__ + '.' + clazz.__name__
> ...
>  >>> qual(types.GeneratorType)
> '__builtin__.generator'
>  >>> qual(types.FunctionType)
> '__builtin__.function'
>  >>> qual(types.MethodType)
> '__builtin__.instancemethod'
>  >>> qual(types.NoneType)
> '__builtin__.NoneType'
>  >>> qual(types.GeneratorType)
> '__builtin__.generator'
>  >>> __builtin__.generator
> AttributeError: 'module' object has no attribute 'generator'
> [[[etc.]]]
> 
> IMO classes ought to actually appear in __builtin__ if they claim they 
> are defined there. Doing otherwise breaks reflection, as you have to add 
> a special case for these class names to use the appropriate object from 
> the types module instead.  Thoughts?

I agree.

> If it isn't desirable to have these 
> names appear in __builtin__, perhaps their '__module__' should be 
> changed to another module where they are defined?

+1

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From amk at amk.ca  Tue Aug 10 00:16:42 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Tue Aug 10 00:17:08 2004
Subject: [Python-Dev] Third Bug Day outcome
In-Reply-To: <006101c47d90$8228efa0$191fc797@oemcomputer>
References: <20040809213820.GC11199@rogue.amk.ca>
	<006101c47d90$8228efa0$191fc797@oemcomputer>
Message-ID: <20040809221642.GA11485@rogue.amk.ca>

On Sun, Aug 08, 2004 at 05:41:48PM -0400, Raymond Hettinger wrote:
> What is involved beyond the Wiki page and announcing a day?

The most burdensome part is just hanging around all day so there's
someone who can close bugs, commit changes, answer questions, etc.
For the first two bug days, I was the only committer around so that
meant I had to stay on IRC for 8 hours.  This past bug day, there were
lots of committers around, so I could have easily skipped out for a
few hours to go grocery shopping, run errands, or whatever.

Also, 9 to 5 is a good length for the bug day; by 3 or 4 PM I'm
starting to get frazzled, but the day is almost over so that's OK.
The first bug day was announced as being 12 hours long, which would
have ben deadly, but luckily SF CVS stopped working around 6PM on that
first day and saved me.

--amk

From FBatista at uniFON.com.ar  Tue Aug 10 00:19:04 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Aug 10 00:23:01 2004
Subject: [Python-Dev] Third Bug Day outcome
Message-ID: <A128D751272CD411BC9200508BC2194D053C7790@escpl.tcp.com.ar>

[A.M. Kuchling]

#- Also, 9 to 5 is a good length for the bug day; by 3 or 4 PM I'm
#- starting to get frazzled, but the day is almost over so that's OK.
#- The first bug day was announced as being 12 hours long, which would
#- have ben deadly, but luckily SF CVS stopped working around 
#- 6PM on that
#- first day and saved me.

9 to 5 where?

Maybe for the next Bug Day should be two "IRC hosts", one in America and
other in Europe. so, the'll be 6 hs each (top) and the whole day will be of
12 hours.

Not-being-able-to-be-him-because-of-56.6k-modem-ly yours,

.	Facundo

From fperez528 at yahoo.com  Tue Aug 10 00:23:43 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Tue Aug 10 00:23:48 2004
Subject: [Python-Dev] Re: Re: Re: Decorators: vertical bar syntax
References: <04Aug8.183153pdt."58612"@synergy1.parc.xerox.com>
	<200408091504.i79F4lD22945@guido.python.org>
	<cf8dgs$l8a$1@sea.gmane.org>
	<200408092014.i79KEh123856@guido.python.org>
	<cf8ri9$kcc$1@sea.gmane.org>
	<F03D42C1-EA50-11D8-9D40-000A95686CD8@redivi.com>
Message-ID: <cf8thf$o5f$1@sea.gmane.org>

Bob Ippolito wrote:

> 
> On Aug 9, 2004, at 5:50 PM, Fernando Perez wrote:
> 
>> I feel like I've said as much as is reasonable on this topic, so I'll
>> probably
>> shut up and wait for something to happen.  I'll then see if IPython
>> needs
>> fixing to live in a post-decorators world or not, and work accordingly.
> 
> Wouldn't it be reasonable to just change the magic character that
> IPython (and maybe even Leo) uses from '@' to ':', '=', or '/'?   As
> far as I know, it's practically[1] never valid Python to start a line
> with any of these symbols (this list of not definitive, of course).
> '/' has the advantage, for IPython, that it's easy to type (no shift
> involved, at least on my US keyboard) and should be familiar to many
> people as a "command" character from IRC clients, searching in vim and
> less, etc.

Indeed, if @ becomes the decorator character, I'll post to the ipython user/dev
lists and request community feedback.  Any of these is an option, and I've
also considered using %.  Note that in most cases in ipython, you don't really
type the @ character explicitly, since ipython auto-detects magics when there
is no chance of ambiguity.  You only need the explicit @ when you have both a
magic and a variable with the same name.

As I said, this is techincally trivial to fix.  It's just a bunch of busywork,
and it will break user's _habits_.

I'll keep your point of non-shifting characters in mind when the time comes,
thanks for pointing that out.

> [1] It can be done with a backslash line continuation or in a
> multi-line docstring, but I wouldn't really consider that the beginning
> of a line in the practical sense...

For ipython these only make sense in single-lines, so this is not a problem.

Thanks for the feedback.

Best,

f

From amk at amk.ca  Tue Aug 10 00:34:38 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Tue Aug 10 00:35:04 2004
Subject: [Python-Dev] Re: Python-dev get-togethers?
In-Reply-To: <20040809221642.GA11485@rogue.amk.ca>
References: <20040809213820.GC11199@rogue.amk.ca>
	<006101c47d90$8228efa0$191fc797@oemcomputer>
	<20040809221642.GA11485@rogue.amk.ca>
Message-ID: <20040809223438.GB11485@rogue.amk.ca>

BTW, Neal Norwitz recently suggested to me that having local, physical
meetings of python-dev'vers would be useful for planning (and perhaps
development) purposes.  While Guido and Jeremy are no longer in the DC
area, there's still a number of people here -- Tim, Fred, Barry, Jim,
Neil Schemenauer, Neal Norwitz, Steve Holden, me -- who could usefully
chew over issues or work on some particularly difficult task or bug.

(I don't think it's very useful to congregate physically just to work
on run-of-the-mill bugs and patches; as a group we've become pretty
skilled at working remotely on simpler things.)

We could meet at someone's house, do two or four hours of work or
discussion, and then turn it into a party.  Would anyone else in the
DC metro area be interested in coming to such a thing?  Please mail me
privately if you'd be interested, or if you could offer your home as a
location.

(This wouldn't happen until later in the fall, October/Novemberish;
for now I just want to see if there's any interest.)

--amk
From tim.peters at gmail.com  Tue Aug 10 00:39:01 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 10 00:39:11 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae0408091354605dc58b@mail.gmail.com>
References: <1f7befae0408091354605dc58b@mail.gmail.com>
Message-ID: <1f7befae040809153946522f6d@mail.gmail.com>

Here's a cute one:

"""
import compiler, sys

f = open('../Lib/test/test_parser.py')
guts = f.read()
f.close()

def ouch(n):
    if n == 0:
        return compiler.compile(guts, "<string>", "exec")
    else:
        return ouch(n-1)

for n in range(100, 250):
    try:
        ouch(n)
        msg = 'ok'
    except Exception, msg:
        msg = str(sys.exc_info()[0]) + ' ' + str(msg)
    print n, msg
"""

Under 2.3.4, that works as expected:  when n hit a large enough value,
from then on it was all repetitions of recursion-depth exceptions:

,,,
147 ok
148 ok
149 ok
150 exceptions.RuntimeError maximum recursion depth exceeded
151 exceptions.RuntimeError maximum recursion depth exceeded
152 exceptions.RuntimeError maximum recursion depth exceeded
...

Under CVS, it's very different.  Under a debug build, I get no output
*at all*(!).  Not even if I change the loop to start at 0.  Under a
release build:

...
120 ok
121 ok
122 ok
123 exceptions.RuntimeError maximum recursion depth exceeded
124 exceptions.RuntimeError maximum recursion depth exceeded
125 exceptions.RuntimeError maximum recursion depth exceeded
126 exceptions.KeyError 307
127 exceptions.RuntimeError maximum recursion depth exceeded
128 exceptions.KeyError 306
129 exceptions.RuntimeError maximum recursion depth exceeded
130 exceptions.KeyError 305
131 exceptions.RuntimeError maximum recursion depth exceeded
132 exceptions.KeyError 304
133 exceptions.RuntimeError maximum recursion depth exceeded
134 exceptions.KeyError 303
135 exceptions.KeyError 302
136 exceptions.RuntimeError maximum recursion depth exceeded
137 exceptions.KeyError 301
138 exceptions.RuntimeError maximum recursion depth exceeded
139 exceptions.KeyError 300
140 exceptions.RuntimeError maximum recursion depth exceeded
141 exceptions.RuntimeError maximum recursion depth exceeded
142 exceptions.KeyError 299
143 exceptions.RuntimeError maximum recursion depth exceeded
144 exceptions.KeyError 297
145 exceptions.KeyError 296
146 exceptions.RuntimeError maximum recursion depth exceeded
147 exceptions.KeyError 295
148 exceptions.RuntimeError maximum recursion depth exceeded
149 exceptions.KeyError 294
150 exceptions.RuntimeError maximum recursion depth exceeded
151 exceptions.RuntimeError maximum recursion depth exceeded
152 exceptions.RuntimeError maximum recursion depth exceeded
153 exceptions.KeyError 309
154 exceptions.RuntimeError maximum recursion depth exceeded
155 exceptions.RuntimeError maximum recursion depth exceeded
156 exceptions.KeyError 307
157 exceptions.RuntimeError maximum recursion depth exceeded
158 exceptions.KeyError 306
159 exceptions.RuntimeError maximum recursion depth exceeded
160 exceptions.KeyError 305
161 exceptions.RuntimeError maximum recursion depth exceeded
162 exceptions.KeyError 304
163 exceptions.RuntimeError maximum recursion depth exceeded
164 exceptions.KeyError 303
165 exceptions.KeyError 302
166 exceptions.RuntimeError maximum recursion depth exceeded
...

and there's a seemingly non-random but hard-to-fathom jumble of
recursion-depth and KeyError exceptions thereafter too.

So something's really hosed in CVS, or in MSVC 7.1, or ...
From tdelaney at avaya.com  Tue Aug 10 00:40:59 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Aug 10 00:41:05 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>

Walter D?rwald wrote:

> I can't understand why we can't have a new keyword for decorators.
> If I remember correctly the introduction of yield didn't result
> in such a public outcry. We'd have to change our programs once
> if a variable names collides with the new keyword, but that's better
> than having to look at @s for the rest of our Python days.

Not such a large outcry, but lots of us were concerned about "having to look inside the function for 'yield' to determine if it were a generator or a function" and were calling for a 'gen' or 'generator' keyword.

Strangely enough, exactly the *opposite* argument is now being used to prevent syntaxes such as:

    def func (args):
        @decorator

as was used then to allow:

    def func (args):
        yield value

Tim Delaney
From aahz at pythoncraft.com  Tue Aug 10 00:55:37 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Aug 10 00:55:40 2004
Subject: [Python-Dev] Python startup time?
Message-ID: <20040809225537.GA23187@panix.com>

I just remembered that shortly before release of Python 2.3 there was a
big fuss about the startup time.  Has anyone revisited the issue since
then?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From mhammond at skippinet.com.au  Tue Aug 10 01:14:50 2004
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Tue Aug 10 01:14:39 2004
Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows
In-Reply-To: <20040807150229.GC2529@rogue.amk.ca>
Message-ID: <05af01c47e66$b1ab6420$0200a8c0@eden>

> OK; I just care about platforms where the alternative constant is
> necessary.  It sounds to me like win32 is therefore the only relevant
> sys.platform string to check for.  Thanks!

I believe the Windows CE port uses "wince", and I see no reason not to think
async code will end up running there (assuming the platform itself
continues).

Mark.

From tdelaney at avaya.com  Tue Aug 10 01:19:18 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Aug 10 01:19:24 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CAC2@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

> Please remember the procedure.  The community is to come up with *one*
> alternative that they universally prefer over @decorators; otherwise,
> @decorators-before-def will stay.  (I'll even give you *two* if the
> community can agree that both are acceptable and really wishes to let
> me pick one.)

And yet in the poll to gain support for one option, we've been actively
discouraged by Anthony from participating on the grounds that "weight of
opinion" is no good ... ;)

(No - not an accusation - just a suggestion that the procedure is
possibly not clear to all the major participants).

Tim Delaney
From nbastin at opnet.com  Tue Aug 10 01:42:12 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Tue Aug 10 01:42:36 2004
Subject: [Python-Dev] Python startup time?
In-Reply-To: <20040809225537.GA23187@panix.com>
References: <20040809225537.GA23187@panix.com>
Message-ID: <BCADF9F6-EA5D-11D8-AAEA-000D932927FE@opnet.com>


On Aug 9, 2004, at 6:55 PM, Aahz wrote:

> I just remembered that shortly before release of Python 2.3 there was a
> big fuss about the startup time.  Has anyone revisited the issue since
> then?

Experience from pycon sprint revealed several things to me:

1) 2.3 startup is faster than 2.2 if you don't import site
2) 2.2 didn't really import anything on startup, so it was always fast
3) Never benchmark without installing, as performance is really bad in 
CVS (distutils)

If people who really care about startup time don't need site, then 
they're good.  If they do, then the real slowdown comes from warnings 
and linecache, if I recall correctly.

--
Nick

From nbastin at opnet.com  Tue Aug 10 01:45:06 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Tue Aug 10 01:45:14 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__
	but	aren't
In-Reply-To: <4117F7D5.3090907@zope.com>
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<4117F7D5.3090907@zope.com>
Message-ID: <24A02222-EA5E-11D8-AAEA-000D932927FE@opnet.com>


On Aug 9, 2004, at 6:16 PM, Jim Fulton wrote:

> James Y Knight wrote:
>> IMO classes ought to actually appear in __builtin__ if they claim 
>> they are defined there. Doing otherwise breaks reflection, as you 
>> have to add a special case for these class names to use the 
>> appropriate object from the types module instead.  Thoughts?
>
> I agree.

I think this should stand for functions as well.  help() is a good 
example.  It's a bit confusing for newbies that help.__module__ is 
'site', but 'site' isn't in their local namespace.

--
Nick

From python at rcn.com  Mon Aug  9 02:26:54 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug 10 02:27:03 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae040809153946522f6d@mail.gmail.com>
Message-ID: <007c01c47da7$92b71920$191fc797@oemcomputer>

> and there's a seemingly non-random but hard-to-fathom jumble of
> recursion-depth and KeyError exceptions thereafter too.
> 
> So something's really hosed in CVS, or in MSVC 7.1, or ...

This also occurs on MSVC++ 6.0.

I'm working back in time and it still occurs on the 6/21/2004 checkout.
That excludes my eval locals argument, Armin's string optimization, and
Nick's C profiling re-org.


Raymond

From mkc at mathdogs.com  Tue Aug 10 03:15:52 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Tue Aug 10 03:15:56 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<lkdwu0bcz7r.fsf@panix2.panix.com>
	<20040809033554.GA21340@panix.com>
Message-ID: <lkdekmf7pyv.fsf@panix2.panix.com>

Aahz <aahz@pythoncraft.com> writes:
> This section needs fleshing out to explain why re.structmatch works better
> for some purposes than forcing people to switch to a full-blown parser.

That's a good point.  I spent a while thinking about this, but didn't document
why I thought it was still worthwhile.  (See my nearby reply to Stephen
Turnbull for some thoughts.)

Mike


From tjreedy at udel.edu  Tue Aug 10 03:18:14 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue Aug 10 03:18:18 2004
Subject: [Python-Dev] Decorator classification: location, format, details
Message-ID: <cf97om$bg9$1@sea.gmane.org>

I found the recent discussion of decorators rather confusing until I
noticed that objections and options to prefixed @deco fell into three
categories: location, format, and details.  Perhaps this structuring might
help the debate.

LOCATION (6 proposals/alternatives)

Multiple def prefix - dedented relative to def and covering all indented
under it.

Prefix to single def - the alpha2 innovation.

Infix in header, before or after function name.

Suffix to header, after param list, before :.

Top of body

postfiX - the 2.3 status quo

FORMAT

The format depends somewhat but not completely on location, and there have
been one to many proposed for each.  By 'format' I here mean a symbolic
template without literals (except possibly \n).  In grammar terms, a
production rhs without terminals.

For alpha2 prefix, the format is, simplified, sequence of decos, where deco
is pfix user_decorator.  One alternative prefix form is open seq of
user_decorators close.  Another is a prefixed suite.

For 2.3, the postfiX form is func_name = user_decorator(func_name).

DETAILS

These are the constant literals supplied by the designer (or grammar)
rather than the symbolic variables filled in by the user/programmer.
Grammatically, these are the terminal productions.

For alpha2, pfix is '@'.  Suggested alternatives include '=', '|', and
'deco'.


I notice in the PEP, the wiki, and here that discussions of a particular
option with all 3 aspects fixed tend to mix discussion of the pros and cons
of its location, which actually apply to all formats in that location, and
discussion the pros and cons of the format, which mostly would apply to the
format in any position.  I think greater separation would be less
confusing.  Is it possible to first pick a location, then a format, then
fill in the details?

Terry J. Reedy



From greg at cosc.canterbury.ac.nz  Tue Aug 10 03:27:52 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug 10 03:27:58 2004
Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2,
	and @decorators)
In-Reply-To: <200408091525.i79FPLM23023@guido.python.org>
Message-ID: <200408100127.i7A1RqAi007468@cosc353.cosc.canterbury.ac.nz>

> My own gut feeling is that it's not important to allow parentheses on
> "bare" (from-less) imports, since you can easily repeat the 'import'
> keyword on a new line;

It might not be strictly necessary, but for consistency
I think it should be alllowed.

By the way, rather than parentheses, I would prefer to
be able to simply end the line with a comma and continue
on to the next line:

  from Bubble import foo, spam, tomato,
    grail, arthur, camembert

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From aahz at pythoncraft.com  Tue Aug 10 03:36:44 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Aug 10 03:36:46 2004
Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2,
	and @decorators)
In-Reply-To: <200408100127.i7A1RqAi007468@cosc353.cosc.canterbury.ac.nz>
References: <200408091525.i79FPLM23023@guido.python.org>
	<200408100127.i7A1RqAi007468@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040810013644.GB5503@panix.com>

>> My own gut feeling is that it's not important to allow parentheses on
>> "bare" (from-less) imports, since you can easily repeat the 'import'
>> keyword on a new line;
> 
> It might not be strictly necessary, but for consistency
> I think it should be alllowed.

Unless other support is forthcoming, this won't happen.

> By the way, rather than parentheses, I would prefer to
> be able to simply end the line with a comma and continue
> on to the next line:
> 
>   from Bubble import foo, spam, tomato,
>     grail, arthur, camembert

Rejected earlier on parsing grounds.  I've updated the PEP.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From mkc at mathdogs.com  Tue Aug 10 03:38:08 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Tue Aug 10 03:38:13 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<lkdwu0bcz7r.fsf@panix2.panix.com>
	<87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <lkdacx37oxr.fsf@panix2.panix.com>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
> >>>>> "Mike" == Mike Coleman <mkc@mathdogs.com> writes:
>     Mike>     m0 = re.match(r'([A-Z]+|[a-z]+)*', 'XxxxYzz')

> Sure, but regexp syntax is a horrible way to express that.

Do you mean, horrible compared to spelling it out using a Python loop that
walks through the array, or horrible compared to some more serious parsing
package?

For the former, I would disagree.  I see code like this a lot and it drives me
crazy.  Reminds me of the bad old days of building 'while' loops out of 'if's
and 'goto's.

For the latter, I think it depends on the complexity of the matching, and the
level of effort required to learn and distribute the "not-included" parsing
package.  I certainly wouldn't want to see someone try to write a language
front-end with this, but for a lot of text-scraping activities, I think it
would be very useful.

> This feature would be an attractive nuisance, IMHO.

I agree that, like list comprehensions (for example), it needs to be applied
with good judgement.

Turning it around, though, why *shouldn't* there be a good mechanism for
returning the multiple matches for multiply matching groups?  Why should this
be an exception?  If you agree that there should be a mechanism, it certainly
doesn't have to be the one in the PEP, but what would be better?  I'd welcome
alternative ideas here.


>     Mike>     p = r'((?:(?:^|:)([^:\n]*))*\n)*\Z'
> 
> This is a _easy_ one, but even it absolutely requires being written
> with (?xm) and lots of comments, don't you think?

I think it's preferable--that's why I did it.  :-)

> If you're going to be writing a multiline, verbose regular expression, why
> not write a grammar instead, which (assuming a modicum of library support)
> will be shorter and self-documenting?

If there were a suitable parsing package in the standard library, I agree that
this would probably be a lot less useful.

As things stand right now, though, it's a serious irritation that we have a
standard mechanism that *almost* does this, but quits at the last moment.  If
I may wax anthropomorphic, the 're.match' function says to me as a programmer

    *You* know what structure this RE represents, and *I* know what
    structure it represents, too, because I had to figure it out to 
    do the match.  But too bad, sucker, I'm not going to tell you what
    I found!

Irritating as hell.

Mike

From tim.peters at gmail.com  Tue Aug 10 03:48:37 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 10 03:48:44 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <007c01c47da7$92b71920$191fc797@oemcomputer>
References: <007c01c47da7$92b71920$191fc797@oemcomputer>
Message-ID: <1f7befae04080918481e1f8053@mail.gmail.com>

>> and there's a seemingly non-random but hard-to-fathom jumble of
>> recursion-depth and KeyError exceptions thereafter too.
>>
>> So something's really hosed in CVS, or in MSVC 7.1, or ...

[Raymond Hettinger]
> This also occurs on MSVC++ 6.0.
>
> I'm working back in time and it still occurs on the 6/21/2004 checkout.
> That excludes my eval locals argument, Armin's string optimization, and
> Nick's C profiling re-org.

Thank you for trying this stuff!  You already ruled out my #1 guess,
namely that the "catch an exception from alloca()" trick we use on
Windows to detect impending stack overflow stopped working in 7.1. 
But if it sucks under 6.0 too ...

Another possibility is that it's something new & evil in the CVS
compiler package (when my driver does "import compiler" under 2.3.4,
of course it picks up 2.3.4's compiler package).

Do you also get no output at all under a 6.0 debug build?

Has anyone on Linux tried this yet?
From greg at cosc.canterbury.ac.nz  Tue Aug 10 03:52:30 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug 10 03:52:40 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
In-Reply-To: <d620mad6.fsf@yahoo.co.uk>
Message-ID: <200408100152.i7A1qUYD007511@cosc353.cosc.canterbury.ac.nz>

> Worse still is the fact that the exponent in the e notation is a
> decimal exponent, but the type used is binary!

Indeed. If someone really wants a binary float, I think they
should have to write it in hex. If they can't cope with that,
they've no business using binary floats.

<0xd37c*pow(2,-0x10) wink>,

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From d.holton at vanderbilt.edu  Fri Aug  6 19:13:16 2004
From: d.holton at vanderbilt.edu (Doug Holton)
Date: Tue Aug 10 03:58:56 2004
Subject: [Python-Dev] request: add keywords for static and class methods
 only [related to decorators discussion]
Message-ID: <4113BC2C.1010206@vanderbilt.edu>

After reading some posts here and the wiki page discussing the 
decorators:  http://www.python.org/moin/PythonDecorators
One idea that some people have suggested is to separate out standard 
uses for decorators like function metadata, vs. built-in features of 
python, like static methods and class methods.

So I'm suggesting, don't use decorators for static and class methods.

Perhaps for 2.5 or whenever, we should add keyword support for built-in 
features, just like java:

These two keywords at least:

def static mymethod (arg1, arg2):
     ....

def classmethod mymethod( cls, arg1, arg2 ):
     ....

And then you have to decide if you want to support other java built-in 
features and keywords like private, protected, public, synchronized.

See the bottom of the wiki page for ideas for the long term future that 
combine these keywords with decorators and other possible features.

From barnesc at engr.orst.edu  Sat Aug  7 02:38:27 2004
From: barnesc at engr.orst.edu (barnesc@engr.orst.edu)
Date: Tue Aug 10 03:59:00 2004
Subject: [Python-Dev] Another decorator syntax idea
Message-ID: <1091839107.4114248325784@webmail.oregonstate.edu>


Another decorator syntax idea

Using Guido's example:

--------------------
@ Symbols
--------------------

class C(object):

    @staticmethod
    @funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", \
               status="experimental", author="BDFL")
    def longMethodNameForEffect(longArgumentOne=None,
                                longArgumentTwo=42):
        """This method blah, blah.

        It supports the following arguments:
	- longArgumentOne -- a string giving ...
        - longArgumentTwo -- a number giving ...

	blah, blah.

	"""
        raise NotYetImplemented

--------------------
decorate keyword
--------------------

class C(object):

    def longMethodNameForEffect(longArgumentOne=None,
                                longArgumentTwo=42):
        decorate:
            staticmethod
            funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]", \
                      status="experimental", author="BDFL")]
        """This method blah, blah.

        It supports the following arguments:
	- longArgumentOne -- a string giving ...
        - longArgumentTwo -- a number giving ...

	blah, blah.

	"""
        raise NotYetImplemented

Pros:
 - Doesn't hide the function name behind @ garbage.
 - 'decorate' is easier to look at than '@'.
 - 'decorate' is more meaningful to most readers than '@'.
 - Block structure isolates decorators from surrounding code,
   making them more readable.
 - Indentation rules are consistent with the rest of Python.

Cons:
 - Not pear shaped.

 - Connelly
From bokr at oz.net  Sun Aug  8 01:51:58 2004
From: bokr at oz.net (Bengt Richter)
Date: Tue Aug 10 03:59:04 2004
Subject: [Python-Dev] @decorator syntax is sugar, but for what exactly?
Message-ID: <41153969.226164657@news.oz.net>

ISTM that
    @limited_expression_producing_function
    @another
    def func(): pass

is syntactic sugar for creating a hidden list of functions. (Using '|' in place of '@'
doesn't change the picture much (except for people whose tools depend on '@' ;-)).

I.e., (not having the source or time to delve) the apparent semantics of the above
is something roughly like

    __funclist__ = []
    __funclist__.append(limited_expression_producing_function)
    __funclist__.append(another)
    def func():pass
    while __funclist__: func = __funclist__.pop()(func)
    del __funclist__

Is this a special case of a more general idea? E.g., could it apply to
right after ANY next name is bound, in general, not just a name bound by def?

thus (untested)

    def censor(cls):
        cls.__repr__ = lambda self: '<CENSORED>'
        return cls
    ...
    @censor
    class C(object): pass

could have the expected effect (after metaclass effects, if any, presumably, BTW)
(note that censor could instead e.g. wrap selected methods or add class variable data etc.,
though IIRC __metaclass__ can create some things that are read-only later)

This is still very narrowly defined by prefix context. Is this context also
a special case default of something more general? IOW the default choice for
namespace is the lexically enclosing one. What about, e.g., being able to specify
decoration in one place at the top of a module and decorate (in the same way, using
the same function list) all methods of a specified (by name) list of classes?

I.e., a more general control over what to do when what names are bound in what namespace
could be envisaged. This begins to feel like event-driven processing. Could @deco1 @deco2
be sugar for the special case (don't take this literally, just illustrating semantics ;-)

    when(event='next_binding', namespace='immediate', symbols='any', funclist=(deco1,deco2))
    def foo(): pass

of something general that could also include other and possibly repeated events like on completion
of an arg list for a particular function or method being called or any rebinding of particular
symbols in a specified namespace (not just the immediate one), e.g., for debugging or profiling etc.

Since I am copying this to python-dev, I'll re-ask whether @decorator has any peculiar
thread safety pitfalls that should be warned against, just in case.

Please take this as a probe into the intended semantics, not as a proposal for any particular
functionality ;-)

Regards,
Bengt Richter
From bsder at mail.allcaps.org  Mon Aug  9 19:48:44 2004
From: bsder at mail.allcaps.org (Andrew P. Lentvorski, Jr.)
Date: Tue Aug 10 03:59:07 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
In-Reply-To: <hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
Message-ID: <5B8ECC2E-EA2C-11D8-B30C-000A95C874EE@mail.allcaps.org>


On Aug 9, 2004, at 10:29 AM, Mark Hahn wrote:

> Forgive me if I'm being a pest, but no one has commented on the real 
> reason
> I asked the question.  What does everyone think of the idea of having 
> these
> three built-in numeric types?
>
> 1) An Int implemented with infinite precision integer (Python Longs) 
> with
> the constant/__str__ form of +-NN such as 0, -123, 173_394, etc.
>
> 2) A Decimal implemented with the .Net decimal float (or the IBM 
> decimal if
> the .Net decimal sucks too much) with the constant/__str__ form of 
> +-NN.NN
> such as 0.0, -123.0, 173_394.0, 173394.912786765, etc.
>
> 3) A binary Float implemented with the hardware floating point with the
> constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3,
> 2.35e-78, etc.

Be careful defining string representations of numbers.  It is a 
non-trivial
problem to adhere to the correct internationalization standards.

Forgive me if I'm being dense, but don't we already have 1 & 3 as built
in types?

2 is unlikely to be a builtin until folks get some experience using the
Decimal module.  Yes, the .Net Decimal sucks.

In addition, the desired evolution of Python seems to be replacing the
binary floating point as the default floating point type with the
decimal floating point type by default rather than growing another
builtin numeric type.  This is obviously contingent upon the Decimal 
module
getting some real-life use as well as it getting some speed improvements
in the future.

> There would be no automatic conversion except the / operator would 
> convert
> from Int to Decimal

Int goes to the default type of float (which is currently binary FP) 
already
upon using the / operator.

>  and the Math module would convert Int and Decimal
> values to Float for almost all functions (except simple ones like abs, 
> min,
> max).

This is not necessarily a good idea.  Exponentials are very useful in
financial situations and want to be decimals, not binary floats, in 
those
cases.

-a

From greg at cosc.canterbury.ac.nz  Tue Aug 10 04:00:55 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug 10 04:00:59 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
In-Reply-To: <um9olp66c1at.qpsv216awprc.dlg@40tude.net>
Message-ID: <200408100200.i7A20tsq007540@cosc353.cosc.canterbury.ac.nz>

> I have been assuming that there is a fundamental need for an integer
> type that has no fraction.  There are many places like indexing
> where you semantically want integers.

That doesn't necessarily mean you need an integer *type*.  Indexing
code can still reject numbers whose value isn't an integer.

There may be other reasons for wanting an integer type, though. For
instance, allowing integers to become abitrarily large doesn't usually
cause any problem, whereas allowing the precision of fractional
numbers to increase unboundedly can lead to unexpectedly high memory
consumption and slowness.  So you probably want arbitrary precision
integers, but not arbitrary precision fractional numbers (at least by
default).

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From bob at redivi.com  Tue Aug 10 04:10:37 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Aug 10 04:10:45 2004
Subject: [Python-Dev] request: add keywords for static and class methods
	only [related to decorators discussion]
In-Reply-To: <4113BC2C.1010206@vanderbilt.edu>
References: <4113BC2C.1010206@vanderbilt.edu>
Message-ID: <788747D0-EA72-11D8-B58B-000A95686CD8@redivi.com>

On Aug 6, 2004, at 1:13 PM, Doug Holton wrote:

> After reading some posts here and the wiki page discussing the 
> decorators:  http://www.python.org/moin/PythonDecorators
> One idea that some people have suggested is to separate out standard 
> uses for decorators like function metadata, vs. built-in features of 
> python, like static methods and class methods.
>
> So I'm suggesting, don't use decorators for static and class methods.
>
> Perhaps for 2.5 or whenever, we should add keyword support for 
> built-in features, just like java:

Now that's a -1 if I've ever seen one.

-bob
From greg at cosc.canterbury.ac.nz  Tue Aug 10 04:11:50 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug 10 04:11:55 2004
Subject: [Python-Dev] Re: [Prothon-user] Re: Decimal type question [Prothon]
	(was: Re: Re: Decimal type question [Prothon])
In-Reply-To: <1pj8x4u2ekigy$.21gr6je8l10a$.dlg@40tude.net>
Message-ID: <200408100211.i7A2BoqZ007551@cosc353.cosc.canterbury.ac.nz>

> How about a basic built-in numeric default type called Decimal with all the
> default numeric constant formats (123, 123.0, & 123e0) and a separate
> binary double hardware type called Float with the constant format of 0f0,
> 1f0, -1.3f-34, etc. where the letter "f" replaces the letter "e"?

Since the base is the important distinction between these,
if the decimal one is to be called Decimal, the binary one
should be called Binary. The exponent character should be
'b' to reinforce this.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Tue Aug 10 04:17:55 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug 10 04:18:18 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
Message-ID: <200408100217.i7A2Ht9X007573@cosc353.cosc.canterbury.ac.nz>

> Strangely enough, exactly the *opposite* argument is now being used
> to prevent syntaxes such as:
> 
>     def func (args):
>         @decorator
> 
> as was used then to allow:
> 
>     def func (args):
>         yield value

It's not quite the same argument, because 'yield' can appear
anywhere inside the body, buried arbitrarily deeply. Decorators
would only appear at the beginning.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From tim.peters at gmail.com  Tue Aug 10 04:20:18 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 10 04:20:22 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
In-Reply-To: <um9olp66c1at.qpsv216awprc.dlg@40tude.net>
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
	<1f7befae04080911203b46485c@mail.gmail.com>
	<um9olp66c1at.qpsv216awprc.dlg@40tude.net>
Message-ID: <1f7befae040809192054372908@mail.gmail.com>

[Mark Hahn]
> ...
> I'm already planning on using Jim's IronPython implementation of your
> BigInts so I might as well port your Decimal code over for the Prothon
> Decimal also.  The only thing I've ever stolen from Python for Prothon was
> your bigint code so I might as well be consistent in my thievery. :-)

It may not be an easy port, as it freely uses lots of Python features,
including Jim Fulton's new-in-2.4 thread-local storage API (just like
the status and control registers in your HW FPU carry thread-local
state, the IBM spec has thread-local status and control data too).

A saving grace is that IBM's Mike Colishaw worked up thousands of
exacting test cases distributed in a plain text file format, and so a
broken implementation is remarkably easy to detect.
From anthony at interlink.com.au  Tue Aug 10 04:19:31 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug 10 04:20:25 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CAC2@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CAC2@au3010avexu1.global.avaya.com>
Message-ID: <411830B3.3050809@interlink.com.au>

Delaney, Timothy C (Timothy) wrote:
> And yet in the poll to gain support for one option, we've been actively
> discouraged by Anthony from participating on the grounds that "weight of
> opinion" is no good ... ;)

Please don't misinterpret what I wrote. I wrote, on python-list/c.l.py
that merely "collecting votes" was not a useful task - collecting useful
arguments _is_. Many of the posts here (and even more so on c.l.py) are
simply stating and restating the same arguments, or else people posting
"I hate form blah".

In the useful case, I'd point to the discussion on "|" vs "@". People
are working through the issues.

> (No - not an accusation - just a suggestion that the procedure is
> possibly not clear to all the major participants).

The goal here is to attempt to reach a concensus.


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From python at rcn.com  Tue Aug 10 04:30:00 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug 10 04:30:08 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
In-Reply-To: <1f7befae040809192054372908@mail.gmail.com>
Message-ID: <001a01c47e81$efb03500$2e05a044@oemcomputer>

> It may not be an easy port, as it freely uses lots of Python features,
> including Jim Fulton's new-in-2.4 thread-local storage API

FWIW, I made that part optional by putting in backwards compatibility
code.  I periodically verify that it runs fine under Py2.3.


Raymond

From greg at cosc.canterbury.ac.nz  Tue Aug 10 04:44:21 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug 10 04:44:47 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <41175442.4010902@livinglogic.de>
Message-ID: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz>

=?ISO-8859-1?Q?Walter_D=F6rwald?= <walter@livinglogic.de>:

> I can't understand why we can't have a new keyword for decorators.
> If I remember correctly the introduction of yield didn't result
> in such a public outcry.

The problem isn't so much the mechanics of adding a keyword,
but choosing which keyword to use. The expected uses of
decorators are so diverse that no single word would sound
right for all of them.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From foom at fuhm.net  Tue Aug 10 04:48:37 2004
From: foom at fuhm.net (James Y Knight)
Date: Tue Aug 10 04:48:42 2004
Subject: [Python-Dev] @decorator syntax is sugar, but for what exactly?
In-Reply-To: <41153969.226164657@news.oz.net>
References: <41153969.226164657@news.oz.net>
Message-ID: <C7AFE85E-EA77-11D8-8D12-000A95A50FB2@fuhm.net>

On Aug 7, 2004, at 7:51 PM, Bengt Richter wrote:
> ISTM that
>     @limited_expression_producing_function
>     @another
>     def func(): pass
>
> is syntactic sugar for creating a hidden list of functions. (Using '|' 
> in place of '@'
> doesn't change the picture much (except for people whose tools depend 
> on '@' ;-)).


As I understand it, it's really syntax sugar for:
     func = limited_expression_producing_function(another(
         defbutdontbind func(): pass))

except that isn't actually valid python source code. ;) It's nearly, 
but not exactly, the same as
     def func(): pass
     func = another(func)
     func = limited_expression_producing_function(func)


> Is this a special case of a more general idea? E.g., could it apply to
> right after ANY next name is bound, in general, not just a name bound 
> by def?

Right now decorators only work before def.

I posted an idea earlier about a @public decorator, and wrote a quick 
proof-of-concept patch to allow decorators in front of funcdef, 
classdef, and "NAME '=' testlist". This much *can* work with little 
code change, if the functionality is deemed as desirable.

However, as I mentioned, @public in front of a arbitrary binding 
expression (e.g. not class or function) isn't useful unless the 
decorator gets the name of the variable being bound. I didn't quite 
figure out how to manage that part. :)

James

From skip at pobox.com  Tue Aug 10 05:00:21 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 10 05:00:43 2004
Subject: [Python-Dev] Decorators: vertical bar syntax
In-Reply-To: <019301c47db9$c269ec80$a9498890@neil>
References: <200408090216.i792GjER005834@cosc353.cosc.canterbury.ac.nz>
	<019301c47db9$c269ec80$a9498890@neil>
Message-ID: <16664.14917.365982.249725@montanaro.dyndns.org>


    Neil>    A motivation for this was that one of the examples

    Neil> @accepts(int,int)
    Neil> @returns(float)
    Neil> def bar(low,high):

    Neil> which separates related information too much for me. 

Yes, but when I wrote that example, the list-after-def form was the current
king-of-the-hill:

    def bar(low,high) [accepts(int,int), returns(float)]:
        ...

Looks fine to me. ;-)

Skip
From foom at fuhm.net  Tue Aug 10 05:16:39 2004
From: foom at fuhm.net (James Y Knight)
Date: Tue Aug 10 05:16:41 2004
Subject: [Python-Dev] Decorator order implemented backwards?
Message-ID: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>

I haven't seen anyone remark upon this yet, but the order of decorator 
application appears to be implemented backwards.

According to the PEP:
> Proposed Syntax
>
> The current syntax for function decorators as implemented in Python 
> 2.4a2 is:
> @dec2
> @dec1
> def func(arg1, arg2, ...):
>     pass
>
>
> This is equivalent to:
> def func(arg1, arg2, ...):
>     pass
> func = dec2(dec1(func))

However, it seems the '@' form is really doing dec1(dec2(func)), not 
dec2(dec1(func)).
The test_decorators.py reveals that it is, indeed, testing *for* the 
wrong behavior.

test_decorators.py:
>     def test_order(self):
>         class C(object):
>             @funcattrs(abc=1) @staticmethod
>             def foo(): return 42
>         # This wouldn't work if staticmethod was called first
>         self.assertEqual(C.foo(), 42)
>         self.assertEqual(C().foo(), 42)

So, um...I assume this is a bug in the implementation, not the PEP, 
because the PEP's way makes sense and the way it's implemented doesn't.

James

From python at rcn.com  Tue Aug 10 05:17:02 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug 10 05:17:09 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae04080918481e1f8053@mail.gmail.com>
Message-ID: <000001c47e88$81ef1d40$5635c797@oemcomputer>

> >> and there's a seemingly non-random but hard-to-fathom jumble of
> >> recursion-depth and KeyError exceptions thereafter too.
> >>
> >> So something's really hosed in CVS, or in MSVC 7.1, or ...
> 
> [Raymond Hettinger]
> > This also occurs on MSVC++ 6.0.
> >
> > I'm working back in time and it still occurs on the 6/21/2004
checkout.
> > That excludes my eval locals argument, Armin's string optimization,
and
> > Nick's C profiling re-org.
> 
> Thank you for trying this stuff!  You already ruled out my #1 guess,
> namely that the "catch an exception from alloca()" trick we use on
> Windows to detect impending stack overflow stopped working in 7.1.
> But if it sucks under 6.0 too ...

Yes, the usual suspects are innocent.


> Another possibility is that it's something new & evil in the CVS
> compiler package (when my driver does "import compiler" under 2.3.4,
> of course it picks up 2.3.4's compiler package).
> 
> Do you also get no output at all under a 6.0 debug build?

I've gone back to Jan 15, 2004 and can still reproduce the error on a
release build.  Haven't made a debug build yet.  BTW, if you try this,
remember, the opcodes were difference, so do a full rebuild and clear
away the pyc files.


Raymond

From mark at prothon.org  Tue Aug 10 05:24:00 2004
From: mark at prothon.org (Mark Hahn)
Date: Tue Aug 10 05:23:53 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<hn3ki191sdrt$.lk1mso101q6k.dlg@40tude.net>
	<5B8ECC2E-EA2C-11D8-B30C-000A95C874EE@mail.allcaps.org>
Message-ID: <dgmpflwnif8r$.1vr00eq6kroop$.dlg@40tude.net>

On Mon, 9 Aug 2004 10:48:44 -0700, Andrew P. Lentvorski, Jr. wrote:

> On Aug 9, 2004, at 10:29 AM, Mark Hahn wrote:
> 
>> Forgive me if I'm being a pest, but no one has commented on the real 
>> reason
>> I asked the question.  What does everyone think of the idea of having 
>> these
>> three built-in numeric types?
>>
>> 1) An Int implemented with infinite precision integer (Python Longs) 
>> with
>> the constant/__str__ form of +-NN such as 0, -123, 173_394, etc.
>>
>> 2) A Decimal implemented with the .Net decimal float (or the IBM 
>> decimal if
>> the .Net decimal sucks too much) with the constant/__str__ form of 
>> +-NN.NN
>> such as 0.0, -123.0, 173_394.0, 173394.912786765, etc.
>>
>> 3) A binary Float implemented with the hardware floating point with the
>> constant/__str__ form of +-NN.NN+-eNN such as 0e0, -123e0, 173.39e+3,
>> 2.35e-78, etc.
> 
> Be careful defining string representations of numbers.  It is a 
> non-trivial
> problem to adhere to the correct internationalization standards.
> 
> Forgive me if I'm being dense, but don't we already have 1 & 3 as built
> in types?

There is no "we".  This thread is about Prothon which has nothing yet since
it hasn't ever been released.  Your feedback is still relevant and helpful
though.

> 2 is unlikely to be a builtin until folks get some experience using the
> Decimal module.  Yes, the .Net Decimal sucks.
> 
> In addition, the desired evolution of Python seems to be replacing the
> binary floating point as the default floating point type with the
> decimal floating point type by default rather than growing another
> builtin numeric type.  This is obviously contingent upon the Decimal 
> module
> getting some real-life use as well as it getting some speed improvements
> in the future.
> 
>> There would be no automatic conversion except the / operator would 
>> convert
>> from Int to Decimal
> 
> Int goes to the default type of float (which is currently binary FP) 
> already
> upon using the / operator.
> 
>>  and the Math module would convert Int and Decimal
>> values to Float for almost all functions (except simple ones like abs, 
>> min,
>> max).
> 
> This is not necessarily a good idea.  Exponentials are very useful in
> financial situations and want to be decimals, not binary floats, in 
> those
> cases.

Good point.

From tdelaney at avaya.com  Tue Aug 10 05:26:57 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Aug 10 05:27:04 2004
Subject: [Python-Dev] Decorator order implemented backwards?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF3D1F@au3010avexu1.global.avaya.com>

James Y Knight wrote:

> So, um...I assume this is a bug in the implementation, not the PEP,
> because the PEP's way makes sense and the way it's implemented
> doesn't. 

Especially since it will be most common to put @staticmethod or
@classmethod first, which will play havoc with any decorator following
that is expecting a function object ...

Tim Delaney
From barry at python.org  Tue Aug 10 05:29:38 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 10 05:29:25 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz>
References: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz>
Message-ID: <1092108578.18208.10.camel@localhost>

On Mon, 2004-08-09 at 22:44, Greg Ewing wrote:
> =?ISO-8859-1?Q?Walter_D=F6rwald?= <walter@livinglogic.de>:
> 
> > I can't understand why we can't have a new keyword for decorators.
> > If I remember correctly the introduction of yield didn't result
> > in such a public outcry.
> 
> The problem isn't so much the mechanics of adding a keyword,
> but choosing which keyword to use. The expected uses of
> decorators are so diverse that no single word would sound
> right for all of them.

Actually, that's an interesting case study.  Myself and others advocated
for a keyword other than 'def' for introducing generators.  That wasn't
how things turned out and in hindsight I think Guido made the right
decision.  I'm confident the same thing will happen with decorators.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040809/ca58ff72/attachment-0001.pgp
From tim.peters at gmail.com  Tue Aug 10 06:06:16 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 10 06:06:20 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <1092108578.18208.10.camel@localhost>
References: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz>
	<1092108578.18208.10.camel@localhost>
Message-ID: <1f7befae040809210679c2d4a0@mail.gmail.com>

[Barry Warsaw]
> Actually, that's an interesting case study.  Myself and others advocated
> for a keyword other than 'def' for introducing generators.  That wasn't
> how things turned out and in hindsight I think Guido made the right
> decision.  I'm confident the same thing will happen with decorators.

Ya, but it's sure taking Guido long enough to figure that out in this case!

def staticmethod:
def author(name='Barry'):
def returns([int]):
def method(self):
     return 42

a-keyword-for-all-seasons-ly y'rs  - tim
From martin at v.loewis.de  Tue Aug 10 06:26:05 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 10 06:26:04 2004
Subject: [Python-Dev] Python startup time?
In-Reply-To: <20040809225537.GA23187@panix.com>
References: <20040809225537.GA23187@panix.com>
Message-ID: <41184E5D.9090002@v.loewis.de>

Aahz wrote:
> I just remembered that shortly before release of Python 2.3 there was a
> big fuss about the startup time.  Has anyone revisited the issue since
> then?

As Nick says, we looked at it at PyCon. I found that a lot of calls to
string interning are made, and changed the marshal format to share
interned strings, instead of duplicating them. This causes good savings
in disk space; whether it also saves startup time significantly is hard
to prove.

Regards,
Martin

From anthony at interlink.com.au  Tue Aug 10 07:00:27 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug 10 07:06:05 2004
Subject: [Python-Dev] Decorator order implemented backwards?
In-Reply-To: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
Message-ID: <4118566B.2090907@interlink.com.au>

James Y Knight wrote:
> I haven't seen anyone remark upon this yet, but the order of decorator 
> application appears to be implemented backwards.
>>     def test_order(self):
>>         class C(object):
>>             @funcattrs(abc=1) @staticmethod
>>             def foo(): return 42
>>         # This wouldn't work if staticmethod was called first
>>         self.assertEqual(C.foo(), 42)
>>         self.assertEqual(C().foo(), 42)

How odd. I'm positive at one point I tested this and it was
the right way round.

 >>> def dec1(x):
...     print "dec1"
...     return x
...
 >>> def dec2(x):
...     print "dec2"
...     return x
...
 >>>
 >>> @dec1
... @dec2
... def foo(): pass
...
dec1
dec2
 >>>

This is definitely wrong. Wrong wrong wrong.

Anthony
From anthony at interlink.com.au  Tue Aug 10 07:08:20 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug 10 07:08:53 2004
Subject: [Python-Dev] Decorator order implemented backwards?
In-Reply-To: <4118566B.2090907@interlink.com.au>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au>
Message-ID: <41185844.80906@interlink.com.au>

Anthony Baxter wrote:
> James Y Knight wrote:
> 
>> I haven't seen anyone remark upon this yet, but the order of decorator 
>> application appears to be implemented backwards.
>>
>>>     def test_order(self):
>>>         class C(object):
>>>             @funcattrs(abc=1) @staticmethod
>>>             def foo(): return 42
>>>         # This wouldn't work if staticmethod was called first
>>>         self.assertEqual(C.foo(), 42)
>>>         self.assertEqual(C().foo(), 42)
> 
> 
> How odd. I'm positive at one point I tested this and it was
> the right way round.

Mark, is it possible that this changed between the first and
final versions of the decorator patch? The SF report doesn't
list any of the older versions...

Thanks,
Anthony
From guido at python.org  Tue Aug 10 07:23:04 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 10 07:23:12 2004
Subject: [Python-Dev] Decorator order implemented backwards?
In-Reply-To: Your message of "Tue, 10 Aug 2004 13:26:57 +1000."
	<338366A6D2E2CA4C9DAEAE652E12A1DE01CF3D1F@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF3D1F@au3010avexu1.global.avaya.com>
Message-ID: <200408100523.i7A5N4Z24708@guido.python.org>

> > So, um...I assume this is a bug in the implementation, not the PEP,
> > because the PEP's way makes sense and the way it's implemented
> > doesn't. 
> 
> Especially since it will be most common to put @staticmethod or
> @classmethod first, which will play havoc with any decorator following
> that is expecting a function object ...

Um, right, this is backwards from what I had wanted.  Sorry, I
should've caught that before 2.4a2 went out!

--Guido van Rossum (home page: http://www.python.org/~guido/)
From python at rcn.com  Tue Aug 10 07:52:43 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug 10 07:52:50 2004
Subject: [Python-Dev] @decorator syntax is sugar, but for what exactly?
In-Reply-To: <41153969.226164657@news.oz.net>
Message-ID: <000001c47e9e$417c1540$5635c797@oemcomputer>



> -----Original Message-----
> From: python-dev-bounces+python=rcn.com@python.org [mailto:python-dev-
[Bengt Richter]
> Since I am copying this to python-dev, I'll re-ask whether @decorator
has
> any peculiar
> thread safety pitfalls that should be warned against, just in case.

I would think that no thread safety issues would arise unless there were
a stateful decorator.  


Raymond

From fredrik at pythonware.com  Tue Aug 10 08:12:31 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Aug 10 08:10:58 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com><lkdwu0bcz7r.fsf@panix2.panix.com><87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp>
	<lkdacx37oxr.fsf@panix2.panix.com>
Message-ID: <cf9ot9$7bh$1@sea.gmane.org>

Mike Coleman wrote:

> If I may wax anthropomorphic, the 're.match' function says to me as a programmer
>
>    *You* know what structure this RE represents, and *I* know what
>    structure it represents, too, because I had to figure it out to
>    do the match.

that only shows that you dont understand how regular expressions work.

a regular expression defines a set of strings, and the RE engine is designed to
tell you if a string you have is a member of this set.  the engine's not a parser,
and it has a very vague notion of "structure" (groups are implemented by
"marks", which are registers that keeps track of where the engine last passed
a given position; changing that to "lists of all possible matches" would require
a major rewrite).

you're probably better off using the scanner mechanism:

    http://effbot.org/zone/xml-scanner.htm

or the sre.Scanner class (see the source code).  the scanner concept could
need some documentation, and it would be nice to be able to switch patterns
during parsing.  (time for a scanner PEP, anyone?)

</F> 



From ncoghlan at iinet.net.au  Tue Aug 10 08:35:04 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue Aug 10 08:35:17 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <000001c47e88$81ef1d40$5635c797@oemcomputer>
References: <000001c47e88$81ef1d40$5635c797@oemcomputer>
Message-ID: <41186C98.9050604@iinet.net.au>

Raymond Hettinger wrote:

> I've gone back to Jan 15, 2004 and can still reproduce the error on a
> release build.  Haven't made a debug build yet.  BTW, if you try this,
> remember, the opcodes were difference, so do a full rebuild and clear
> away the pyc files.

Thank you Raymond!

You just solved my problem with the totally psychotic behaviour I was 
getting after building with the free toolkit.

I must have had some broken .pyc's lying around, and they were sending 
ceval.c insane. . .

Now maybe I can start being useful around here :)

Cheers,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From fredrik at pythonware.com  Tue Aug 10 08:42:25 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Aug 10 08:40:50 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu><1092074454.8687.26.camel@localhost><eppstein-16B12F.11450609082004@sea.gmane.org><cf8l1e$9rk$1@sea.gmane.org>
	<eppstein-5CF885.13492009082004@sea.gmane.org>
Message-ID: <cf9qla$aer$1@sea.gmane.org>

David Eppstein wrote:

> I think quoted string recognition in Python is still regular, not
> context-free.  Anyway, the syntax highlighters in editors that I've used
> don't seem to have a problem with it.

shouldn't a simple parenthesis counter suffice?  you have to count
parens in many cases anyway (e.g. to find function bodies).

</F> 



From tim.peters at gmail.com  Tue Aug 10 08:48:15 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 10 08:48:18 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <000001c47e88$81ef1d40$5635c797@oemcomputer>
References: <000001c47e88$81ef1d40$5635c797@oemcomputer>
Message-ID: <1f7befae0408092348583f0953@mail.gmail.com>

Well, this gets nasty.

In a debug build, and starting the loop at 0, I can't get off the
gound in the MS 7.1 debugger.  It dies quickly with an access
violation in the bowels of ntdll.dll, and I don't have source for
that.

PyOS_CheckStack on Windows does this to detect stack overflow (it
catches an MS exception if the C runtime can't allocate enough room on
the stack):

	alloca(PYOS_STACK_MARGIN * sizeof(void*));

It's trying to see whether there's still room for 2K pointers on the
stack.  If I multiply that by 2, or by 3, nothing changes.  But if I
multiply it by 4, everything changes.  Then the "oops!  we're gonna
blow the stack!" exit from PyOS_CheckStack is taken.  It returns 1 to
_Py_CheckRecursiveCall, which sets a "stack overflow" MemoryError and
returns -1 to its caller.

That's

	if (Py_EnterRecursiveCall(" in cmp"))
		return NULL;

in PyObject_RichCompare.  That's just trying to compare two ints.

So NULL gets returned to PyObject_RichCompareBool, which in turn
returns -1 to lookdict.  AAAARGHGH!  lookdict "isn't allowed" to raise
exceptions, so it does a PyErr_Clear(), goes on to futilely chase the
entire dict looking for another match on the hash code, and we've
effectively turned a MemoryError into a KeyError.  I expect that
explains a lot about what we see in the release-build runs.

If I multiply the stack check by 20, I can finally get some results
out of the debug build:

0 exceptions.KeyError 299
1 exceptions.MemoryError Stack overflow
2 exceptions.MemoryError Stack overflow
3 exceptions.MemoryError Stack overflow
4 exceptions.KeyError 295
5 exceptions.MemoryError Stack overflow
6 exceptions.KeyError 294
7 exceptions.MemoryError Stack overflow
8 exceptions.MemoryError Stack overflow
9 exceptions.MemoryError Stack overflow
10 exceptions.MemoryError Stack overflow
11 exceptions.MemoryError Stack overflow
12 exceptions.MemoryError Stack overflow
13 exceptions.MemoryError Stack overflow
14 exceptions.KeyError 309
15 exceptions.KeyError 296
...

So we're blowing the C stack left and right in this test case, and
sometimes dict lookup turns that into a KeyError.

The question is what we did since 2.3.4 that apparently increases our
stack demands, and grossly increases them in a debug build(!).  Could
be that the compile package is more heavily recursive now too (no
idea).  test_parser.py in 2,3.4 contained the same deeply nested
tuples, so that's not what changed.

Back in a release build, and restoring the original Windows
stack-check code, but leaving the driver loop starting at 0, I have to
sys.setrecursionlimit(16) to avoid getting any KeyErrors. 
sys.setrecursionlimit(878) is the minimum that allows at least one
"ok" to show up:

0 ok
1 exceptions.RuntimeError maximum recursion depth exceeded
2 exceptions.RuntimeError maximum recursion depth exceeded
3 exceptions.RuntimeError maximum recursion depth exceeded
4 exceptions.KeyError 307
5 exceptions.RuntimeError maximum recursion depth exceeded
6 exceptions.KeyError 306
7 exceptions.RuntimeError maximum recursion depth exceeded
8 exceptions.KeyError 305
...

Misc/find_recursionlimit.py in CVS manages to print

    Limit of 1000 is fine

before it craps out in a release build; in a debug build, it doesn't
produce *any* output.  If I change the limit it starts with to 100, it
manages to get to

   Limit of 400 is fine

in a debug build before stopping without a clue.

Hmm!  But over in 2.3.4 build, a release build also stopped with 1000,
and a debug build also exited mysteriously.  But after reducing its
starting point to 100, it got to

    Limit of 700 is fine

before crapping out.

BTW, in 2.3.4 and CVS, when a debug run craps out mysteriously like
this, it has an exit code of 128.  That's scary:

    http://support.microsoft.com/support/kb/articles/q184/8/02.asp
From eppstein at ics.uci.edu  Tue Aug 10 09:15:35 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Tue Aug 10 09:15:40 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu>
	<cf8l1e$9rk$1@sea.gmane.org>
	<eppstein-5CF885.13492009082004@sea.gmane.org>
	<cf9qla$aer$1@sea.gmane.org>
Message-ID: <eppstein-8BA820.00153510082004@sea.gmane.org>

In article <cf9qla$aer$1@sea.gmane.org>,
 "Fredrik Lundh" <fredrik@pythonware.com> wrote:

> > I think quoted string recognition in Python is still regular, not
> > context-free.  Anyway, the syntax highlighters in editors that I've used
> > don't seem to have a problem with it.
> 
> shouldn't a simple parenthesis counter suffice?  you have to count
> parens in many cases anyway (e.g. to find function bodies).

I think it would suffice (with some care to ignore parens in comments 
and strings, and assuming there aren't any mismatches), but counting is 
not something you can do in a regexp.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From fredrik at pythonware.com  Tue Aug 10 09:34:08 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Aug 10 09:32:31 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
References: <41145C05.1020501@gradient.cis.upenn.edu><cf8l1e$9rk$1@sea.gmane.org><eppstein-5CF885.13492009082004@sea.gmane.org><cf9qla$aer$1@sea.gmane.org>
	<eppstein-8BA820.00153510082004@sea.gmane.org>
Message-ID: <cf9tma$gh1$1@sea.gmane.org>

David Eppstein wrote:

>> shouldn't a simple parenthesis counter suffice?  you have to count
>> parens in many cases anyway (e.g. to find function bodies).
>
> I think it would suffice (with some care to ignore parens in comments
> and strings, and assuming there aren't any mismatches), but counting is
> not something you can do in a regexp.

no, but adding a single integer register to the scanning loop isn't
that much work.

</F> 



From Paul.Moore at atosorigin.com  Tue Aug 10 10:21:06 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Tue Aug 10 10:21:11 2004
Subject: [Python-Dev] Decorator order implemented backwards?
Message-ID: <16E1010E4581B049ABC51D4975CEDB88052CB386@UKDCX001.uk.int.atosorigin.com>

From: James Y Knight
> So, um...I assume this is a bug in the implementation,
> not the PEP, because the PEP's way makes sense and the
> way it's implemented doesn't.

Interestingly, I found the PEP's description felt "backwards"
to me, but didn't mention it because I didn't want to start
another long discussion over something which has proved to
be subjective in past discussions. I'd vote for correcting
the PEP rather than the code (not least because it's probably
easier :-))

But consistency is the only real requirement here.

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________
From fredrik at pythonware.com  Tue Aug 10 11:00:08 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Aug 10 11:00:17 2004
Subject: [Python-Dev] Re: re.split on empty patterns
References: <20040807145142.GB2529@rogue.amk.ca>
	<1f7befae040807105878b47c61@mail.gmail.com>
Message-ID: <cfa2qo$ru9$1@sea.gmane.org>

Tim Peters wrote:

> > Anyway, we therefore can't just make this the default in 2.4.  We
> > could trigger a warning when emptyok is not supplied and a split
> > pattern results in a zero-length match; users could supply
> > emptyok=False to avoid the warning.  Patterns that never have a
> > zero-length match would never get the warning.  2.5 could then set
> > emptyok to True.
> >
> > Note: raising the warning might cause a serious performance hit for
> > patterns that get zero-length matches a lot, which would make 2.4
> > slower in certain cases.
>
> If you don't intend to change the default, there's no problem.  I like
> "no problem".  This isn't so useful so often that it can't afford to
> wait for Python 3 to change.  In the meantime, "emptyok" is an odd
> name since it's always "ok" to have an empty match.  "split0=True"
> reads better to me, since the effect is to split on a 0-length match.
> split_on_empty_match would be too wordy.

the RE module already have an established method for specifying flags,
either with (?x) markers in the pattern strings, or via the "flags" argument
to re.compile and others.

any reason why you cannot use a new standard flag for this purpose?

(e.g. (?e) and re.EMPTY_OK)

adding a flags argument to re.split is less important; if someone really needs
this feature, they can use (?e) or re.compile(..., re.EMPTY_OK).split()

for extra credit, add EMPTY_OK behaviour to find, finditer, and sub/subn.

</F>



From gmccaughan at synaptics-uk.com  Tue Aug 10 11:47:50 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Tue Aug 10 11:48:22 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
In-Reply-To: <22FD7E0D-EA3B-11D8-B417-000A95C874EE@mail.allcaps.org>
References: <1aa0tmdegut1n$.456rgexpcp9f$.dlg@40tude.net>
	<d620mad6.fsf@yahoo.co.uk>
	<22FD7E0D-EA3B-11D8-B417-000A95C874EE@mail.allcaps.org>
Message-ID: <200408101047.50275.gmccaughan@synaptics-uk.com>

On Monday 2004-08-09 20:34, Andrew P. Lentvorski, Jr. wrote:
> 
> On Aug 9, 2004, at 11:31 AM, Paul Moore wrote:
> 
> > I think that having 2 different fractional types distinguished by
> > something as subtle as the presence of an exponent (something no other
> > language does to my knowledge) is an extremely bad idea.
> 
> Doesn't Common Lisp use 1.0e+10 to signify singles and 1.0d+10 to
> signify doubles?
> 
> That doesn't make it a good idea, but there is precedent.

What CL does is a bit hairier than that. CL has, potentially,
*four* different float formats. (An implementation is allowed
to coalesce some or all of them.) s,f,d,l are used as exponent
letters for short, single, double, and long floats, respectively;
e means "use the default". The default default is single-float,
but users can change that if they prefer (say) doubles. (I'm
sure that if CL were being designed now rather than N years ago
double-float would be the default default.)

At least one implementation known to me (CLISP) does have
four different kinds of float. CLISP's long-floats have
user-settable precision, which can be as big as you like,
which makes CLISP a very handy platform for some kinds of
numerical experiments...

Fortran also distinguishes between 1.0e10 and 1.0d10, in much
the same way. In Fortran 90 and later, where there's potentially
an even greater plethora of floating types than in CL, there's
a more general mechanism for specifying the type of a floating-point
constant. It's very nasty. Floating-point types are represented
as integers. You define a parameter (i.e., a variable that isn't
allowed to change) that holds the type you're interested in,
and then you suffix your constants with "_" plus the parameter
name. Thus:

    integer, parameter my_real_type = selected_real_kind(15,100)
    real(my_real_type) x
    x = 1.234_my_real_type

It's not clear to me that any of this is very valuable in a
statically-typed language without type inference: the type
that a constant should be is almost always deducible from
context. (More precisely: there's always a way to deduce
a type that's almost always the one wanted.)

-- 
g

From skip at pobox.com  Tue Aug 10 14:55:53 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 10 14:56:10 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <2m7js83733.fsf@starship.python.net>
References: <41176596.5060209@hlabs.spb.ru>
	<2m7js83733.fsf@starship.python.net>
Message-ID: <16664.50649.835422.998574@montanaro.dyndns.org>


    >> Is there a plan for implementing a base class for int and long (like
    >> basestring for str and unicode):

    Michael> Not that I'm aware of.

We've discussed the numeric type hierarchy in the past.  I think it would be
worth considering all the numeric types at once and come up with a
reasonable hierarchy that includes float, complex and decimal types as well.
(It's clear there is no "best" way to do that.)

Skip
From skip at pobox.com  Tue Aug 10 14:56:09 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 10 14:56:29 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <41175442.4010902@livinglogic.de>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
	<cf0gd1$smv$1@sea.gmane.org> <1091818207.7529.34.camel@localhost>
	<4113D5E1.6040006@interlink.com.au>
	<41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de>
Message-ID: <16664.50665.170916.600964@montanaro.dyndns.org>


    Walter> So how about:

    Walter> make classmethod
    Walter> def foo(cls, bar):
    Walter>     ...

I don't think anybody's come up with a candidate keyword that reads well in
most/all situations.  Consider:

    make accepts(int, (int, float))
    make returns(int)
    def foo(arg1, arg2):
        return arg1 * arg2

While you can contort "make" to mean what you want in this case it's clearly
a much different meaning ("ensure that (or force) this function to accept
and return these types) than "make classmethod" ("turn this function into a
class method").  Other potential keywords have similar problems.

Skip

From skip at pobox.com  Tue Aug 10 14:56:27 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 10 14:56:40 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <A5C91C16-EA08-11D8-9661-000A95686CD8@redivi.com>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
	<cf0gd1$smv$1@sea.gmane.org> <1091818207.7529.34.camel@localhost>
	<4113D5E1.6040006@interlink.com.au>
	<41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de>
	<6682D8E6-EA04-11D8-8B30-0003934AD54A@chello.se>
	<A5C91C16-EA08-11D8-9661-000A95686CD8@redivi.com>
Message-ID: <16664.50683.86050.621985@montanaro.dyndns.org>


    >> I think "adorn" would make a better keyword in that case, as in
    >> "adorn with" this decorator the method foo.

    Bob> I don't think that's appropriate at all.  Most decorators are
    Bob> transformations that change the function object into something else
    Bob> entirely, most likely by wrapping it but quite possibly not.

Correctamundo.  In fact, there's no need that the decorator return a
function (or callable) object at all.  You could define a function to
compute a lookup table of some sort.  The decorator might simply invoke the
function and return its value.  Perfectly legal.  Slightly more than
adornment.  Somewhat silly example:

    def define(func):
        return func()

    @ define
    def grayscale():
        table = []
        for i in range(256):
            table.append((i,i,i))
        return table

Skip
From erik at heneryd.com  Tue Aug 10 14:57:10 2004
From: erik at heneryd.com (Erik Heneryd)
Date: Tue Aug 10 14:57:15 2004
Subject: [Python-Dev] Re: pre-PEP: Complete, Structured Regular Expression
	Group Matching
In-Reply-To: <lkdzn56aljx.fsf@panix2.panix.com>
References: <411429FB.6080603@gradient.cis.upenn.edu>	<lkd1xijedzx.fsf@panix2.panix.com>
	<brhn9guy.fsf@yahoo.co.uk> <lkdzn56aljx.fsf@panix2.panix.com>
Message-ID: <4118C626.4070203@heneryd.com>

Mike Coleman wrote:
> Paul Moore <pf_moore@yahoo.co.uk> writes:

>>The only other comment I have is that the semantics seem pretty
>>complex - I think that in practice, they do more or less what you want
>>them to, but the description is pretty obscure.

I'd say that is probably a warning flag, but I don't know, at the same 
time it looks like it could be kind of useful.

>>And although I can see that the error return has some value, I suspect that
>>it might actually complicate real use.
> 
> 
> Someone else suggested throwing an exception instead, which I'm now leaning in
> favor of, too.

Exceptions are the way to go.  You should get what you ask for, or a 
heads up.  Errors should never pass silently.

> 
> 
>>A suggestion - would it be possible to implement re.structmatch as a
>>pure Python prototype, to thrash out some of the usability questions?
>>If the function appears valuable in such a form, arguing for
>>incorporation into the re module would be a lot easier.
> 
> 
> I suspect that in this case a pure Python implementation might actually be a
> lot harder than implementing it as a C patch.  This is because so much of the
> work is already done in the current re module; the C patch would probably not
> be that big.  In order to do it in Python I'd end up reimplementing re.match
> in Python, which I wouldn't look forward to.

I hacked up a Python prototype, mostly as a challenge.  It's brittle and 
it's ugly, but it kind of works.  Though, if /F says it would require a 
major rewrite of the C-engine, that's probably true... <wink>


Erik

-------------- next part --------------
A non-text attachment was scrubbed...
Name: structmatch.py
Type: text/x-python
Size: 5760 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040810/a5e602ab/structmatch.py
From erik at heneryd.com  Tue Aug 10 15:00:50 2004
From: erik at heneryd.com (Erik Heneryd)
Date: Tue Aug 10 15:00:56 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular
	Expression Group Matching
In-Reply-To: <cf9ot9$7bh$1@sea.gmane.org>
References: <lkdekmlvv8k.fsf@panix2.panix.com><lkdwu0bcz7r.fsf@panix2.panix.com><87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp>	<lkdacx37oxr.fsf@panix2.panix.com>
	<cf9ot9$7bh$1@sea.gmane.org>
Message-ID: <4118C702.8020707@heneryd.com>

Fredrik Lundh wrote:
> a regular expression defines a set of strings, and the RE engine is designed to
> tell you if a string you have is a member of this set.  the engine's not a parser,

Well, as you know (and point out now and then), regexes aren't regular 
expressions in the CS sense.  And most people don't use em just to see 
if a string matches or not, but to see what parts matches waht.


> you're probably better off using the scanner mechanism:
> 
>     http://effbot.org/zone/xml-scanner.htm
> 
> or the sre.Scanner class (see the source code).  the scanner concept could
> need some documentation, and it would be nice to be able to switch patterns
> during parsing.  (time for a scanner PEP, anyone?)

I'd love to see scanners as an official part of the stdlib.  Isn't this 
just a matter of documentation?


Erik

From fredrik at pythonware.com  Tue Aug 10 15:06:36 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Aug 10 15:06:41 2004
Subject: [Python-Dev] Re: Re: pre-PEP: Complete,
	Structured Regular ExpressionGroup Matching
References: <411429FB.6080603@gradient.cis.upenn.edu>	<lkd1xijedzx.fsf@panix2.panix.com><brhn9guy.fsf@yahoo.co.uk>
	<lkdzn56aljx.fsf@panix2.panix.com> <4118C626.4070203@heneryd.com>
Message-ID: <cfah8s$van$1@sea.gmane.org>

Erik Heneryd wrote:

> I hacked up a Python prototype, mostly as a challenge.  It's brittle and
> it's ugly, but it kind of works.  Though, if /F says it would require a
> major rewrite of the C-engine, that's probably true... <wink>

if done on the C level, that is.  partial compilation on the Python level
is slightly easier, as you've just shown.  clever!

(but probably rather slow; I still recommend building something on top
of the scanner object)

</F>



From fredrik at pythonware.com  Tue Aug 10 15:08:53 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Aug 10 15:10:31 2004
Subject: [Python-Dev] Re: Re: Re: Call for defense of @decorators
References: <200408051636.i75Gaak03654@guido.python.org><20040805174252.GA27820@burma.localdomain><20040805112318.B13062@ActiveState.com><20040805183444.GA29796@burma.localdomain><20040805190549.GC29796@burma.localdomain><4112A18D.5090801@v.loewis.de><20040805215331.GB31288@burma.localdomain><ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost><1f7befae040806090644be9c04@mail.gmail.com><cf0gd1$smv$1@sea.gmane.org>
	<1091818207.7529.34.camel@localhost><4113D5E1.6040006@interlink.com.au><41141255.5010408@stackless.com>
	<41175442.4010902@livinglogic.de>
	<16664.50665.170916.600964@montanaro.dyndns.org>
Message-ID: <cfahd5$vn7$1@sea.gmane.org>

Skip Montanaro wrote:

> I don't think anybody's come up with a candidate keyword that reads well in
> most/all situations.  Consider:
>
>     make accepts(int, (int, float))
>     make returns(int)
>     def foo(arg1, arg2):
>         return arg1 * arg2

how about "at", pronounced "annotation type" ? (duck)

</F>



From lists at hlabs.spb.ru  Tue Aug 10 19:25:07 2004
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Tue Aug 10 15:19:10 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <16664.50649.835422.998574@montanaro.dyndns.org>
References: <41176596.5060209@hlabs.spb.ru>	<2m7js83733.fsf@starship.python.net>
	<16664.50649.835422.998574@montanaro.dyndns.org>
Message-ID: <411904F3.1080703@hlabs.spb.ru>

Skip Montanaro wrote:
>     >> Is there a plan for implementing a base class for int and long (like
>     >> basestring for str and unicode):
> 
>     Michael> Not that I'm aware of.
> 
> We've discussed the numeric type hierarchy in the past.  I think it would be
> worth considering all the numeric types at once and come up with a
> reasonable hierarchy that includes float, complex and decimal types as well.
> (It's clear there is no "best" way to do that.)

Something like following?

object
     numeric
         integer
             int
             long
         float
         complex


PS I've already have implementation for baseinteger, but can't made a 
patch since cvs.python.sourceforge.net always told me 'Connection timed 
out'... :-/

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru
From jepler at unpythonic.net  Tue Aug 10 15:26:13 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Tue Aug 10 15:26:27 2004
Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows
In-Reply-To: <20040807142327.GA2529@rogue.amk.ca>
References: <20040807142327.GA2529@rogue.amk.ca>
Message-ID: <20040810132612.GB28542@unpythonic.net>

Do any systems *but* win32 have SO_EXLUSIVEADDRUSE ?  (and if they do,
is it better than SO_REUSEADDR for the same reasons it is on win32?)

.. so why not write
    if hasattr(socket, 'SO_EXCLUSIVEADDRUSE'):
        reuse_constant = socket.SO_EXCLUSIVEADDRUSE
    else:
        reuse_constant = socket.SO_REUSEADDR

SO_EXCLUSIVEADDRUSE sounds like the exact opposite of SO_REUSEADDR,
according to a page I found when trying to find out whether any other
platform has SO_EXCLUSEIVEADDRUSE:
    http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/using_so_exclusiveaddruse.asp
.. SO_REUSEADDR's semantics on Windows are apparently different from
Unix, but SO_EXCLUSEIVEADDRUSE is different still.

I'm no win32-platform programmer, just trying to make sense of what I
read on the web.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040810/2fd33ee5/attachment.pgp
From walter at livinglogic.de  Tue Aug 10 15:28:23 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Aug 10 15:28:34 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <16664.50665.170916.600964@montanaro.dyndns.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
	<cf0gd1$smv$1@sea.gmane.org> <1091818207.7529.34.camel@localhost>
	<4113D5E1.6040006@interlink.com.au>
	<41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de>
	<16664.50665.170916.600964@montanaro.dyndns.org>
Message-ID: <4118CD77.30204@livinglogic.de>

Skip Montanaro wrote:

>     Walter> So how about:
> 
>     Walter> make classmethod
>     Walter> def foo(cls, bar):
>     Walter>     ...
> 
> I don't think anybody's come up with a candidate keyword that reads well in
> most/all situations.  Consider:
> 
>     make accepts(int, (int, float))
>     make returns(int)
>     def foo(arg1, arg2):
>         return arg1 * arg2
> 
> While you can contort "make" to mean what you want in this case it's clearly
> a much different meaning ("ensure that (or force) this function to accept
> and return these types) than "make classmethod" ("turn this function into a
> class method").  Other potential keywords have similar problems.

If you know what the keyword is, you can rename the decorator
accordingly:

     make argcheck(int, (int, float))
     make returncheck(int)

But we could choose a keyword that doesn't mean anything, e.g.
'dec' or 'decorate'.

Bye,
    Walter D?rwald



From mwh at python.net  Tue Aug 10 15:35:16 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug 10 15:35:18 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <411904F3.1080703@hlabs.spb.ru> (Dmitry Vasiliev's message of
	"Tue, 10 Aug 2004 17:25:07 +0000")
References: <41176596.5060209@hlabs.spb.ru>
	<2m7js83733.fsf@starship.python.net>
	<16664.50649.835422.998574@montanaro.dyndns.org>
	<411904F3.1080703@hlabs.spb.ru>
Message-ID: <2mpt5z15gr.fsf@starship.python.net>

Dmitry Vasiliev <lists@hlabs.spb.ru> writes:

> Skip Montanaro wrote:
>>     >> Is there a plan for implementing a base class for int and long (like
>>     >> basestring for str and unicode):
>>     Michael> Not that I'm aware of.
>> We've discussed the numeric type hierarchy in the past.  I think it
>> would be
>> worth considering all the numeric types at once and come up with a
>> reasonable hierarchy that includes float, complex and decimal types as well.
>> (It's clear there is no "best" way to do that.)
>
> Something like following?
>
> object
>      numeric
>          integer
>              int
>              long
>          float
>          complex
>
>
> PS I've already have implementation for baseinteger, but can't made a
> patch since cvs.python.sourceforge.net always told me 'Connection
> timed out'... :-/

I think you want cvs.sourceforge.net these days.

Cheers,
mwh

-- 
  I located the link but haven't bothered to re-read the article,
  preferring to post nonsense to usenet before checking my facts.
                                      -- Ben Wolfson, comp.lang.python
From skip at pobox.com  Tue Aug 10 15:35:14 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 10 15:35:37 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <411904F3.1080703@hlabs.spb.ru>
References: <41176596.5060209@hlabs.spb.ru>
	<2m7js83733.fsf@starship.python.net>
	<16664.50649.835422.998574@montanaro.dyndns.org>
	<411904F3.1080703@hlabs.spb.ru>
Message-ID: <16664.53010.746248.733772@montanaro.dyndns.org>


    >> I think it would be worth considering all the numeric types at once
    >> and come up with a reasonable hierarchy that includes float, complex
    >> and decimal types as well.  (It's clear there is no "best" way to do
    >> that.)

    Dmitry> Something like following?

    Dmitry> object
    Dmitry>      numeric
    Dmitry>          integer
    Dmitry>              int
    Dmitry>              long
    Dmitry>          float
    Dmitry>          complex

At some level it seems that float is a specialization of complex (w/ imag
field == 0).  Also, it's not clear that float or complex don't have
something other than a sibling relationship to integers.

I don't have the time to dig up the previous discussions.  Three days out of
town left me we nearly a thousand Python-related mail messages.  I have no
idea how (if at all) I'll be able to boil any of that down into reasonable
edits to PEP 318.

Skip
From mwh at python.net  Tue Aug 10 15:37:46 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug 10 15:37:48 2004
Subject: [Python-Dev] Third Bug Day outcome
In-Reply-To: <20040809213820.GC11199@rogue.amk.ca> (A. M. Kuchling's message
	of "Mon, 9 Aug 2004 17:38:20 -0400")
References: <20040809213820.GC11199@rogue.amk.ca>
Message-ID: <2mllgn15cl.fsf@starship.python.net>

"A.M. Kuchling" <amk@amk.ca> writes:

> The bug day on Saturday went well, as usual.  19 bugs and 12 patches
> were closed.  That's not as good as the first bug day (30 bugs), but
> is competitive with the second (18 bugs, 21 patches).  Lots of the
> easy bugs have been cleaned out, I expect, so each remaining bug takes
> more time to fix.
>
> The composition of the turnout was the surprising thing. My plaintive
> bleating on python-dev resulted in much higher participation by people
> with CVS committers, but there weren't many non-committer people
> around (Seo Sanghyeon and Mike Coleman were the two non-developers I
> noticed).

Well apart from jlgijsbers who you mentioned in your post :-)

I don't know how much longer we should let him get away with not being
a commiter, though <wink>.

Cheers,
mwh

-- 
  ... so the notion that it is meaningful to pass pointers to memory
  objects into which any random function may write random values
  without having a clue where they point, has _not_ been debunked as
  the sheer idiocy it really is.        -- Erik Naggum, comp.lang.lisp
From amk at amk.ca  Tue Aug 10 15:40:23 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Tue Aug 10 15:40:52 2004
Subject: [Python-Dev] More PEPs to mark as final
Message-ID: <20040810134023.GA13993@rogue.amk.ca>

The following PEPs all seem to be completed; can their authors (or
someone) please confirm, and mark them as 'Final' if the PEPs are in
fact completed?

273  Import Modules from Zip Archives
278  Universal Newline Support
282  A Logging System
302  New Import Hooks    
305  CSV File API  

--amk
From lists at hlabs.spb.ru  Tue Aug 10 19:58:23 2004
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Tue Aug 10 15:52:32 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <2mpt5z15gr.fsf@starship.python.net>
References: <41176596.5060209@hlabs.spb.ru>	<2m7js83733.fsf@starship.python.net>	<16664.50649.835422.998574@montanaro.dyndns.org>	<411904F3.1080703@hlabs.spb.ru>
	<2mpt5z15gr.fsf@starship.python.net>
Message-ID: <41190CBF.6010405@hlabs.spb.ru>

Michael Hudson wrote:
> Dmitry Vasiliev <lists@hlabs.spb.ru> writes:
[SKIP]
>>PS I've already have implementation for baseinteger, but can't made a
>>patch since cvs.python.sourceforge.net always told me 'Connection
>>timed out'... :-/
> 
> I think you want cvs.sourceforge.net these days.

You've right, thanks.

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru
From mwh at python.net  Tue Aug 10 15:55:19 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug 10 15:55:21 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__
	but aren't
In-Reply-To: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net> (James Y.
	Knight's message of "Mon, 9 Aug 2004 18:14:12 -0400")
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
Message-ID: <2mekmf14jc.fsf@starship.python.net>

James Y Knight <foom@fuhm.net> writes:

> There's a fair number of classes that claim they are defined in
> __builtin__, but do not actually appear there. For example:

"__builtin__" is the "I don't know" answer in type_module for
non-HEAPTYPEs.  I'm certainly not sure that's totally wise...

> IMO classes ought to actually appear in __builtin__ if they claim they
> are defined there. Doing otherwise breaks reflection, as you have to
> add a special case for these?class names to use the appropriate object
> from the types module instead. Thoughts? If it isn't desirable to have
> these names appear in __builtin__, perhaps their '__module__' should
> be changed to another module where they are defined?

Such as?  There really isn't a module where e.g. GeneratorType is
defined.

Cheers,
mwh

-- 
3. Syntactic sugar causes cancer of the semicolon.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html
From exarkun at divmod.com  Tue Aug 10 16:02:36 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Tue Aug 10 16:02:46 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__
	but aren't
In-Reply-To: <2mekmf14jc.fsf@starship.python.net>
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<2mekmf14jc.fsf@starship.python.net>
Message-ID: <4118D57C.9090306@divmod.com>

Michael Hudson wrote:
> James Y Knight <foom@fuhm.net> writes:
>>IMO classes ought to actually appear in __builtin__ if they claim they
>>are defined there. Doing otherwise breaks reflection, as you have to
>>add a special case for these class names to use the appropriate object
>>from the types module instead. Thoughts? If it isn't desirable to have
>>these names appear in __builtin__, perhaps their '__module__' should
>>be changed to another module where they are defined?
> 
> 
> Such as?  There really isn't a module where e.g. GeneratorType is
> defined.
> 

   Seems perfectly reasonable and useful to add GeneratorType and others 
to the types module.  I have code, for example, like this, in a couple 
places:

     def _getACell(o):
         x = o
         def y():
             y = x
         return [o for o in gc.get_referrers(x)
                 if type(o).__name__ == 'cell'][0]

     def _f():
         types.CellType = type(_getACell(object()))
     _f()

   Jp

From erik at heneryd.com  Tue Aug 10 16:04:39 2004
From: erik at heneryd.com (Erik Heneryd)
Date: Tue Aug 10 16:04:44 2004
Subject: [Python-Dev] Re: Re: pre-PEP: Complete,
	Structured Regular ExpressionGroup Matching
In-Reply-To: <cfah8s$van$1@sea.gmane.org>
References: <411429FB.6080603@gradient.cis.upenn.edu>	<lkd1xijedzx.fsf@panix2.panix.com><brhn9guy.fsf@yahoo.co.uk>	<lkdzn56aljx.fsf@panix2.panix.com>
	<4118C626.4070203@heneryd.com> <cfah8s$van$1@sea.gmane.org>
Message-ID: <4118D5F7.1090709@heneryd.com>

Fredrik Lundh wrote:
> if done on the C level, that is.  partial compilation on the Python level
> is slightly easier, as you've just shown.  clever!
> 
> (but probably rather slow; I still recommend building something on top
> of the scanner object)

Yes, ofcourse it's slow, as I recursivly compile subpatterns and try 
them out, but as things are, it's probably much easier to implement it 
this way than using scanners, which I suspect is a lot of work (given 
the structmatch interface)...


Erik
From mwh at python.net  Tue Aug 10 16:05:40 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug 10 16:05:42 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__
	but aren't
In-Reply-To: <4118D57C.9090306@divmod.com> (Jp Calderone's message of "Tue,
	10 Aug 2004 10:02:36 -0400")
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com>
Message-ID: <2macx31423.fsf@starship.python.net>

Jp Calderone <exarkun@divmod.com> writes:

> Michael Hudson wrote:
>> James Y Knight <foom@fuhm.net> writes:
>>>IMO classes ought to actually appear in __builtin__ if they claim they
>>>are defined there. Doing otherwise breaks reflection, as you have to
>>>add a special case for these class names to use the appropriate object
>>>from the types module instead. Thoughts? If it isn't desirable to have
>>>these names appear in __builtin__, perhaps their '__module__' should
>>>be changed to another module where they are defined?
>> Such as?  There really isn't a module where e.g. GeneratorType is
>> defined.
>> 
>
>    Seems perfectly reasonable and useful to add GeneratorType and
>    others to the types module.  I have code, for example, like this,
>    in a couple places:

Well, it's already there, but types.GeneratorType.__name__ is
'generator'... it could be changed to 'GeneratorType', I guess.

Cheers,
mwh

-- 
    . <- the point                                your article -> .
    |------------------------- a long way ------------------------|
                                        -- Cristophe Rhodes, ucam.chat
From gmccaughan at synaptics-uk.com  Tue Aug 10 16:21:55 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Tue Aug 10 16:22:28 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <16664.53010.746248.733772@montanaro.dyndns.org>
References: <41176596.5060209@hlabs.spb.ru> <411904F3.1080703@hlabs.spb.ru>
	<16664.53010.746248.733772@montanaro.dyndns.org>
Message-ID: <200408101521.55005.gmccaughan@synaptics-uk.com>

On Tuesday 2004-08-10 14:35, Skip Montanaro wrote:
> 
>     >> I think it would be worth considering all the numeric types at once
>     >> and come up with a reasonable hierarchy that includes float, complex
>     >> and decimal types as well.  (It's clear there is no "best" way to do
>     >> that.)
> 
>     Dmitry> Something like following?
> 
>     Dmitry> object
>     Dmitry>      numeric
>     Dmitry>          integer
>     Dmitry>              int
>     Dmitry>              long
>     Dmitry>          float
>     Dmitry>          complex
> 
> At some level it seems that float is a specialization of complex (w/ imag
> field == 0).  Also, it's not clear that float or complex don't have
> something other than a sibling relationship to integers.

Start by thinking about the mathematical types, which have
nice clear subset relationships between them:

    number
      complex
        real
          integer

Python doesn't have anything between "number" and "complex" and
there's no particular reason to think it ever will, so let's
simplify:

    number
      real
        integer

Now consider the different implementation types and fit them
in:

    number
      complex
      real
        (Decimal)
        float
        integer
          int
          long

I've reintroduced "complex" here as a name for the implementation
type, which in some sense should really be called complex_float
or something like that.

By way of clarification of the principles here, this is what
would happen to the hierarchy if Python were to acquire a
type for ratios of integers. I'll call that type "fraction"
and the underlying mathematical type "rational". Then we'd
get:

    number
      complex
      real
        (Decimal)
        float
        rational
          fraction
          integer
            int
            long

(You could argue that floats are rationals really, but that's
misleading even though it's true. The *structure* of arithmetic
operations on floats isn't the same as that on rationals.)

-- 
g

From fredrik at pythonware.com  Tue Aug 10 16:41:12 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Aug 10 16:41:22 2004
Subject: [Python-Dev] Re: Unifying Long Integers and Integers: baseint
References: <41176596.5060209@hlabs.spb.ru><2m7js83733.fsf@starship.python.net><16664.50649.835422.998574@montanaro.dyndns.org><411904F3.1080703@hlabs.spb.ru>
	<16664.53010.746248.733772@montanaro.dyndns.org>
Message-ID: <cfamq9$flm$1@sea.gmane.org>

Skip Montanaro wrote:

> I don't have the time to dig up the previous discussions.

why not start with the PEPs?

    http://www.python.org/peps/pep-0228.html
    http://www.python.org/peps/pep-0242.html
    http://www.python.org/peps/pep-0313.html
    (etc)

</F>




From pje at telecommunity.com  Tue Aug 10 17:03:52 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug 10 17:00:39 2004
Subject: [Python-Dev] More PEPs to mark as final
In-Reply-To: <20040810134023.GA13993@rogue.amk.ca>
Message-ID: <5.1.1.6.0.20040810105316.02a51230@mail.telecommunity.com>

At 09:40 AM 8/10/04 -0400, A.M. Kuchling wrote:
>The following PEPs all seem to be completed; can their authors (or
>someone) please confirm, and mark them as 'Final' if the PEPs are in
>fact completed?
>
>273  Import Modules from Zip Archives
>278  Universal Newline Support
>282  A Logging System
>302  New Import Hooks
>305  CSV File API

If you're asking whether these are implemented, at least PEP 302 isn't 
quite finished.  If I understand it correctly, there should be an 
'imp.get_loader', and its "phase 2" plan calls for putting "frozen, 
built-in and sys.path importer objects on sys.meta_path, allowing for some 
extra flexibility".

From guido at python.org  Tue Aug 10 17:01:17 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 10 17:01:27 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but
	aren't
In-Reply-To: Your message of "Tue, 10 Aug 2004 15:05:40 BST."
	<2macx31423.fsf@starship.python.net> 
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com> 
	<2macx31423.fsf@starship.python.net> 
Message-ID: <200408101501.i7AF1HZ25725@guido.python.org>

> Well, it's already there, but types.GeneratorType.__name__ is
> 'generator'... it could be changed to 'GeneratorType', I guess.

No, no, no!  The "new" convention for built-in type names is all
lowercase.  CamelCaseType is old and only used in the context of
types.py.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From skip at pobox.com  Tue Aug 10 17:03:03 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 10 17:03:22 2004
Subject: [Python-Dev] use of #pragma GCC system_header to suppress
	_XOPEN_SOURCE warnings
In-Reply-To: <4113E461.20600@v.loewis.de>
References: <16658.42673.542930.749915@montanaro.dyndns.org>
	<4112A968.2030401@v.loewis.de>
	<16659.57319.931941.158458@montanaro.dyndns.org>
	<4113E461.20600@v.loewis.de>
Message-ID: <16664.58279.489645.778551@montanaro.dyndns.org>


    Martin> I don't think it is a goal that Python generates no warnings if
    Martin> compiled with a C++ compiler (I don't think you actually *can*
    Martin> compile all of Python with a C++ compiler).

Our goal is less ambitious than that.  It is to build locally written
extension modules and other stuff (both python-related and otherwise)
without warnings, not necessarily all software we build for internal use.

    >> This suggests that Sun expects the application or its build tools to
    >> define _XOPEN_SOURCE, not the compiler or its header files.  That
    >> suggests a bug in g++.

    Martin> Indeed, and I believe this is a known bug. g++ needs to define
    Martin> _XOPEN_SOURCE so that some of Sun's system headers declare a
    Martin> number of prototypes which are then referred-to in inline
    Martin> functions of libstdc++ header files; ...

    Martin> This gives us a number of options:
    Martin> 1. On Solaris, define _XOPEN_SOURCE to the same value that g++
    Martin>    uses.  This should then cause no redefinition, but might
    Martin>    cause some functions go away. We would need an autoconf test
    Martin>    to find out what the value of _XOPEN_SOURCE is that g++ uses.

Of the three this seemed most plausible to me.  A first pass at a
configure.in mod is available at

    http://python.org/sf/1006629

Assigned to Martin.

Skip
From jim at zope.com  Tue Aug 10 17:19:04 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Aug 10 17:19:09 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__
	but	aren't
In-Reply-To: <24A02222-EA5E-11D8-AAEA-000D932927FE@opnet.com>
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<4117F7D5.3090907@zope.com>
	<24A02222-EA5E-11D8-AAEA-000D932927FE@opnet.com>
Message-ID: <4118E768.4050104@zope.com>

Nick Bastin wrote:
> 
> On Aug 9, 2004, at 6:16 PM, Jim Fulton wrote:
> 
>> James Y Knight wrote:
>>
>>> IMO classes ought to actually appear in __builtin__ if they claim 
>>> they are defined there. Doing otherwise breaks reflection, as you 
>>> have to add a special case for these class names to use the 
>>> appropriate object from the types module instead.  Thoughts?
>>
>>
>> I agree.
> 
> 
> I think this should stand for functions as well.  help() is a good 
> example.  It's a bit confusing for newbies that help.__module__ is 
> 'site', but 'site' isn't in their local namespace.

Yup.

Jim


-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From skip at pobox.com  Tue Aug 10 17:25:53 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 10 17:26:13 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: <F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org>
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
Message-ID: <16664.59649.633382.860519@montanaro.dyndns.org>


(still catching up...)

    Simon> I just wanted to say that I believe it should be allowed to
    Simon> decorate classes. There's not reason enough (reading the thread
    Simon> linked to by the PEP) to limit decorators this way. Like said
    Simon> several times in that thread, metaclasses are harder (to write
    Simon> and to understand) than decorators.

While it might be perceived as a limitation there's nothing preventing its
addition in a later version if warranted.  I believe most/all the various
proposals on the PythonDecorators wiki, not just the pie syntax, could be
easily adapted to class decoration.

Skip
From colanderman at gmail.com  Tue Aug 10 18:07:20 2004
From: colanderman at gmail.com (Chris King)
Date: Tue Aug 10 18:07:34 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
Message-ID: <875c7e0704081009076cd1cc77@mail.gmail.com>

On Tue, 10 Aug 2004 08:40:59 +1000, Delaney, Timothy C (Timothy)
<tdelaney@avaya.com> wrote:

> Not such a large outcry, but lots of us were concerned about "having to look inside the function for 'yield' to determine if it were a generator or a function" and were calling for a 'gen' or 'generator' keyword.

Building off of this:

It's not really that important to a user to know whether a function is
a generator or not: the difference (to a user) is the same difference
as whether a function returns a single item or a list.  A function's
return type isn't visibly declared at the top of the function, it
(along with everything else you need to know when calling a function)
is in the docstring.

Now, is it really that important to a user whether a method is a class
or static method?  They're all called the same way (so long as they're
bound), and most users couldn't care less how a function is
implemented.  If the distinction is really important to the user, then
they will presumably be prudent enough to check for this by looking
either just below the function name (or wherever decorators end up),
or by reading the docstring if the function author decided to document
that.

What's more, aside from classmethod and staticmethod, most other
decorations will be for the module's own use.  Placing such
insignificant (to the user!) details such as these above the
function's name and signature (what I think is the most important
aspect of a function!) just seems like we're asking for obfuscation.

A counter argument may be that decorators need to be readily visible
to module authors.  Well, to the author of a module, I'd say the code
is the most important aspect of a function -- module authors should be
worrying more about why their calculate_pi() funcion is returning
3.24895, rather than whether it's a class or static method.  Of
course, the error could have been caused by a misdeclaration of method
type, but I'm sure that would be the cause much less often than, say,
an off-by-one error in the code.

+1 for decorators anywhere after the function name and signature, even
if it involves pies.
From foom at fuhm.net  Tue Aug 10 18:17:14 2004
From: foom at fuhm.net (James Y Knight)
Date: Tue Aug 10 18:17:16 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but
	aren't
In-Reply-To: <200408101501.i7AF1HZ25725@guido.python.org>
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com>
	<2macx31423.fsf@starship.python.net>
	<200408101501.i7AF1HZ25725@guido.python.org>
Message-ID: <BE05E963-EAE8-11D8-9899-000A95A50FB2@fuhm.net>

On Aug 10, 2004, at 11:01 AM, Guido van Rossum wrote:
> No, no, no!  The "new" convention for built-in type names is all
> lowercase.  CamelCaseType is old and only used in the context of
> types.py.

Sooo should (for 'generator' in objects that claim to be in 
__builtins__ but aren't),
1) 'generator' be added to __builtins__
2) 'generator' be added to types.py and its __module__ be set to 'types'
3) 'generator' be added to <newmodule>.py and its __module__ be set to 
'<newmodule>' (and a name for the module chosen)

James

From pje at telecommunity.com  Tue Aug 10 19:21:42 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug 10 19:17:26 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <875c7e0704081009076cd1cc77@mail.gmail.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
	<338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
Message-ID: <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com>

At 12:07 PM 8/10/04 -0400, Chris King wrote:

>Now, is it really that important to a user whether a method is a class
>or static method?  They're all called the same way (so long as they're
>bound), and most users couldn't care less how a function is
>implemented.  If the distinction is really important to the user, then
>they will presumably be prudent enough to check for this by looking
>either just below the function name (or wherever decorators end up),
>or by reading the docstring if the function author decided to document
>that.

This doesn't apply to other decorators.  For example, some decorators turn 
a function into a property.  Others change the (effective) call signature.

Indeed, staticmethod changes the call signature, since there is no special 
first argument.

In frameworks where methods can be wrapped in transactions, lock 
synchronization, security checks, remote communication, etc., these are all 
decorations that are potentially part of the interface and important for 
the reader to know, even if they don't look at the method body (apart from 
the doc string).

From bob at redivi.com  Tue Aug 10 19:34:57 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Aug 10 19:35:29 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
	<338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
	<5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com>
Message-ID: <991C2154-EAF3-11D8-91E7-000A95686CD8@redivi.com>


On Aug 10, 2004, at 1:21 PM, Phillip J. Eby wrote:

> At 12:07 PM 8/10/04 -0400, Chris King wrote:
>
>> Now, is it really that important to a user whether a method is a class
>> or static method?  They're all called the same way (so long as they're
>> bound), and most users couldn't care less how a function is
>> implemented.  If the distinction is really important to the user, then
>> they will presumably be prudent enough to check for this by looking
>> either just below the function name (or wherever decorators end up),
>> or by reading the docstring if the function author decided to document
>> that.
>
> This doesn't apply to other decorators.  For example, some decorators 
> turn a function into a property.  Others change the (effective) call 
> signature.
>
> Indeed, staticmethod changes the call signature, since there is no 
> special first argument.
>
> In frameworks where methods can be wrapped in transactions, lock 
> synchronization, security checks, remote communication, etc., these 
> are all decorations that are potentially part of the interface and 
> important for the reader to know, even if they don't look at the 
> method body (apart from the doc string).

Also, most people that call static methods or class methods probably 
aren't calling them bound to an *instance*.  At least I know that when 
I use classmethods, it's usually to *create* instances.

-bob
From colanderman at gmail.com  Tue Aug 10 20:03:33 2004
From: colanderman at gmail.com (Chris King)
Date: Tue Aug 10 20:03:36 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
	<338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
	<5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com>
Message-ID: <875c7e070408101103710836f6@mail.gmail.com>

On Tue, 10 Aug 2004 13:21:42 -0400, Phillip J. Eby
<pje@telecommunity.com> wrote:

> This doesn't apply to other decorators.  For example, some decorators turn
> a function into a property.

Ew.  If you did that, you'd be declaring get properties one way (with
decorators) and get/set properties another (using property()).

> Others change the (effective) call signature.

But will a casual observer be able to see exactly how a given
decorator is mutating the call signature?  If such a function is meant
for external use, its effective signature should be in the docstring,
not unlike those of C extension functions.  On the other hand, the
developer will know which functions are using such a decorator, or
will know to look for whether such a decorator is being used in a
function.

> Indeed, staticmethod changes the call signature, since there is no special
> first argument.

No, static methods are called with the same signature you see defined,
just like a normal function.  Not seeing 'self' as the first argument
should be enough to tip you off that this isn't a bound method.

Concerning users, whether a method is a static or class method should
be indicated in the method's docstring.  I should hope any user reads
a function's docstring for usage information before using it.

> In frameworks where methods can be wrapped in transactions, lock
> synchronization, security checks, remote communication, etc., these are all
> decorations that are potentially part of the interface and important for
> the reader to know, even if they don't look at the method body (apart from
> the doc string).

Assuming they look at the docstring, perhaps the method author would
be kind enough to specify "This function performs an exclusively
locked record transaction.", or, better yet, the decorator itself
could extend the docstring with its own "record-locking method"
indicator.  Making decorators more visible in the source doesn't make
then more visible in the docstring.

If the user can't be bothered to read a three-line description that is
a docstring, I doubt they will be bothered to read (and possibly
decipher) the information contained in decorators, no matter where
they're placed.
From dave at boost-consulting.com  Tue Aug 10 20:12:36 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Tue Aug 10 20:13:15 2004
Subject: [Python-Dev] Re: pep 318, Decorators for Functions,
	Methods and Classes
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org>
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
	<16664.59649.633382.860519@montanaro.dyndns.org>
Message-ID: <ur7qex3or.fsf@boost-consulting.com>

Skip Montanaro <skip@pobox.com> writes:

> (still catching up...)
>
>     Simon> I just wanted to say that I believe it should be allowed to
>     Simon> decorate classes. There's not reason enough (reading the thread
>     Simon> linked to by the PEP) to limit decorators this way. Like said
>     Simon> several times in that thread, metaclasses are harder (to write
>     Simon> and to understand) than decorators.
>
> While it might be perceived as a limitation there's nothing preventing its
> addition in a later version if warranted.  I believe most/all the various
> proposals on the PythonDecorators wiki, not just the pie syntax, could be
> easily adapted to class decoration.

I was going for replacing the module's dict with a dict subclass, and
hooking __setitem__ it as an alternative to using settrace to
implement this.  It's not allowed, though :(.  We also need a way to
say, "replace the dict into which new definitions are falling" so we
can implement decorators inside of class definitions.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From mark at hahnca.com  Tue Aug 10 20:56:05 2004
From: mark at hahnca.com (Mark Hahn)
Date: Tue Aug 10 20:56:19 2004
Subject: [Python-Dev] Prothon on CPython intrepreter? [PROTHON]
Message-ID: <000a01c47f0b$b3cf4690$0b01a8c0@mark>

I got such excellent help from Python-dev on the Prothon decimal question
that I've come back for a really big question. :-)

I've announced recently that Prothon was switching from a homegrown
(CProthon) interpreter to a .Net implementation like IronPython.  The logic
was that Prothon would get a library and a community much faster.  What it
got very quickly was a backlash from people, including Guido, who wanted
Prothon to stay away from the dark side and remain pure like CPython.

I've said in the past that Prothon couldn't use the CPython interpreter.  I
made this determination late last year when I first started the Prothon
project because CPython was incompatible with my goals of pre-emptive
threads with no GIL, object locking, etc.  

Now I find myself compromising my goals for .Net in that I cannot implement
object-locking and lightweight threads.  I am also going to be mixing real
classes with my prototypes, although this is something I have determined I
want to do for other reasons (it's a long story).

I'm sure you've heard the joke that ends with "we've established you're a
whore, now we are just haggling over the price".  If I can compromise
Prothon for .Net, why can't I compromise it some more for CPython?  So I am
thinking of doing a Prothon implementation in CPython before doing the .Net
version.  I will get a library and community fast like I would have with
.Net.  

This CPython->.Net->CProthon roadmap will get Prothon out quickly and slowly
easy my wild ideas for the interpreter into Prothon.  It will also make
CPython a multiple language VM for the first time.

So, I'll finally get to some questions:

1) Guido: Does this address the reason you asked me not to switch to .Net,
or did you specifically want me to do CProthon?  I think this will still be
a good sandbox for Python.  It should be quite interesting to see how far I
can stretch the Python interpreter.  Who knows how many of my CProthon
engine features I might be able to add to CPython.

2) Lenard Lindstrom said he thought I could "Add some extra byte code
instructions to handle Prothon's distinct scoping rules and implement
prototypes using PyType_Type". He also said "All attribute accesses would be
through a special attribute handler method".  I don't know the Python
interpreter code yet, but does this sound workable to everyone?  It doesn't
have to be Python compatible.

3) Could Tim Peter's welcome offer to Prothon here on Python-dev be
stretched to include the many questions I would have when modifying CPython?
In other words would this project be welcomed as a sandbox for Python or
would it be an unwelcome drain on your limited resources?

If any of you want to see background material on Prothon, my VanPy talk is
at: http://prothon.org/pub/prothon/docs/vanpy/ProthonTalk.htm.
The section titled "engine" is about the homegrown engine and wouldn't be
applicable to anything on .Net or CPython.

Thanks for you attention.  I would love to find a solution that lets me work
with the wonderful Python community.

From jjl at pobox.com  Tue Aug 10 21:11:34 2004
From: jjl at pobox.com (John J Lee)
Date: Tue Aug 10 21:11:38 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae04080918481e1f8053@mail.gmail.com>
References: <007c01c47da7$92b71920$191fc797@oemcomputer>
	<1f7befae04080918481e1f8053@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0408102008220.535@alice>

On Mon, 9 Aug 2004, Tim Peters wrote:
[...]
> Has anyone on Linux tried this yet?

On Linux, I get precisely the same output from Python CVS that you posted,
at least as far as 148 (didn't check any further):

[...]
118 ok
119 ok
120 ok
121 ok
122 ok
123 exceptions.RuntimeError maximum recursion depth exceeded
124 exceptions.RuntimeError maximum recursion depth exceeded
125 exceptions.RuntimeError maximum recursion depth exceeded
126 exceptions.KeyError 307
127 exceptions.RuntimeError maximum recursion depth exceeded
128 exceptions.KeyError 306
129 exceptions.RuntimeError maximum recursion depth exceeded
130 exceptions.KeyError 305
131 exceptions.RuntimeError maximum recursion depth exceeded
132 exceptions.KeyError 304
133 exceptions.RuntimeError maximum recursion depth exceeded
134 exceptions.KeyError 303
135 exceptions.KeyError 302
136 exceptions.RuntimeError maximum recursion depth exceeded
137 exceptions.KeyError 301
138 exceptions.RuntimeError maximum recursion depth exceeded
139 exceptions.KeyError 300
140 exceptions.RuntimeError maximum recursion depth exceeded
141 exceptions.RuntimeError maximum recursion depth exceeded
142 exceptions.KeyError 299
143 exceptions.RuntimeError maximum recursion depth exceeded
144 exceptions.KeyError 297
145 exceptions.KeyError 296
146 exceptions.RuntimeError maximum recursion depth exceeded
147 exceptions.KeyError 295
148 exceptions.RuntimeError maximum recursion depth exceeded
[...]


John
From arigo at tunes.org  Tue Aug 10 21:16:38 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue Aug 10 21:21:15 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae04080918481e1f8053@mail.gmail.com>
References: <007c01c47da7$92b71920$191fc797@oemcomputer>
	<1f7befae04080918481e1f8053@mail.gmail.com>
Message-ID: <20040810191638.GA24820@vicky.ecs.soton.ac.uk>

Hello Tim,

On Mon, Aug 09, 2004 at 09:48:37PM -0400, Tim Peters wrote:
> Has anyone on Linux tried this yet?

I get this: in a release build,

...
120 ok
121 ok
122 ok
123 exceptions.RuntimeError maximum recursion depth exceeded
124 exceptions.RuntimeError maximum recursion depth exceeded
125 exceptions.RuntimeError maximum recursion depth exceeded
126 exceptions.RuntimeError maximum recursion depth exceeded
127 exceptions.RuntimeError maximum recursion depth exceeded
128 exceptions.RuntimeError maximum recursion depth exceeded
...

In a release 21st of March version with the fresh generator expression patch
applied,

...
120 ok
121 ok
122 ok
123 exceptions.RuntimeError maximum recursion depth exceeded
124 exceptions.RuntimeError maximum recursion depth exceeded
125 exceptions.RuntimeError maximum recursion depth exceeded
126 exceptions.KeyError 305
127 exceptions.RuntimeError maximum recursion depth exceeded
128 exceptions.KeyError 304
129 exceptions.RuntimeError maximum recursion depth exceeded
...

In a recent debug build,

...
120 ok
121 ok
122 ok
123 exceptions.RuntimeError maximum recursion depth exceeded
124 exceptions.RuntimeError maximum recursion depth exceeded
125 exceptions.RuntimeError maximum recursion depth exceeded
126 exceptions.KeyError 307
127 exceptions.RuntimeError maximum recursion depth exceeded
128 exceptions.KeyError 306
129 exceptions.RuntimeError maximum recursion depth exceeded
130 exceptions.KeyError 305
131 exceptions.RuntimeError maximum recursion depth exceeded
...

So there is no particular problem running debug builds under Linux, but still
why we get this behavior is anybody's guess.

Also note that even though the compiler package may only have changed a bit,
the parser C module definitely has, e.g. when genexprs and decorators were
introduced.  I wonder what effect this has on the depth of concrete syntax
trees.  I also wonder if the parser module tries to be safe again C stack
overflow when building hugely nested data structures.


Armin
From mwh at python.net  Tue Aug 10 21:00:14 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug 10 21:22:39 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae0408092348583f0953@mail.gmail.com> (Tim Peters's
	message of "Tue, 10 Aug 2004 02:48:15 -0400")
References: <000001c47e88$81ef1d40$5635c797@oemcomputer>
	<1f7befae0408092348583f0953@mail.gmail.com>
Message-ID: <2mvffqzum9.fsf@starship.python.net>

Tim Peters <tim.peters@gmail.com> writes:

> Well, this gets nasty.

You're certainly not wrong about that!

[snip Windows details]

> That's
>
> 	if (Py_EnterRecursiveCall(" in cmp"))
> 		return NULL;
>
> in PyObject_RichCompare.  That's just trying to compare two ints.
>
> So NULL gets returned to PyObject_RichCompareBool, which in turn
> returns -1 to lookdict.  AAAARGHGH!  lookdict "isn't allowed" to raise
> exceptions, so it does a PyErr_Clear(), goes on to futilely chase the
> entire dict looking for another match on the hash code, and we've
> effectively turned a MemoryError into a KeyError.  I expect that
> explains a lot about what we see in the release-build runs.

This part happens on Linux, too, it seems.  Running your script:

114 ok
115 ok
116 ok
117 ok
118 ok
119 ok
120 ok
121 ok
122 ok
123 exceptions.RuntimeError maximum recursion depth exceeded
124 exceptions.RuntimeError maximum recursion depth exceeded
125 exceptions.RuntimeError maximum recursion depth exceeded
126 exceptions.KeyError 307
127 exceptions.RuntimeError maximum recursion depth exceeded
128 exceptions.KeyError 306
129 exceptions.RuntimeError maximum recursion depth exceeded
130 exceptions.KeyError 305
131 exceptions.RuntimeError maximum recursion depth exceeded
132 exceptions.KeyError 304
133 exceptions.RuntimeError maximum recursion depth exceeded
134 exceptions.KeyError 303
135 exceptions.KeyError 302
136 exceptions.RuntimeError maximum recursion depth exceeded
137 exceptions.KeyError 301

This is a debug build, so we're not abusing the stack quite so much on
linux as apparently seems to happen in debug builds on windows.

This isn't going to be something we can pin on the new Windows tools,
is it?  It doesn't seem like a feature that a debug build gets a tiny
stack, but, well...

> The question is what we did since 2.3.4 that apparently increases our
> stack demands, and grossly increases them in a debug build(!).
               ^
on windows; I don't see anything so scary on Linux.  

Hmm, maybe that's a lie, I don't see the crazy KeyErrors with 2.3.

> Could be that the compile package is more heavily recursive now too
> (no idea).

Really don't think so.

> test_parser.py in 2,3.4 contained the same deeply nested tuples, so
> that's not what changed.

Something that *has* changed, however, is that we lost the code that
compared objects recursively and now PyObject_RichCompare uses the
more general Py_EnterRecursiveCall mechanism and not its own counter.
So now we're grossly more likely to hit one of these faux KeyErrors in
2.3 than 2.4 (you'd need to invoke a Python __eq__ method or so).

Would it make more sense for PyObject_RichCompare to still use a
private counter but raise RuntimeError if it is exceeded instead of
trying to be clever?  That would make this problem /less/ likely,
though obviously still possible (functions that claim they 'can't
fail' are lying).

Running Misc/find_recursionlimit.py with a debug build yields ...
well, I gave up and killed it at 22000 (hmm, seems the default 'ulimit
-s' is "unlimited"... I'm sure this hasn't always had this effect with
a threaded build).

Cheers,
mwh

-- 
  > Emacs is a fashion statement.
  No, Gnus is a fashion statement.  Emacs is clothing.  Everyone 
  else is running around naked.
        -- Karl Kleinpaste & Jonadab the Unsightly One, gnu.emacs.gnus
From walter at livinglogic.de  Tue Aug 10 21:24:20 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Aug 10 21:24:23 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4107F0E8.3010005@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>
	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>
	<4107DA6E.30500@livinglogic.de> <4107F0E8.3010005@egenix.com>
Message-ID: <411920E4.9020404@livinglogic.de>

OK, here a my current thoughts on the codec problem:

The optimal solution (ignoring backwards compatibility)
would look like this: codecs.lookup() would return the
following stuff (this could be done by replacing the
4 entry tuple with a real object):

* decode: The stateless decoding function
* encode: The stateless encocing function
* chunkdecoder: The stateful chunk decoder
* chunkencoder: The stateful chunk encoder
* streamreader: The stateful stream decoder
* streamwriter: The stateful stream encoder

The functions and classes look like this:


Stateless decoder:
decode(input, errors='strict'):
     Function that decodes the (str) input object and returns
     a (unicode) output object. The decoder must decode the
     complete input without any remaining undecoded bytes.

Stateless encoder:
encode(input, errors='strict'):
     Function that encodes the complete (unicode) input object and
     returns a (str) output object.

Stateful chunk decoder:
chunkdecoder(errors='strict'):
     A factory function that returns a stateful decoder with the
     following method:

     decode(input, final=False):
         Decodes a chunk of input and return the decoded unicode
         object. This method can be called multiple times and
         the state of the decoder will be kept between calls.
         This includes trailing incomplete byte sequences
         that will be retained until the next call to decode().
         When the argument final is true, this is the last call
         to decode() and trailing incomplete byte sequences will
         not be retained, but a UnicodeError will be raised.

Stateful chunk encoder:
chunkencoder(errors='strict'):
     A factory function that returns a stateful encoder
     with the following method:
     encode(input, final=False):
         Encodes a chunk of input and returns the encoded
         str object. When final is true this is the last
         call to encode().

Stateful stream decoder:
streamreader(stream, errors='strict'):
     A factory function that returns a stateful decoder
     for reading from the byte stream stream, with the
     following methods:

     read(size=-1, chars=-1, final=False):
         Read unicode characters from the stream. When data
         is read from the stream it should be done in chunks of
         size bytes. If size == -1 all the remaining data
         from the stream is read. chars specifies the number
         of characters to read from the stream. read() may return
         less then chars characters if there's not enough data
         available in the byte stream. If chars == -1 as much
         characters are read as are available in the stream.
         Transient errors are ignored and trailing incomplete
         byte sequence are retained when final is false. Otherwise
         a UnicodeError is raised in the case of incomplete byte
         sequences.
     readline(size=-1):
             ...
     next():
             ...
     __iter__():
             ...

Stateful stream encoder:
streamwriter(stream, errors='strict'):
     A factory function that returns a stateful encoder
     for writing unicode data to the byte stream stream,
     with the following methods:

     write(data, final=False):
         Encodes the unicode object data and writes it
         to the stream. If final is true this is the last
         call to write().
     writelines(data):
         ...


I know that this is quite a departure from the current API, and
I'm not sure if we can get all of the functionality without
sacrificing backwards compatibility.

I don't particularly care about the "bytes consumed" return value
from the stateless codec. The codec should always have returned only
the encoded/decoded object, but I guess fixing this would break too
much code. And users who are only interested in the stateless
functionality will probably use unicode.encode/str.decode anyway.

For the stateful API it would be possible to combine the chunk and
stream decoder/encode into one class with the following methods
(for the decoder):

     __init__(stream, errors='strict'):
         Like the current StreamReader constructor, but stream may be
         None, if only the chunk API is used.
     decode(input, final=False):
         Like the current StreamReader (i.e. it returns a (unicode, int)
         tuple.) This does not keep the remaining bytes in a buffer.
         This is the job of the caller.
     feed(input, final=False):
         Decodes input and returns a decoded unicode object. This method
         calls decode() internally and manages the byte buffer.
     read(size=-1, chars=-1, final=False):
     readline(size=-1):
     next():
     __iter__():
         See above.

As before implementers of decoders only need to implement decode().

To be able to support the final argument the decoding functions
in _codecsmodule.c could get an additional argument. With this
they could be used for the stateless codecs too and we can reduce
the number of functions again.

Unfortunately adding the final argument breaks all of the current
codecs, but dropping the final argument requires one of two
changes:
1) When the input stream is exhausted, the bytes read are parsed
    as if final=True. That's the way the CJK codecs currently
    handle it, but unfortunately this doesn't work with the feed
    decoder.
2) Simply ignore any remaing undecoded bytes at the end of the
    stream.

If we really have to drop the final argument, I'd prefer 2).

I've uploaded a second version of the patch. It implements
the final argument, adds the feed() method to StreamReader and
again merges the duplicate decoding functions in the codecs
module. Note that the patch isn't really finished (the final
argument isn't completely supported in the encoders and the
CJK and escape codecs are unchanged), but it should be sufficient
as a base for discussion.

Bye,
    Walter D?rwald


From fdrake at acm.org  Tue Aug 10 21:25:57 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Aug 10 21:26:09 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <20040810191638.GA24820@vicky.ecs.soton.ac.uk>
References: <007c01c47da7$92b71920$191fc797@oemcomputer>
	<1f7befae04080918481e1f8053@mail.gmail.com>
	<20040810191638.GA24820@vicky.ecs.soton.ac.uk>
Message-ID: <200408101525.57934.fdrake@acm.org>

On Tuesday 10 August 2004 03:16 pm, Armin Rigo wrote:
 > syntax trees.  I also wonder if the parser module tries to be safe again C
 > stack overflow when building hugely nested data structures.

No, it really doesn't, though it probably should be a little careful.

Switching to a less recursive approach may be a good idea, but not sure what 
would cause the most pain in implementation.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From arigo at tunes.org  Tue Aug 10 21:23:03 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue Aug 10 21:27:40 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <20040810191638.GA24820@vicky.ecs.soton.ac.uk>
References: <007c01c47da7$92b71920$191fc797@oemcomputer>
	<1f7befae04080918481e1f8053@mail.gmail.com>
	<20040810191638.GA24820@vicky.ecs.soton.ac.uk>
Message-ID: <20040810192303.GB24820@vicky.ecs.soton.ac.uk>

Hello again,

On Tue, Aug 10, 2004 at 08:16:38PM +0100, Armin Rigo wrote:
> I get this: in a release build,
> 
> ...
> 120 ok
> 121 ok
> 122 ok
> 123 exceptions.RuntimeError maximum recursion depth exceeded
> 124 exceptions.RuntimeError maximum recursion depth exceeded
> 125 exceptions.RuntimeError maximum recursion depth exceeded
> 126 exceptions.RuntimeError maximum recursion depth exceeded
> 127 exceptions.RuntimeError maximum recursion depth exceeded
> 128 exceptions.RuntimeError maximum recursion depth exceeded
> ...

Sorry, my mistake.  This was after I removed from object.c the two
Py_Enter/LeaveRecursiveCall(" in cmp").  Not suprizingly, I then only get
"regular" recursion errors.

A pure CVS version gets me the same results as you and John.


Armin
From jhylton at gmail.com  Tue Aug 10 21:31:39 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Tue Aug 10 21:31:42 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <20040810191638.GA24820@vicky.ecs.soton.ac.uk>
References: <007c01c47da7$92b71920$191fc797@oemcomputer>
	<1f7befae04080918481e1f8053@mail.gmail.com>
	<20040810191638.GA24820@vicky.ecs.soton.ac.uk>
Message-ID: <e8bf7a530408101231421838e1@mail.gmail.com>

On Tue, 10 Aug 2004 20:16:38 +0100, Armin Rigo <arigo@tunes.org> wrote:
> Also note that even though the compiler package may only have changed a bit,
> the parser C module definitely has, e.g. when genexprs and decorators were
> introduced.  I wonder what effect this has on the depth of concrete syntax
> trees.  I also wonder if the parser module tries to be safe again C stack
> overflow when building hugely nested data structures.

I don't think the compiler package has changed much at all, but it
consumes the deeply nested concrete syntax trees created by the parser
module.  The compiler.transformer module uses recursion a lot. 
Several releases ago I added some hacks to reduce the number of frames
(basically did tail call optimization by hand in a few spots).  It may
be that those hacks are no longer sufficient.

where's-that-ast-branch-when-you-need-it-ly y'rs,
Jeremy
From tim.peters at gmail.com  Tue Aug 10 21:58:22 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 10 21:58:24 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <2mvffqzum9.fsf@starship.python.net>
References: <000001c47e88$81ef1d40$5635c797@oemcomputer>
	<1f7befae0408092348583f0953@mail.gmail.com>
	<2mvffqzum9.fsf@starship.python.net>
Message-ID: <1f7befae0408101258431039a1@mail.gmail.com>

FYI, "the stack" under an MS compiler defaults to 1MB.  It's easy to
experiment with different stack allocations from the cmdline, like
this example:

editbin /stack:2000000 python.exe

The ~2MB there is enough that test_compiler runs to completion
normally under a release-build Python, and under a debug-build Python.

In a debug-build Windows Python, each MB allocated to the stack allows
about 400 recursion levels (according to Misc/find_recursionlimit.py),
and running compiler.compile() directly on test_parser.py requires
more than 800 levels, so a 2MB stack must be near the insanity limit
for the debug build on this test.

So the easiest way to get the -uall test suite running on Windows
again is to fiddle linker flags to boost the stack size.  I'm not sure
real apps need it.  If they do and we don't "fix it", what happens is
that the process mysteriously just vanishes (no error msg, nothing in
the system or application event logs either) with a 128 exit code. 
That's a disaster.  Alas, I don't have a theory for how we could be
managing to screw up the OS so badly -- but then I guess you really
can't spell Windows XP without Windows <0.9 wink>.
From python at rcn.com  Tue Aug 10 22:18:17 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug 10 22:18:30 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae0408101258431039a1@mail.gmail.com>
Message-ID: <008301c47f17$2c6af940$e622c797@oemcomputer>

Another datapoint:  the problem existed prior to 1/15/2004.  That
eliminates many possibilities.


Raymond

From pje at telecommunity.com  Tue Aug 10 22:31:16 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug 10 22:27:01 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <875c7e070408101103710836f6@mail.gmail.com>
References: <5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com>
	<338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
	<338366A6D2E2CA4C9DAEAE652E12A1DE01C8CA89@au3010avexu1.global.avaya.com>
	<5.1.1.6.0.20040810131834.01ede700@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040810162646.05242aa0@mail.telecommunity.com>

At 02:03 PM 8/10/04 -0400, Chris King wrote:
>On Tue, 10 Aug 2004 13:21:42 -0400, Phillip J. Eby
><pje@telecommunity.com> wrote:
>
> > This doesn't apply to other decorators.  For example, some decorators turn
> > a function into a property.
>
>Ew.  If you did that, you'd be declaring get properties one way (with
>decorators) and get/set properties another (using property()).

Google "property_getter", and you'll find an implementation for this:

     @property_getter
     def foo(self):
         return self.__foo

     @property_setter
     def foo(self,value):
         self.__foo = value

     @property_deleter
     def foo(self):
         del self.__foo

From marktrussell at btopenworld.com  Tue Aug 10 23:07:20 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Tue Aug 10 23:07:19 2004
Subject: [Python-Dev] Decorator order implemented backwards?
In-Reply-To: <41185844.80906@interlink.com.au>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au>
Message-ID: <1092172040.1681.9.camel@localhost>

On Tue, 2004-08-10 at 06:08, Anthony Baxter wrote:
> Mark, is it possible that this changed between the first and
> final versions of the decorator patch? The SF report doesn't
> list any of the older versions...

No, it has always been this way round.  In fact test_order was inherited
from Guido's original version of test_decorators.py (from patch #926860)
where it read:

    def test_order(self):
        class C(object):
            [funcattrs(abc=1), staticmethod]
            def foo(): return 42
        # This wouldn't work if staticmethod was called first
        self.assertEqual(C.foo(), 42)
        self.assertEqual(C().foo(), 42)

(i.e. identical to the current version except for the change in
syntax).  In fact I relied on the fact that this test passed to convince
me I had the order right!  But I should have spotted the inconsistency
between that and the documentation that I wrote for the reference
manual.

I'll do a patch to fix the order and the corresponding tests.

While I'm at it, do we want to drop support for multiple decorators on a
single line?  Nobody has spoken up for it, and in fact forcing
one-per-line would simplify the grammar (as well as making it easier for
toy parsers to find decorators).

Mark

From bob at redivi.com  Tue Aug 10 23:24:54 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Aug 10 23:25:00 2004
Subject: [Python-Dev] Drop support for multiple decorators per line? (was:
	Decorator order implemented backwards?)
In-Reply-To: <1092172040.1681.9.camel@localhost>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost>
Message-ID: <B90C2588-EB13-11D8-8849-000A95686CD8@redivi.com>

On Aug 10, 2004, at 5:07 PM, Mark Russell wrote:

> While I'm at it, do we want to drop support for multiple decorators on 
> a
> single line?  Nobody has spoken up for it, and in fact forcing
> one-per-line would simplify the grammar (as well as making it easier 
> for
> toy parsers to find decorators).

+1

-bob
From andrew at indranet.co.nz  Tue Aug 10 23:30:42 2004
From: andrew at indranet.co.nz (Andrew McGregor)
Date: Tue Aug 10 23:30:59 2004
Subject: [Python-Dev] Re: Decorators: vertical bar syntax
In-Reply-To: <Pine.LNX.4.58.0408081410420.6199@rnd.onego.ru>
References: <004501c47cdb$355743d0$6602a8c0@arkdesktop>
	<Pine.LNX.4.58.0408081410420.6199@rnd.onego.ru>
Message-ID: <E4ED8851A18BF8604B2E586A@[192.168.1.247]>


--On Sunday, 8 August 2004 2:16 p.m. +0400 Roman Suzi <rnd@onego.ru> wrote:

> I just anted to rewrite those "real-life-looking" examples with my
> proposed '%' syntax:
>
> def foo() % (classmethod, accepts(int,int), returns(float),):
>
> or formatted:
>
> def foo() % (
>    classmethod,
>    accepts(int,int),
>    returns(float),):
>
> Or it can be done the this way:
>
> def foo():
>    ...
>
> foo %= (classmethod, accepts(int,int), returns(float),)
>
> The operation could be called "decorative apply", if the LHS
> has __call__ attribute. Just a new form of % operation with
> special syntax for "def" operator.

(unlurking for a moment...)

Now that actually looks like Python!  Great suggestion.

---------
Andrew McGregor
Director, Scientific Advisor
IndraNet Technologies Ltd
http://www.indranet-technologies.com/

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GS/E/B/PA/SS d+(++) s+:+ a C++$ ULS++++ !P+++(---)$ L++++$ E++ W++ !N
w(+++) !O() M++ V--() Y+$ PGP+ t- !5? X- !R !tv@ b++(++++) DI++ D+++@ G
e+++ h(*)@ r%
------END GEEK CODE BLOCK------
From barry at python.org  Wed Aug 11 00:00:56 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 11 00:00:45 2004
Subject: [Python-Dev] Drop support for multiple decorators per line?
	(was: Decorator order implemented backwards?)
In-Reply-To: <B90C2588-EB13-11D8-8849-000A95686CD8@redivi.com>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost>
	<B90C2588-EB13-11D8-8849-000A95686CD8@redivi.com>
Message-ID: <1092175256.21841.20.camel@localhost>

On Tue, 2004-08-10 at 17:24, Bob Ippolito wrote:
> On Aug 10, 2004, at 5:07 PM, Mark Russell wrote:
> 
> > While I'm at it, do we want to drop support for multiple decorators on 
> > a
> > single line?  Nobody has spoken up for it, and in fact forcing
> > one-per-line would simplify the grammar (as well as making it easier 
> > for
> > toy parsers to find decorators).
> 
> +1

+1

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040810/2fafa2c2/attachment.pgp
From andrew at indranet.co.nz  Wed Aug 11 00:19:20 2004
From: andrew at indranet.co.nz (Andrew McGregor)
Date: Wed Aug 11 00:20:33 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <16664.50665.170916.600964@montanaro.dyndns.org>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>	<ceuk0e$rut$1@sea.gmane.org>
	<1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>	<cf0gd1$smv$1@sea.gmane.org>
	<1091818207.7529.34.camel@localhost>	<4113D5E1.6040006@interlink.com.au>
	<41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de>
	<16664.50665.170916.600964@montanaro.dyndns.org>
Message-ID: <DAE618D9A063D6BD6E7D58DC@[192.168.1.247]>



--On Tuesday, 10 August 2004 7:56 a.m. -0500 Skip Montanaro 
<skip@pobox.com> wrote:


> I don't think anybody's come up with a candidate keyword that reads well
> in most/all situations.  Consider:
>
>     make accepts(int, (int, float))
>     make returns(int)
>     def foo(arg1, arg2):
>         return arg1 * arg2


     make accepts(int, (int, float)):
         make returns(int):
             def foo(arg1, arg2):
                 return arg1 * arg2

looks more like Python to me.  And I know it's nested, but it reads better 
IMO.

Aren't we really trying to recreate def as a multi-line lambda that binds 
when it's in a context that is not expecting a function argument in block 
form?  One way of looking at def or any of the other keywords that 
introduce blocks is as operators that take block(s) as argument(s).

---------
Andrew McGregor
Director, Scientific Advisor
IndraNet Technologies Ltd
http://www.indranet-technologies.com/

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GS/E/B/PA/SS d+(++) s+:+ a C++$ ULS++++ !P+++(---)$ L++++$ E++ W++ !N
w(+++) !O() M++ V--() Y+$ PGP+ t- !5? X- !R !tv@ b++(++++) DI++ D+++@ G
e+++ h(*)@ r%
------END GEEK CODE BLOCK------
From barry at python.org  Wed Aug 11 00:42:51 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 11 00:42:43 2004
Subject: [Python-Dev] Update PEP 292
Message-ID: <1092177771.21839.54.camel@localhost>

Raymond and I had a chance to talk about PEP 292 on bug day, and we've
exchanged a few private emails and implementations.  I think we've
finally knocked everything into shape for inclusion in Python 2.4's next
alpha.

In the sandbox you will find the updated implementation, along with
updated tests and documentation (libstring.tex).

The updated PEP is at:

http://www.python.org/peps/pep-0292.html

A summary of the most important changes:

      * Renamed 'dstring' to 'template', and removed the other ?string
        classes
      * Added a 'safe_template' subclass for non KeyError-throwing
        substitutions
      * Got rid of safedicts; Raymond figured out the trick to retain
        optional braces in the safe substitution case without needing
        that class.
      * Renamed the 'cre' class attribute for subclasses to 'pattern'

Now for the controversial bit <wink>.

I still think it's worthwhile to turn the string module into a package. 
The sandbox has an implementation of this that is completely backward
compatible.  I like it because it segregates the deprecated functions
into a separate sub-module and makes the whole shebang easier to
maintain.  It also provides an existing namespace for future expansion,
as has been discussed before.

Please note: PEP 292 is not dependent on the string module -> package
reorganization, so it isn't documented in that PEP.  I don't think it's
worth writing a PEP just for this library reorganization.  Take a look
at the sandbox if you're interested.

http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/string/

As far as this aspect of the changes go, I would like BDFL pronouncement
on it.  I'll abide by Guido's decision here.  If he nixes the re-org,
then I propose to put the template and safe_template classes in
Lib/string.py.  The libstring.tex changes will still be relevant.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040810/15fbe4b9/attachment.pgp
From john at grulic.org.ar  Wed Aug 11 00:46:44 2004
From: john at grulic.org.ar (John Lenton)
Date: Wed Aug 11 00:47:09 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <200408090105.i7915mfS005734@cosc353.cosc.canterbury.ac.nz>
References: <200408061445.i76EjqT07026@guido.python.org>
	<200408090105.i7915mfS005734@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040810224644.GA9758@grulic.org.ar>

On Mon, Aug 09, 2004 at 01:05:48PM +1200, Greg Ewing wrote:
> 
> Perhaps someone can post some real-life use cases written with this
> syntax, so we can see what it would *really* look like in typical use,
> as opposed to made-up worst-case examples?

well, how about this:

    def someMethod(klass, anarg, otharg, lastarg):
        """
            Summary.

            Long description that explains the details about someMethod.
            Aliquam venenatis orci in risus. Nunc ornare aliquam
            lectus. Integer ligula turpis, posuere id, commodo ut, molestie
            id, est. Donec eu odio. Fusce at tellus in erat iaculis
            suscipit. Nulla metus dui, tristique vel, posuere sed,
            consectetuer sed, pede. Pellentesque habitant morbi tristique
            senectus et netus et malesuada fames ac turpis egestas. Mauris in
            orci sed metus porta auctor.

            @pre some_validator(lastarg)
            @post rv_validator(__return__)

            @type anarg: aClass
            @param anarg: desc of anarg
            @type otharg: othClass
            @param otharg: desc of otharg
            @type lastarg: lastClass
            @param lastarg: longer desc of lastarg because it's special

            @rtype: rClass
            @return: return value description
        """
        pass
    someMethod = classmethod(someMethod)

(yes, I have code that looks like this, roughly, from memory; I
make no claims as to typicality)

this could be rendered as (inventing some stuff here, probably
suboptimally; also, I assume you can access the args of the
function by name---I'm unable to verify that at this time):

    @param(anarg, aClass, 'desc of anarg')
    @param(otharg, othClass, 'desc of otharg')
    @param(lastarg, lastClass, "longer desc of lastarg because it's special")
    @return(rClass, 'return value description')
    @precondition(some_validator, lastarg)
    @postcondition(rv_validator)
    @classmethod
    def someMethod(klass, anarg, otharg, lastarg):
        """
            Summary.

            Long description that explains the details about someMethod.
            Aliquam venenatis orci in risus. Nunc ornare aliquam
            lectus. Integer ligula turpis, posuere id, commodo ut, molestie
            id, est. Donec eu odio. Fusce at tellus in erat iaculis
            suscipit. Nulla metus dui, tristique vel, posuere sed,
            consectetuer sed, pede. Pellentesque habitant morbi tristique
            senectus et netus et malesuada fames ac turpis egestas. Mauris in
            orci sed metus porta auctor.

        """
        pass

versus

    def someMethod(klass, anarg, otharg, lastarg) \
        [param(anarg, aClass, 'desc of anarg'),
         param(otharg, othClass, 'desc of otharg'),
         param(lastarg, lastClass, "longer desc of lastarg because it's special"),
         return(rClass, 'return value description'),
         precondition(some_validator, lastarg),
         postcondition(rv_validator),
         classmethod]:
        """
            Summary.

            Long description that explains the details about someMethod.
            Aliquam venenatis orci in risus. Nunc ornare aliquam
            lectus. Integer ligula turpis, posuere id, commodo ut, molestie
            id, est. Donec eu odio. Fusce at tellus in erat iaculis
            suscipit. Nulla metus dui, tristique vel, posuere sed,
            consectetuer sed, pede. Pellentesque habitant morbi tristique
            senectus et netus et malesuada fames ac turpis egestas. Mauris in
            orci sed metus porta auctor.

        """
        pass

I must say I found decorators' '@' ugly until I thought about
using it for doing what epydoc and contract do in docstrings...

-- 
John Lenton (john@grulic.org.ar) -- Random fortune:
Se olvida una buena acci?n, y no un buen bofet?n. 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040810/d4de6078/attachment.pgp
From guido at python.org  Wed Aug 11 02:02:42 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug 11 02:02:50 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but
	aren't
In-Reply-To: Your message of "Tue, 10 Aug 2004 12:17:14 EDT."
	<BE05E963-EAE8-11D8-9899-000A95A50FB2@fuhm.net> 
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com>
	<2macx31423.fsf@starship.python.net>
	<200408101501.i7AF1HZ25725@guido.python.org> 
	<BE05E963-EAE8-11D8-9899-000A95A50FB2@fuhm.net> 
Message-ID: <200408110002.i7B02gu26415@guido.python.org>

> Sooo should (for 'generator' in objects that claim to be in 
> __builtins__ but aren't),
> 1) 'generator' be added to __builtins__
> 2) 'generator' be added to types.py and its __module__ be set to 'types'
> 3) 'generator' be added to <newmodule>.py and its __module__ be set to 
> '<newmodule>' (and a name for the module chosen)

I guess (1).

Please submit a patch to SF...

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Wed Aug 11 02:09:54 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug 11 02:10:01 2004
Subject: [Python-Dev] Decorator order implemented backwards?
In-Reply-To: Your message of "Tue, 10 Aug 2004 22:07:20 BST."
	<1092172040.1681.9.camel@localhost> 
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au> 
	<1092172040.1681.9.camel@localhost> 
Message-ID: <200408110009.i7B09s626481@guido.python.org>

> No, it has always been this way round.  In fact test_order was inherited
> from Guido's original version of test_decorators.py (from patch #926860)
> where it read:
> 
>     def test_order(self):
>         class C(object):
>             [funcattrs(abc=1), staticmethod]
>             def foo(): return 42
>         # This wouldn't work if staticmethod was called first
>         self.assertEqual(C.foo(), 42)
>         self.assertEqual(C().foo(), 42)
> 
> (i.e. identical to the current version except for the change in
> syntax).  In fact I relied on the fact that this test passed to
> convince me I had the order right!  But I should have spotted the
> inconsistency between that and the documentation that I wrote for
> the reference manual.

Oops.  When using a list, it makes sense to apply the decorators
left-to-right.  But when using @deco, I think it makes more sense that
the decorators closer to the def are applied first.  IOW the syntax is
conceptually right-recursive:

    definition ::= '@' decorator definition | 'def' ...

> I'll do a patch to fix the order and the corresponding tests.

Great.

> While I'm at it, do we want to drop support for multiple decorators
> on a single line?  Nobody has spoken up for it, and in fact forcing
> one-per-line would simplify the grammar (as well as making it easier
> for toy parsers to find decorators).

+1

--Guido van Rossum (home page: http://www.python.org/~guido/)
From jack at performancedrivers.com  Wed Aug 11 03:11:16 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Wed Aug 11 03:11:22 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: <200408072105.i77L5Ea10965@guido.python.org>
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org>
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
	<200408072105.i77L5Ea10965@guido.python.org>
Message-ID: <20040811011116.GF23725@performancedrivers.com>

On Sat, Aug 07, 2004 at 02:05:14PM -0700, Guido van Rossum wrote:
> > I just wanted to say that I believe it should be allowed to decorate
> > classes. There's not reason enough (reading the thread linked to by the
> > PEP) to limit decorators this way. Like said several times in that
> > thread, metaclasses are harder (to write and to understand) than
> > decorators.
> > 
> > Also, considering that you said you would have actually wanted
> > docstrings above the definition of a function, wouldn't this apply
> > to classes as well?
> > 
> > Anyway, it's an artificial limit and it would be better to be able
> > to test it out during the alpha.
> 
> Are you volunteering to implement it?
> 

I'm game, I would prefer to use class decorators in most of the places
I currently use metaclasses.  I dropped the original patch author (Mark
Russel) an email yesterday after looking through the 4kloc patch fishing
for advice.

If someone would clue me in on which files are generated and which
ones need to be done by hand it would be a big help.  It wasn't obvious
to me (other than compile.c and tokenizer.c) on first reading.

TIA,

-Jack
From mkc at mathdogs.com  Wed Aug 11 03:29:32 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Wed Aug 11 03:29:37 2004
Subject: [Python-Dev] Re: re.split on empty patterns
References: <20040807145142.GB2529@rogue.amk.ca>
	<1f7befae040807105878b47c61@mail.gmail.com>
	<cfa2qo$ru9$1@sea.gmane.org>
Message-ID: <lkdk6w6xy0z.fsf@panix2.panix.com>

"Fredrik Lundh" <fredrik@pythonware.com> writes:
> the RE module already have an established method for specifying flags,
> either with (?x) markers in the pattern strings, or via the "flags" argument
> to re.compile and others.
> 
> any reason why you cannot use a new standard flag for this purpose?
> 
> (e.g. (?e) and re.EMPTY_OK)

I considered this when I was writing the patch, but it seemed like the
"(?x)"-style flags all have to do with the meaning and interpretation of the
pattern.  They're quite general and make sense for all of the methods that use
patterns.

The "empty okay" concept, on the other hand, seems to have to do specifically
with how re.split uses the pattern.  That is, in the process of finding
matches decides (or not) to discard the empty matches.  Would you even want
the other methods to have this "do what I say but also discard empty matches"
wart?  IMO, the only reason for the existence of the option is to preserve
backward compatibility with previous code that assumes empties will be
discarded.  This is something we want to go away eventually, IMO.

> for extra credit, add EMPTY_OK behaviour to find, finditer, and sub/subn.

I'd vote -1, since there's no backward compatibility issue for these.  If
someone writing new code doesn't want to match empty strings, they can easily
write a pattern that doesn't match empty strings.

Mike

From bac at OCF.Berkeley.EDU  Wed Aug 11 03:30:24 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Aug 11 03:30:58 2004
Subject: [Python-Dev] Prothon on CPython intrepreter? [PROTHON]
In-Reply-To: <000a01c47f0b$b3cf4690$0b01a8c0@mark>
References: <000a01c47f0b$b3cf4690$0b01a8c0@mark>
Message-ID: <411976B0.2020705@ocf.berkeley.edu>

Mark Hahn wrote:

[SNIP]
> I'm sure you've heard the joke that ends with "we've established you're a
> whore, now we are just haggling over the price".  If I can compromise
> Prothon for .Net, why can't I compromise it some more for CPython?  So I am
> thinking of doing a Prothon implementation in CPython before doing the .Net
> version.  I will get a library and community fast like I would have with
> .Net.  
> 

I am personally very glad to hear this!

> This CPython->.Net->CProthon roadmap will get Prothon out quickly and slowly
> easy my wild ideas for the interpreter into Prothon.  It will also make
> CPython a multiple language VM for the first time.
> 
> So, I'll finally get to some questions:
> 
[SNIP - question for Guido]

> 2) Lenard Lindstrom said he thought I could "Add some extra byte code
> instructions to handle Prothon's distinct scoping rules and implement
> prototypes using PyType_Type".

That is definitely possible.  If you look at the opcodes that currently 
exist (http://www.python.org/dev/doc/devel/lib/bytecodes.html for the 
in-dev docs; doesn't list LIST_APPEND or NOP, though ... Raymond, you 
planning on documenting these, or should I file a bug report?), there 
are opcodes for loading from locals, globals, names (when you don't know 
which scope), and cells (closure stuff).  Basically the opcode just 
specifies where to look in the code object for the value.  So yes, you 
could emit another opcode if you needed to for scope and you made the 
appropriate changes to the code object if you need other fields.

> He also said "All attribute accesses would be
> through a special attribute handler method".  I don't know the Python
> interpreter code yet, but does this sound workable to everyone?  It doesn't
> have to be Python compatible.
> 

Well, all attribute access goes through descriptors already (if I 
remember correctly from PyCON 2003's sprint).  You could just change the 
default descriptor to do what you want.

> 3) Could Tim Peter's welcome offer to Prothon here on Python-dev be
> stretched to include the many questions I would have when modifying CPython?
> In other words would this project be welcomed as a sandbox for Python or
> would it be an unwelcome drain on your limited resources?
> 

=)  You are are free to ask, just don't be floored if you are warnocked 
on occasion (warnocking, since I have only heard this in the Perl world 
form the perl-6-internals list, is when a message gets no response).  I 
personally have no issue and actually welcome the discussion since it 
could help with improving CPython with new ideas.  And this is coming 
from someone who will have to read the emails.  =)  I know I am 
personally willing to answer your questions.

I do have a piece of advice before you delve into this too deeply, and 
that is to consider looking at the AST branch (tagged as ast-branch in 
CVS).  Jeremy Hylton has been leading a rewrite of the compiler to take 
the concrete syntax tree, generate an AST for it, and then emit opcode 
from that.  It is *much* easier to use than how compile.c works now.  It 
also, in my opinion, makes things clearer by breaking up compilation 
into more, smaller steps.  It isn't in the main trunk yet, but it will 
be someday (I personally plan to work on it once my thesis is done so 
that I can hopefully integrate my work into the core more easily and it 
is now tradition for some people to sprint on it at PyCON =) .

Who knows, you might be able to just generate your own version of the 
AST or just have a different compiler for the AST itself and thus cut 
down and code duplication.

-Brett
From bac at OCF.Berkeley.EDU  Wed Aug 11 03:36:47 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Aug 11 03:36:53 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <1092177771.21839.54.camel@localhost>
References: <1092177771.21839.54.camel@localhost>
Message-ID: <4119782F.8040902@ocf.berkeley.edu>

Barry Warsaw wrote:

[SNIP]
> Now for the controversial bit <wink>.
> 

Can't leave well enough alone, can you, Barry?  =)

> I still think it's worthwhile to turn the string module into a package. 
> The sandbox has an implementation of this that is completely backward
> compatible.  I like it because it segregates the deprecated functions
> into a separate sub-module and makes the whole shebang easier to
> maintain.  It also provides an existing namespace for future expansion,
> as has been discussed before.
> 

In case Guido cares, I personally am (and have been) +1 on the 
re-organization.

-Brett
From greg at cosc.canterbury.ac.nz  Wed Aug 11 03:46:59 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug 11 03:47:07 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <2mvffqzum9.fsf@starship.python.net>
Message-ID: <200408110146.i7B1kxua009746@cosc353.cosc.canterbury.ac.nz>

> AAAARGHGH!  lookdict "isn't allowed" to raise
> exceptions, so it does a PyErr_Clear(), goes on to futilely chase the
> entire dict looking for another match on the hash code, and we've
> effectively turned a MemoryError into a KeyError.

This suggests to me that the very concept of a function
that's not allowed to return errors is dubious, since
even just calling another function could provoke a
MemoryError.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From mark at prothon.org  Wed Aug 11 03:54:38 2004
From: mark at prothon.org (Mark Hahn)
Date: Wed Aug 11 03:54:28 2004
Subject: [Python-Dev] Re: Prothon on CPython intrepreter? [PROTHON]
References: <000a01c47f0b$b3cf4690$0b01a8c0@mark>
	<411976B0.2020705@ocf.berkeley.edu>
Message-ID: <2przc3fdgcvh.1swvqarmq0ngp.dlg@40tude.net>

On Tue, 10 Aug 2004 18:30:24 -0700, Brett C. wrote:

...<snip>...

Thanks for your words of support.

> I do have a piece of advice before you delve into this too deeply, and 
> that is to consider looking at the AST branch (tagged as ast-branch in 
> CVS).  Jeremy Hylton has been leading a rewrite of the compiler to take 
> the concrete syntax tree, generate an AST for it, and then emit opcode 
> from that.  It is *much* easier to use than how compile.c works now.  It 
> also, in my opinion, makes things clearer by breaking up compilation 
> into more, smaller steps.  It isn't in the main trunk yet, but it will 
> be someday (I personally plan to work on it once my thesis is done so 
> that I can hopefully integrate my work into the core more easily and it 
> is now tradition for some people to sprint on it at PyCON =) .
> 
> Who knows, you might be able to just generate your own version of the 
> AST or just have a different compiler for the AST itself and thus cut 
> down and code duplication.

I would like to use the AST since IronPython does.  This might make moving
from CPython to .Net easier later. Also Paul Prescod has been bugging me to
use the AST in general.   

I am planning on using stackless to get the lightweight threads so I don't
know how much trouble it will be to integrate all these versions.  Clearly
my codebase will be separate from everyone elses so I don't have to worry
about compatibility, but it would be nice to somehow make it easy to get
fixes from the Python codebase into mine easily.

From greg at cosc.canterbury.ac.nz  Wed Aug 11 03:57:03 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug 11 03:57:09 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <20040810224644.GA9758@grulic.org.ar>
Message-ID: <200408110157.i7B1v30u009783@cosc353.cosc.canterbury.ac.nz>

> > Perhaps someone can post some real-life use cases written with this
> > syntax,
> 
> well, how about this:
> 
>     def someMethod(klass, anarg, otharg, lastarg):
>         """
>             Summary.
> 
>             Long description that explains the details about someMethod.
>             Aliquam venenatis orci in risus. Nunc ornare aliquam

I was hoping for some real live code, e.g. an excerpt from
PyObjC. This is still a made-up example.

>    @param(anarg, aClass, 'desc of anarg')
>    @param(otharg, othClass, 'desc of otharg')
>    @param(lastarg, lastClass, "longer desc of lastarg because it's special")
>    @return(rClass, 'return value description')
>    @precondition(some_validator, lastarg)
>    @postcondition(rv_validator)
>    @classmethod
>    def someMethod(klass, anarg, otharg, lastarg):
>        """
>            Summary.

And this looks awful - the method name is almost completely
lost under the deluge of preceding junk!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From michael.walter at gmail.com  Wed Aug 11 04:14:30 2004
From: michael.walter at gmail.com (Michael Walter)
Date: Wed Aug 11 04:14:33 2004
Subject: [Python-Dev] Re: def fn (args) [dec,dec]:
In-Reply-To: <200408110157.i7B1v30u009783@cosc353.cosc.canterbury.ac.nz>
References: <200408110157.i7B1v30u009783@cosc353.cosc.canterbury.ac.nz>
Message-ID: <877e9a1704081019142dcaa0f6@mail.gmail.com>

Let's see how we could reduce that in a (hypothetical)
optionally-typed Python, supposed that we use meaningful
variable+classnames (thus usually avoiding the need for parameter
descriptions):

@precondition(some_validator, lastarg)
@postcondition(rv_validator)
@classmethod
def someMethod(klass, anarg : aClass, otharg : othClass, lastarg :
lastClass) -> rClass:
  # foo

By moving preconditions into the type (which is often possible -- in
the other case we often don't want to expose the precondition
publically, anyway, at least in my limited experience), we can reduce
this some more:

@postcondition(rv_validator)
@classmethod
def someMethod(klass, anarg : aClass, otharg : othClass, lastarg :
lastClassSubType) -> rClass:
  # foo

The only remaining question would be whether the postcondition should
be revealed in the method's "interface" (which I consider the
decorations to be part of) - I would also think that usually this
isn't the case (question: would you put them into the code in the form
of asserts instead? or use the old someMethod =
postconditioned(someMethod, rv_validator) syntax?).

I'm pretty sure I've made too many assumptions for too many people
(and I don't really see the point of my post yet), but oh well :)

Cheers,
Michael

On Wed, 11 Aug 2004 13:57:03 +1200, Greg Ewing
<greg@cosc.canterbury.ac.nz> wrote:
> > > Perhaps someone can post some real-life use cases written with this
> > > syntax,
> >
> > well, how about this:
> >
> >     def someMethod(klass, anarg, otharg, lastarg):
> >         """
> >             Summary.
> >
> >             Long description that explains the details about someMethod.
> >             Aliquam venenatis orci in risus. Nunc ornare aliquam
> 
> I was hoping for some real live code, e.g. an excerpt from
> PyObjC. This is still a made-up example.
> 
> >    @param(anarg, aClass, 'desc of anarg')
> >    @param(otharg, othClass, 'desc of otharg')
> >    @param(lastarg, lastClass, "longer desc of lastarg because it's special")
> >    @return(rClass, 'return value description')
> >    @precondition(some_validator, lastarg)
> >    @postcondition(rv_validator)
> >    @classmethod
> >    def someMethod(klass, anarg, otharg, lastarg):
> >        """
> >            Summary.
> 
> And this looks awful - the method name is almost completely
> lost under the deluge of preceding junk!
> 
> 
> 
> Greg Ewing, Computer Science Dept, +--------------------------------------+
> University of Canterbury,          | A citizen of NewZealandCorp, a       |
> Christchurch, New Zealand          | wholly-owned subsidiary of USA Inc.  |
> greg@cosc.canterbury.ac.nz         +--------------------------------------+
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com
>
From tim.peters at gmail.com  Wed Aug 11 04:26:31 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Aug 11 04:26:34 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <200408110146.i7B1kxua009746@cosc353.cosc.canterbury.ac.nz>
References: <200408110146.i7B1kxua009746@cosc353.cosc.canterbury.ac.nz>
Message-ID: <1f7befae04081019262cf60167@mail.gmail.com>

[Tim Peters]
>> AAAARGHGH!  lookdict "isn't allowed" to raise
>> exceptions, so it does a PyErr_Clear(), goes on to futilely chase the
>> entire dict looking for another match on the hash code, and we've
>> effectively turned a MemoryError into a KeyError.

[Greg Ewing]
> This suggests to me that the very concept of a function
> that's not allowed to return errors is dubious, since
> even just calling another function could provoke a
> MemoryError.

I believe I had a more elegant way to say that:  AAAARGHGH! <wink>

Alas, 10 billion call sites, including in external extensions, rely on
that a NULL return from PyDict_GetItem() or PyDict_GetItemString() is
the unexceptional "nope, the key's not there".  So there's no
realistic chance of repairing this before Python 3.0 (if even then). 
In the meantime, KeyError is confusable with almost all exceptions.

For some real fun, ask on a Zope list about hasattr() suppressing all
exceptions.  Not that Guido ever expected a hasttr() call to try to
mess with persistent objects across a networked database as side
effect.

Python originally had only string-keyed dicts, and the basic dict API
still reflects that a dict lookup simply "can't fail" for any reason
other than "key ain't there".  That used to be true.  For that matter,
hasattr() was originally no worse than a string-keyed dict lookup too,
so "can't fail" also made good sense there at the time.
From mkc at mathdogs.com  Wed Aug 11 05:01:21 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Wed Aug 11 05:01:27 2004
Subject: [Python-Dev] Re: pre-PEP: Complete,
	Structured Regular Expression Group Matching
References: <411429FB.6080603@gradient.cis.upenn.edu>
	<lkd1xijedzx.fsf@panix2.panix.com> <brhn9guy.fsf@yahoo.co.uk>
	<lkdzn56aljx.fsf@panix2.panix.com> <4118C626.4070203@heneryd.com>
Message-ID: <lkdfz6uxtry.fsf@panix2.panix.com>

Erik Heneryd <erik@heneryd.com> writes:
> Mike Coleman wrote:
> > Paul Moore <pf_moore@yahoo.co.uk> writes:
> 
> >>The only other comment I have is that the semantics seem pretty
> >>complex - I think that in practice, they do more or less what you want
> >>them to, but the description is pretty obscure.
> 
> I'd say that is probably a warning flag, but I don't know, at the same time it
> looks like it could be kind of useful.

It concerns me a little, too.  No doubt the documentation could be better.
But if you're already familiar with re.match and structure interpolation (or
whatever it's called):

        x = 1
        y = [3, 4, 5]
        z = [x, y, [x]]
        z -> [1, [3 4, 5], [1]]

then to me it has a certain ring of familiarity.

> Exceptions are the way to go.  You should get what you ask for, or a heads up.
> Errors should never pass silently.

I absolutely agree on not being silent on error.  The alternative would be to
return an error index (i.e., an integer) on error and then have the caller
observe the type of the return value (integer on failuer, list on success) to
determine success.  But again, throwing an exception does seem more natural.

> I hacked up a Python prototype, mostly as a challenge.  It's brittle and it's
> ugly, but it kind of works.  Though, if /F says it would require a major
> rewrite of the C-engine, that's probably true... <wink>

Impressive--I thought that'd take a lot more code.  I'll study it.  Thanks for
taking up the challenge!

Mike

From kbk at shore.net  Wed Aug 11 05:15:37 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Aug 11 05:15:42 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200408110315.i7B3FbIe016888@h006008a7bda6.ne.client2.attbi.com>


Patch / Bug Summary
___________________

Patches :  284 open ( +8) /  2514 closed (+21) /  2798 total (+29)
Bugs    :  744 open (-22) /  4361 closed (+44) /  5105 total (+22)
RFE     :  149 open ( +3) /   130 closed ( +0) /   279 total ( +3)

New / Reopened Patches
______________________

Simple File fix for Windows Runtime incompatability  (2004-08-04)
       http://python.org/sf/1003535  opened by  Jeremy Schwartz

fix for bugs 976878, 926369, 875404 (pdb bkpt handling)  (2004-08-04)
       http://python.org/sf/1003640  opened by  Ilya Sandler

Fix for 1002475 (Feedparser not handling \r\n correctly)  (2004-08-05)
CLOSED http://python.org/sf/1003693  opened by  Tony Meyer

add socketpair function to socket module  (2004-08-05)
CLOSED http://python.org/sf/1003700  opened by  Dave Cole

Fix markup in concrete.tex  (2004-08-05)
CLOSED http://python.org/sf/1003861  opened by  Dima Dorfman

adds an index entry for __all__ to the tutorial  (2004-08-05)
CLOSED http://python.org/sf/1003863  opened by  Clinton Roy

comparison of Decimal instance with None fails  (2004-08-05)
CLOSED http://python.org/sf/1004018  opened by  Anthony Tuininga

big code objects (>64K) may be optimized incorrectly  (2004-08-05)
CLOSED http://python.org/sf/1004088  opened by  Cezary Biernacki

untested patch for compile.c (bug 1004088)  (2004-08-05)
CLOSED http://python.org/sf/1004095  opened by  Cezary Biernacki

Make func_name writable  (2004-08-06)
       http://python.org/sf/1004703  opened by  Michael Hudson

curses.wrapper should also forward keyword args  (2004-08-07)
CLOSED http://python.org/sf/1005008  opened by  Rob Nichols

test_curses fails on MacOS X 10.3  (2004-08-07)
CLOSED http://python.org/sf/1005123  opened by  Ronald Oussoren

property to get the docstring from fget  (2004-08-08)
       http://python.org/sf/1005461  opened by  Dima Dorfman

Markup tweaks to @deco examples in libref  (2004-08-08)
CLOSED http://python.org/sf/1005465  opened by  Dima Dorfman

fix inspect.getargs() crash on def foo((bar)) (bug #891637)  (2004-08-08)
       http://python.org/sf/1005466  opened by  Johannes Gijsbers

Disambiguate "min() or max()" exception string  (2004-08-08)
       http://python.org/sf/1005468  opened by  Dima Dorfman

use __name__ = '__main__' in tools/scripts (bug #624893)  (2004-08-08)
CLOSED http://python.org/sf/1005491  opened by  Johannes Gijsbers

Avoid seg fault if list object is modified during list.index  (2004-08-09)
       http://python.org/sf/1005778  opened by  Dima Dorfman

replacing rfc822.formatdate  (2004-08-09)
       http://python.org/sf/1005857  opened by  Anthony Baxter

support --with-tsc on PPC  (2004-08-09)
       http://python.org/sf/1005891  opened by  Michael Hudson

test_curses bugs  (2004-08-09)
       http://python.org/sf/1005892  opened by  Alexandr Zamaraev

curses for win32  (2004-08-09)
       http://python.org/sf/1005895  opened by  Alexandr Zamaraev

Patch to allow building of paper-*/dist.pdf  (2004-08-09)
       http://python.org/sf/1005913  opened by  Jeff Epler

Cygwin standard module build problems  (2004-08-09)
       http://python.org/sf/1006003  opened by  Jason Tishler

Cygwin standard module build problems  (2004-08-09)
       http://python.org/sf/1006003  opened by  Jason Tishler

make inspect.getsource show @decorators  (2004-08-09)
CLOSED http://python.org/sf/1006219  opened by  Simon Percivall

cross compile patch  (2004-08-09)
       http://python.org/sf/1006238  opened by  Daniel Goertzen

Match Solaris def'n of _XOPEN_SOURCE  (2004-08-10)
       http://python.org/sf/1006629  opened by  Skip Montanaro

Remove ODBC library references from Windows build  (2004-08-11)
       http://python.org/sf/1006916  opened by  Nick Coghlan

Remove ODBC library references from Windows build  (2004-08-11)
       http://python.org/sf/1006916  opened by  Nick Coghlan

Build Patch #941881 (PEP 309 in C) on windows  (2004-08-11)
       http://python.org/sf/1006948  opened by  Nick Coghlan

Patches Closed
______________

O(1) xrange membership testing  (2004-08-02)
       http://python.org/sf/1002085  closed by  loewis

Allow any encodings other than latin-1 in interactive interp  (2004-04-24)
       http://python.org/sf/941229  closed by  perky

Fix for 1002475 (Feedparser not handling \r\n correctly)  (2004-08-04)
       http://python.org/sf/1003693  closed by  bwarsaw

add socketpair function to socket module  (2004-08-05)
       http://python.org/sf/1003700  closed by  davecole

fix build of docs in info format  (2004-06-15)
       http://python.org/sf/972991  closed by  doko

Fix markup in concrete.tex  (2004-08-05)
       http://python.org/sf/1003861  closed by  akuchling

adds an index entry for __all__ to the tutorial  (2004-08-05)
       http://python.org/sf/1003863  closed by  mwh

efficient string concatenation  (2004-06-27)
       http://python.org/sf/980695  closed by  rhettinger

comparison of Decimal instance with None fails  (2004-08-05)
       http://python.org/sf/1004018  closed by  rhettinger

big code objects (>64K) may be optimized incorrectly  (2004-08-05)
       http://python.org/sf/1004088  closed by  rhettinger

untested patch for compile.c (bug 1004088)  (2004-08-05)
       http://python.org/sf/1004095  closed by  cezary

curses.wrapper should also forward keyword args  (2004-08-07)
       http://python.org/sf/1005008  closed by  mwh

test_curses fails on MacOS X 10.3  (2004-08-07)
       http://python.org/sf/1005123  closed by  mwh

Nails down the semantics of dict setitem  (2003-06-03)
       http://python.org/sf/748126  closed by  mwh

httplib.HTTPConnection.request() bug  (2004-07-25)
       http://python.org/sf/997626  closed by  jhylton

assert should not generate code if optimized  (2003-11-05)
       http://python.org/sf/836879  closed by  jhylton

Update kwargs in pickle docs to match implementations  (2004-07-28)
       http://python.org/sf/999280  closed by  akuchling

installing modules are explained in old filenames  (2003-12-18)
       http://python.org/sf/862531  closed by  akuchling

Markup tweaks to @deco examples in libref  (2004-08-08)
       http://python.org/sf/1005465  closed by  akuchling

use __name__ = '__main__' in tools/scripts (bug #624893)  (2004-08-08)
       http://python.org/sf/1005491  closed by  akuchling

proposed patch for posixpath.py: getctime()  (2003-04-12)
       http://python.org/sf/720188  closed by  kbk

Cygwin standard module build problems  (2004-08-09)
       http://python.org/sf/1006003  closed by  jlt63

make inspect.getsource show @decorators  (2004-08-09)
       http://python.org/sf/1006219  closed by  percivall

New / Reopened Bugs
___________________

segfault when running smtplib example  (2004-08-04)
       http://python.org/sf/1003195  opened by  Bertram Scharpf

telnetlib on Windows: read_until() freezes  (2004-08-04)
CLOSED http://python.org/sf/1003323  opened by  Yannick Turgeon

Python 1.5.2 security vulnerability  (2004-08-04)
       http://python.org/sf/1003471  opened by  Kirby Kuehl

xrange overflows  (2004-08-05)
       http://python.org/sf/1003935  reopened by  hfuru

xrange overflows  (2004-08-05)
       http://python.org/sf/1003935  opened by  Hallvard B Furuseth

Shortcut keys don't work when CAPS LOCK is on  (2004-08-05)
CLOSED http://python.org/sf/1004217  opened by  otto

imaplib.IMAP4.select doesn't return critical data  (2004-08-05)
CLOSED http://python.org/sf/1004271  opened by  Charles

Type returned from .keys() is not checked  (2004-08-06)
CLOSED http://python.org/sf/1004629  opened by  ben handley

Type returned from .keys() is not checked  (2004-08-06)
CLOSED http://python.org/sf/1004669  opened by  ben handley

Int Type Documentation incomplete  (2004-08-06)
CLOSED http://python.org/sf/1004698  opened by  Colin J. Williams

Carbon.File fails if server vol is mounted after launch  (2004-08-06)
       http://python.org/sf/1004810  opened by  Paul Pharr

Limited User gets error in 2.4a2 MSI  (2004-08-06)
CLOSED http://python.org/sf/1004837  opened by  Martin v. L?wis

iso-2022-jp crashes  (2004-08-07)
CLOSED http://python.org/sf/1005078  opened by  Martin v. L?wis

test__locale fails on MacOS X  (2004-08-07)
       http://python.org/sf/1005113  opened by  Ronald Oussoren

new.code() not cleanly checking its arguments  (2004-08-07)
       http://python.org/sf/1005248  opened by  Armin Rigo

_XOPEN_SOURCE issue on IRIX 5.3  (2004-08-08)
       http://python.org/sf/1005308  opened by  Georg Schwarz

inconsistent acceptance of floats for range()  (2004-08-07)
       http://python.org/sf/1005325  opened by  Brett Cannon

_SC_PAGE_SIZE unknown on IRIX 5.3  (2004-08-08)
       http://python.org/sf/1005568  opened by  Georg Schwarz

compile errors on _codecs_iso2022.c  (2004-08-09)
       http://python.org/sf/1005737  opened by  roadkill

imaplib.IMAP4.getquotaroot error  (2004-08-09)
CLOSED http://python.org/sf/1005933  opened by  Thierry FLORAC

Index entries with "_" are wrong in pdf file  (2004-08-09)
       http://python.org/sf/1005936  opened by  Jeff Epler

leaking snippet  (2003-09-18)
       http://python.org/sf/808596  reopened by  mwh

FutureWarning when running regression tests  (2004-08-09)
       http://python.org/sf/1006001  opened by  Viktor Ferenczi

odbc queries with very long strings fail with no exceptions  (2004-08-09)
CLOSED http://python.org/sf/1006056  opened by  leve

distutils builds too many rpms  (2004-08-10)
CLOSED http://python.org/sf/1006607  opened by  Jeff MacDonald

Possible memory leak in output console  (2004-08-10)
       http://python.org/sf/1006740  opened by  Kirk Strauser

Bugs Closed
___________

socketmodule does not build under cygwin  (2004-08-02)
       http://python.org/sf/1001857  closed by  jlt63

unicode.width broken for combining characters  (2004-07-12)
       http://python.org/sf/989185  closed by  lemburg

test_logging fails if run twice  (2004-08-03)
       http://python.org/sf/1002537  closed by  vsajip

variable reuse in the logging module  (2004-07-16)
       http://python.org/sf/992397  closed by  vsajip

telnetlib on Windows: read_until() freezes  (2004-08-04)
       http://python.org/sf/1003323  closed by  mwh

email message parser doesn't handle \r\n correctly  (2004-08-03)
       http://python.org/sf/1002475  closed by  bwarsaw

Compiler warnings for Modules/readline.c:flex_complete()  (2004-07-09)
       http://python.org/sf/988300  closed by  bcannon

xrange overflows  (2004-08-05)
       http://python.org/sf/1003935  closed by  tim_one

strftime() backwards incompatibility  (2004-07-24)
       http://python.org/sf/997368  closed by  bwarsaw

Shortcut keys don't work when CAPS LOCK is on  (2004-08-05)
       http://python.org/sf/1004217  closed by  kbk

bdist_rpm license documented, but not accepted as valid  (2004-07-05)
       http://python.org/sf/985478  closed by  melicertes

imaplib.IMAP4.select doesn't return critical data  (2004-08-05)
       http://python.org/sf/1004271  closed by  melicertes

Type returned from .keys() is not checked  (2004-08-06)
       http://python.org/sf/1004629  closed by  rhettinger

Type returned from .keys() is not checked  (2004-08-06)
       http://python.org/sf/1004669  closed by  rhettinger

Int Type Documentation incomplete  (2004-08-06)
       http://python.org/sf/1004698  closed by  mwh

Limited User gets error in 2.4a2 MSI  (2004-08-06)
       http://python.org/sf/1004837  closed by  loewis

test_decimal fails if repeated  (2004-08-03)
       http://python.org/sf/1002530  closed by  rhettinger

doctest.DocTestSuite does not support __test__  (2003-07-16)
       http://python.org/sf/772091  closed by  tim_one

iso-2022-jp crashes  (2004-08-07)
       http://python.org/sf/1005078  closed by  loewis

replace file() with open() in libcvs.tex  (2004-07-26)
       http://python.org/sf/998307  closed by  akuchling

httplib fails on Akamai URLs  (2004-01-11)
       http://python.org/sf/874842  closed by  jhylton

pickletools module is not documented  (2004-01-08)
       http://python.org/sf/873146  closed by  akuchling

readline function pointer with not tty's  (2004-07-23)
       http://python.org/sf/996259  closed by  mwh

test_tcl fails --   (2004-02-26)
       http://python.org/sf/905046  closed by  mwh

AIX POLLNVAL definition causes problems  (2004-03-25)
       http://python.org/sf/923315  closed by  akuchling

typo in os.popen4 documentation  (2004-07-26)
       http://python.org/sf/998066  closed by  akuchling

sre.sub documentation is incomplete  (2004-07-14)
       http://python.org/sf/990792  closed by  akuchling

PyObject_CallMethod docs unclear (with dangerous results)  (2004-07-15)
       http://python.org/sf/991883  closed by  akuchling

PyArg_ParseTuple can miss errors with warnings as exceptions  (2004-07-15)
       http://python.org/sf/991812  closed by  mwh

Pyton 2.3.4 Make Test Fails on Mac OS X  (2004-06-10)
       http://python.org/sf/970799  closed by  bcannon

SimpleHTTPServer docs out of date  (2003-12-27)
       http://python.org/sf/866222  closed by  akuchling

List comprehensions leaking control variable name deprecated  (2003-10-20)
       http://python.org/sf/827209  closed by  akuchling

2.3a2 site.py non-existing dirs  (2003-02-25)
       http://python.org/sf/693255  closed by  bcannon

Webchecker error on http://www.naleo.org  (2002-08-08)
       http://python.org/sf/592441  closed by  mwh

Uninitialized variable used in Tools/faqwiz/faqwiz.py  (2003-07-25)
       http://python.org/sf/777659  closed by  mwh

On HPUX 11i universal newlines seems to cause readline(s) to  (2004-06-25)
       http://python.org/sf/979872  closed by  mwh

doctest fails with TypeError  (2003-07-02)
       http://python.org/sf/764504  closed by  tim_one

doctest chokes on recursive members  (2003-12-23)
       http://python.org/sf/864944  closed by  tim_one

doctest confused by super-instances in class-dicts  (2004-02-23)
       http://python.org/sf/902628  closed by  tim_one

texi2html.py not working  (2004-05-06)
       http://python.org/sf/948881  closed by  anthonybaxter

imaplib.IMAP4.getquotaroot error  (2004-08-09)
       http://python.org/sf/1005933  closed by  pierslauder

odbc queries with very long strings fail with no exceptions  (2004-08-09)
       http://python.org/sf/1006056  closed by  loewis

pydoc shows tools/scripts but blows up  (2002-10-17)
       http://python.org/sf/624893  closed by  akuchling

Dubious use of Unicode literals in tutorial  (2003-07-18)
       http://python.org/sf/773351  closed by  ghaering

distutils builds too many rpms  (2004-08-10)
       http://python.org/sf/1006607  closed by  jonez

Ctrl+key combos stop working in IDLE  (2003-10-31)
       http://python.org/sf/833957  closed by  kbk

New / Reopened RFE
__________________

Adding missing ISO 8859 codecs, especially Thai  (2004-08-02)
       http://python.org/sf/1001895  reopened by  loewis

PEP 263: help locating the offending character  (2004-08-06)
       http://python.org/sf/1004533  opened by  Michael Hudson

translate Windows cr-lf when installing scripts on Linux  (2004-08-06)
       http://python.org/sf/1004696  opened by  Tim Cera

extend strptime to understand logging timestamps  (2004-08-10)
       http://python.org/sf/1006786  opened by  Skip Montanaro

RFE Closed
__________

Adding missing ISO 8859 codecs, especially Thai  (2004-08-02)
       http://python.org/sf/1001895  closed by  lemburg

From mkc at mathdogs.com  Wed Aug 11 05:27:21 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Wed Aug 11 05:27:26 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<lkdwu0bcz7r.fsf@panix2.panix.com>
	<87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp>
	<lkdacx37oxr.fsf@panix2.panix.com> <cf9ot9$7bh$1@sea.gmane.org>
Message-ID: <lkdacx2xskm.fsf@panix2.panix.com>

"Fredrik Lundh" <fredrik@pythonware.com> writes:
> Mike Coleman wrote:
> > If I may wax anthropomorphic, the 're.match' function says to me as a programmer
> >
> >    *You* know what structure this RE represents, and *I* know what
> >    structure it represents, too, because I had to figure it out to
> >    do the match.
> 
> that only shows that you dont understand how regular expressions work.

Ouch.

> a regular expression defines a set of strings, and the RE engine is designed to
> tell you if a string you have is a member of this set.

Yes, I agree.  In particular, the re.match function "proves" that a particular
regular expression pattern matches a string or that it does not.  In the
process of doing that, each subexpression in the pattern must be matched
against some (possibly empty) substring of the string.  The re.match function
has all of this knowledge implicitly and ephemerally, and for parenthesized
groups, it even remembers this information and allows the caller access to it.

Now given all that, doesn't it seem odd that I can get complete information
about concatenated subexpressions

    >>> re.match(r'(...)(...)', 'abcdef').groups()
    ('abc', 'def')

and alternating subexpressions

    >>> re.match(r'(a..)|(z..)', 'abcdef').groups()
    ('abc', None)

but not repeating subexpressions

    >>> re.match(r'(..)*', 'abcdef').groups()
    ('ef',)

?  What's the logic of that?  That's the limitation I'd like to remove.

Now, I understand that doing this will be less efficient, which is why this is
a separate method and might not be able to share as much implementation with
re.match as would otherwise be desirable.  But I still think this is a gap
that really needs to be filled.

So, is my proposal the best way to do it?  Not necessarily.  I'd love to see a
better alternative.

> the engine's not a parser, and it has a very vague notion of "structure"
> (groups are implemented by "marks", which are registers that keeps track of
> where the engine last passed a given position; changing that to "lists of
> all possible matches" would require a major rewrite).

I was thinking that the modifications necessary wouldn't be that bad, since
the search algorithm itself is not being disturbed.  I could be wrong of
course.

(Note that we're not saving "all possible matches", just the multiple matches
involved in the successful match of a repeat group.)


> you're probably better off using the scanner mechanism:
> 
>     http://effbot.org/zone/xml-scanner.htm
> 
> or the sre.Scanner class (see the source code).  the scanner concept could
> need some documentation, and it would be nice to be able to switch patterns
> during parsing.  (time for a scanner PEP, anyone?)

I saw this but thought the code was dead.  An example or two of usage in a
docstring would be worth its weight in gold.  It looks like a useful
feature--I'd be +1 for it.

I'm not sure straightaway how to use the scanner mechanism to implement
re.structmatch, though.  I'll look at it.

Mike


From ncoghlan at iinet.net.au  Wed Aug 11 07:46:10 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Aug 11 07:46:23 2004
Subject: [Python-Dev] Removing ODBC references
In-Reply-To: <4114594E.2030606@v.loewis.de>
References: <4111DC2C.8070302@ocf.berkeley.edu>
	<41140C77.1040802@iinet.net.au> <4114594E.2030606@v.loewis.de>
Message-ID: <4119B2A2.6060902@iinet.net.au>

Martin v. L?wis wrote:
> Feel free to contribute patches. I think I already said that I see no
> problem with the Python script being distributed along with Python,
> provided its author(s) are willing to contribute it (it would be good
> if there was a single dedicated contact for maintaining it, though).
> 
> For the odbc references, please contribute patches to take the
> references out of the projects. We really don't need odbc in Python.

Done - posted to SF as Patch #1006916. Should definitely be checked by
someone with Visual Studio, since I just used sed to get rid of them.

Cheers,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA






From ncoghlan at iinet.net.au  Wed Aug 11 07:46:25 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Aug 11 07:46:35 2004
Subject: [Python-Dev] Re: re.split on empty patterns
In-Reply-To: <lkdk6w6xy0z.fsf@panix2.panix.com>
References: <20040807145142.GB2529@rogue.amk.ca>	<1f7befae040807105878b47c61@mail.gmail.com>	<cfa2qo$ru9$1@sea.gmane.org>
	<lkdk6w6xy0z.fsf@panix2.panix.com>
Message-ID: <4119B2B1.9020401@iinet.net.au>

Mike Coleman wrote:
> The "empty okay" concept, on the other hand, seems to have to do specifically
> with how re.split uses the pattern.  That is, in the process of finding
> matches decides (or not) to discard the empty matches.

Hmm, to me, this suggests 'split_empty = False' as your new parameter
for re.split.

Cheers,
Nick.



-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA


From ncoghlan at iinet.net.au  Wed Aug 11 07:47:20 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Aug 11 07:47:29 2004
Subject: [Python-Dev] Decorators with arguments are curries!
In-Reply-To: <5.1.1.6.0.20040807115729.028f3ce0@mail.telecommunity.com>
References: <5.1.1.6.0.20040807115729.028f3ce0@mail.telecommunity.com>
Message-ID: <4119B2E8.7020809@iinet.net.au>

Phillip J. Eby wrote:
> At 10:36 PM 8/7/04 +1000, Andrew Durdin wrote:
> 
>> def bar_decorator(func, param):
>>     print param
>>     return func
>>
> Decorator syntax does *not* provide currying.  You have to write 
> something like this:
> 
Or, if the 'partial' function is in the standard lib by that point, you
can use it to get your decorator. . .

X>>> @partial(bar_decorator(param_val))
X... def myFoo(): pass

Cheers,
Nick.

(P.S. Does the interactive interpreter currently do a line continuation
after the decorator line?)

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA




From ncoghlan at iinet.net.au  Wed Aug 11 07:52:08 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Aug 11 07:52:18 2004
Subject: [Python-Dev] Decorators with arguments are curries!
In-Reply-To: <4119B2E8.7020809@iinet.net.au>
References: <5.1.1.6.0.20040807115729.028f3ce0@mail.telecommunity.com>
	<4119B2E8.7020809@iinet.net.au>
Message-ID: <4119B408.2040700@iinet.net.au>

Nick Coghlan wrote:

> (P.S. Does the interactive interpreter currently do a line continuation
> after the decorator line?)

Never mind. It occurred to me that my build enivironment works now, so I 
just went and checked what current CVS does ;)

Cheers,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From stephen at xemacs.org  Wed Aug 11 08:06:31 2004
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed Aug 11 08:07:52 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae0408101258431039a1@mail.gmail.com> (Tim Peters's
	message of "Tue, 10 Aug 2004 15:58:22 -0400")
References: <000001c47e88$81ef1d40$5635c797@oemcomputer>
	<1f7befae0408092348583f0953@mail.gmail.com>
	<2mvffqzum9.fsf@starship.python.net>
	<1f7befae0408101258431039a1@mail.gmail.com>
Message-ID: <877js6gqe0.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Tim" == Tim Peters <tim.peters@gmail.com> writes:

    Tim> I guess you really can't spell Windows XP without Windows
    Tim> <0.9 wink>.

Gee, and on your consistent recommendation, I was just about to
upgrade from Linux and Mac OS X!<wink>

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.
From lists at hlabs.spb.ru  Wed Aug 11 13:31:58 2004
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Wed Aug 11 09:26:01 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <411774F2.7040408@interlink.com.au>
References: <41176596.5060209@hlabs.spb.ru>	<2m7js83733.fsf@starship.python.net>	<41179A6A.90905@hlabs.spb.ru>
	<411774F2.7040408@interlink.com.au>
Message-ID: <411A03AE.40606@hlabs.spb.ru>

Anthony Baxter wrote:
> Dmitry Vasiliev wrote:
> 
>>
>> Quote from PEP-237:
>>
>> """
>> A new type,
>> integer, may be introduced that is an abstract base type of
>> which both the int and long implementation types are
>> subclassed.  This is useful so that programs can check
>> integer-ness with a single test:
>>
>>      if isinstance(i, integer): ...
>> """
>>
>> So maybe correct question then: is there a plan for implementing the 
>> integer type in Python 2.4?
> 
> Well, we're a couple of weeks from what will hopefully be the last
> alpha. I'd _hope_ anything like this would be in before then (although
> we can have things that change behaviour up until b1, I'd prefer that
> a3->b1 is just fixing bugs that require changes, or minor changes, not
> major changes). OTOH, I'm not sure how deep a change like this would be.
> 
> In any case, if it's a change that you (or anyone else) feels strongly
> about, a patch on SF would be a first step towards making this happen.

So here it is:

http://sourceforge.net/tracker/index.php?func=detail&aid=1007068&group_id=5470&atid=305470

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru
From ncoghlan at iinet.net.au  Wed Aug 11 10:24:24 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Aug 11 10:24:36 2004
Subject: [Python-Dev] Request for bug closures
Message-ID: <4119D7B8.8030507@iinet.net.au>

There are a couple of bugs which can probably be closed:

#1006001 (Future warning in test_format.py):
   Looks like a problem with the user accidentally running the 2.4
regrtest.py with the 2.3 interpreter

#1006740 ('Memory leak' in long running console window):
   Looks like the user has encountered a Windows 'feature', not a Python
bug.

Cheers,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA





From stephen at xemacs.org  Wed Aug 11 10:51:29 2004
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed Aug 11 10:51:38 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured
	Regular Expression Group Matching
In-Reply-To: <lkdacx37oxr.fsf@panix2.panix.com> (Mike Coleman's message of
	"09 Aug 2004 21:38:08 -0400")
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<lkdwu0bcz7r.fsf@panix2.panix.com>
	<87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp>
	<lkdacx37oxr.fsf@panix2.panix.com>
Message-ID: <873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Mike" == Mike Coleman <mkc@mathdogs.com> writes:

    Mike> "Stephen J. Turnbull" <stephen@xemacs.org> writes:

    >> >>>>> "Mike" == Mike Coleman <mkc@mathdogs.com> writes:

    Mike> m0 = re.match(r'([A-Z]+|[a-z]+)*', 'XxxxYzz')

    >> Sure, but regexp syntax is a horrible way to express that.

    Mike> Do you mean, horrible compared to spelling it out using a
    Mike> Python loop that walks through the array,

I didn't have that in mind, but I can live with it.  Regexp doesn't
compare well even with that when you consider error-checking and
program maintenance.

    Mike> or horrible compared to some more serious parsing package?

That's more like it.  It doesn't have to be all that serious, either.

It's not obvious to me how to make grammar rules pretty in Python, but
implementing an SLR parser-generator should be at most a couple of
days' work to get something you could live with.

Note that one thing about the LR parser itself is that the stack
machine is quite generic; you can start with SLR, and go on to add
more powerful parser table generators later.

    Mike> I certainly wouldn't want to see someone try to write a
    Mike> language front-end with this, but for a lot of text-scraping
    Mike> activities, I think it would be very useful.

I agree it would be useful.  I think that a parser generator would be
equally useful for this, as easy to learn as regexps are (assuming
somebody comes up with a tasteful way to express grammar rules), and
more powerful with even a basic implementation.

BTW, do you have a sample implementation for re.structmatch?  Fredrik
seems to have some doubt about ease of implementation.

    >> This feature would be an attractive nuisance, IMHO.

    Mike> I agree that, like list comprehensions (for example), it
    Mike> needs to be applied with good judgement.

I don't see the analogy to list comprehensions.  My objection is not
that re.structmatch can express a request for a huge amount of data in
compact form.  If you've got the RAM, go right ahead.

My objection is that it throws away a lot of structure, and therefore
is liable to return the _wrong_ parse, or simply an error with no hint
as to where the data is malformed.  (AFAICS this is the practical
implication of Fredrik's "theoretical" statement "the RE engine is
designed to tell you if a string you have is a member of this set".
Yes, the Python re engine can provide some annotations for a well-formed
string, but in the end, well-formedness of the string is yes/no, with
no error diagnosis.)

Sure, there are applications like reading a matrix where the structure
is as uniform as the "*" operator implies.  But what do you get from
that?  A list of lists of _strings_ ... and you need a nested loop to
turn those strings into numbers that your linear algebra package can
use.  No?  I just don't see the benefits in code reduction you claim.

    Mike> Turning it around, though, why *shouldn't* there be a good
    Mike> mechanism for returning the multiple matches for multiply
    Mike> matching groups?

Because they're all too often not "multiple matches".  They're matches
for different objects that happen to match the same regular
expression, and the notation for the parser should express that
without requiring comments.  re.structmatch _encourages_ a style where
users deliberately throw away structure; your /etc/group parser is an
example.  (See below.)

Furthermore, since this is going to be recursive, users are going to
end up with "list of list of ..." nested to arbitrary depth, depending
on how complex the regexp is.  Isn't that what we have Lisp for? <wink>

    Mike> As things stand right now, though, it's a serious irritation
    Mike> that we have a standard mechanism that *almost* does this,
    Mike> but quits at the last moment.  If I may wax anthropomorphic,
    Mike> the 're.match' function says to me as a programmer

    Mike>     *You* know what structure this RE represents, and *I*
    Mike> know what structure it represents, too, because I had to
    Mike> figure it out to do the match.  But too bad, sucker, I'm not
    Mike> going to tell you what I found!

    Mike> Irritating as hell.

Problem is, you (for some values of "you") and the program both _know_
what structure it represents, but you have different structures in
mind.  This is a standard source of regexp bugs.  The structmatch
extension will exacerbate the problem.

I suppose you would be satisfied if you could represent

file := file line
line := group ':' passwd ':' number ':' any '\n'
group := '[A-Za-z0-9]+'
passwd := '[A-Za-z0-9]+'
number := '[0-9]+'
any := '.*'

by

'(([A-Za-z0-9]+:)*.*\n)*'

in which case I agree it's pretty harmless.  But note well, you've
already allowed arbitrary words instead of numbers for 'number', and
improving 'passwd' to check for the characteristic signature of a
crypted password or a "no password authentication" flag would be
annoying with re.structmatch, while it's trivial with a parser.

You say, "that's a matter of application of good judgement".  My
experience is that when it comes to regexps, application of good
judgement is pretty rare.  I have a lot of experience with parsing
done by regexp, none of it pleasant.  Have some pity on the c.l.py
regulars who will be debugging the monstrosities that some people will
come up with! <0.5 wink>

I grant that a full-blown parser generator is overkill for your target
set of applications.  But I stick with my judgement that regexps as
they are are an attractive nuisance, and re.structmatch would make the
situation worse in proportion to its usage.  I hope your proposal will
be superseded either by a scanner package, or even by a parser package.


-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.
From mwh at python.net  Wed Aug 11 13:22:25 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Aug 11 13:22:27 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <1092177771.21839.54.camel@localhost> (Barry Warsaw's message
	of "Tue, 10 Aug 2004 18:42:51 -0400")
References: <1092177771.21839.54.camel@localhost>
Message-ID: <2mr7qdzzpq.fsf@starship.python.net>

Barry Warsaw <barry@python.org> writes:

> Raymond and I had a chance to talk about PEP 292 on bug day, and we've
> exchanged a few private emails and implementations.  I think we've
> finally knocked everything into shape for inclusion in Python 2.4's next
> alpha.
>
> In the sandbox you will find the updated implementation, along with
> updated tests and documentation (libstring.tex).
>
> The updated PEP is at:
>
> http://www.python.org/peps/pep-0292.html

It's a silly point, but my instant impression on reading

    This PEP describes a simpler string substitution feature, also
    known as string interpolation.

is "simpler than *what*?".  Perhaps a better title for the PEP would
just be "simple string interpolation".

Otherwise, I like it.

The only issue I have with putting it in string is that this makes it
a touch harder to bundle up for use with scripts that work with older
Python's (I take it the implementation is pure Python and works or can
be fairly easily be made to work with 2.2?).  But I can live with

try:
    from string import template
except ImportError:
    from string_compat import template

Cheers,
mwh

-- 
  Exam invigilation - it doesn't come much harder than that, esp if
  the book you're reading turns out to be worse than expected.
                                  -- Dirk Bruere, sci.physics.research
From dima at trit.org  Wed Aug 11 13:24:41 2004
From: dima at trit.org (Dima Dorfman)
Date: Wed Aug 11 13:27:13 2004
Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2,
	and @decorators)
In-Reply-To: <200408092019.i79KJPw23893@guido.python.org>
References: <410DDFE3.7010006@interlink.com.au>
	<200408021850.i72IoCg28650@guido.python.org>
	<410EF963.40704@interlink.com.au> <20040809043104.GA82909@trit.org>
	<200408091525.i79FPLM23023@guido.python.org>
	<20040809173415.GB28961@panix.com>
	<200408092019.i79KJPw23893@guido.python.org>
Message-ID: <20040811112441.GH82909@trit.org>

Guido van Rossum <guido@python.org> wrote:
> > I'd suggest that only ``from foo import (...)`` be permitted (where
> > "..."  does not include ``*``).  But I don't care much, and I'll
> > update the PEP as soon as it's clear which precise versions are
> > allowed.
> 
> This seems to be the consensus so far, so why don't you add all that
> detail to the PEP.

I just uploaded the updated patch as SF #1007189. That version is
complete and, as agreed, only accepts the from/import form. It could
use a good review, but otherwise it should be ready to go.

Thanks,

Dima.
From fredrik at pythonware.com  Wed Aug 11 13:38:25 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed Aug 11 13:38:52 2004
Subject: [Python-Dev] Re: Update PEP 292
References: <1092177771.21839.54.camel@localhost>
	<2mr7qdzzpq.fsf@starship.python.net>
Message-ID: <cfd0fh$fut$1@sea.gmane.org>

Michael Hudson wrote:

> It's a silly point, but my instant impression on reading
>
>     This PEP describes a simpler string substitution feature, also
>     known as string interpolation.
>
> is "simpler than *what*?".

that's explained in the next sentence, of course.

    This PEP is "simpler" in two respects:
    [...]

> The only issue I have with putting it in string is that this makes it
> a touch harder to bundle up for use with scripts that work with older
> Python's (I take it the implementation is pure Python and works or can
> be fairly easily be made to work with 2.2?).

here's code that implements the "simpler" syntax, and works with 1.6
and later:

    http://effbot.org/zone/re-sub.htm#simple-templating

(from what I saw from the checkins, the PEP implementation isn't exactly
optimal, but I guess we can leave that for the next Optimization Day ;-)

</F>



From mwh at python.net  Wed Aug 11 13:40:09 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Aug 11 13:40:12 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <cfd0fh$fut$1@sea.gmane.org> (Fredrik Lundh's message of "Wed,
	11 Aug 2004 13:38:25 +0200")
References: <1092177771.21839.54.camel@localhost>
	<2mr7qdzzpq.fsf@starship.python.net> <cfd0fh$fut$1@sea.gmane.org>
Message-ID: <2mn011zyw6.fsf@starship.python.net>

"Fredrik Lundh" <fredrik@pythonware.com> writes:

> Michael Hudson wrote:
>
>> It's a silly point, but my instant impression on reading
>>
>>     This PEP describes a simpler string substitution feature, also
>>     known as string interpolation.
>>
>> is "simpler than *what*?".
>
> that's explained in the next sentence, of course.
>
>     This PEP is "simpler" in two respects:
>     [...]

Yes, I know.  It still annoyed me :-)

Cheers,
mwh

-- 
  ZAPHOD:  Listen three eyes, don't try to outwierd me, I get stranger
           things than you free with my breakfast cereal.
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 7
From fperez528 at yahoo.com  Wed Aug 11 14:06:12 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Wed Aug 11 14:06:23 2004
Subject: [Python-Dev] Re: Update PEP 292
References: <1092177771.21839.54.camel@localhost>
Message-ID: <cfd23o$lla$1@sea.gmane.org>

Barry Warsaw wrote:

> Raymond and I had a chance to talk about PEP 292 on bug day, and we've
> exchanged a few private emails and implementations.  I think we've
> finally knocked everything into shape for inclusion in Python 2.4's next
> alpha.
> 
> In the sandbox you will find the updated implementation, along with
> updated tests and documentation (libstring.tex).
> 
> The updated PEP is at:
> 
> http://www.python.org/peps/pep-0292.html

If I understood things correctly, this proposal will not allow the evaluation of
any kind of expression, including attributes.  With the existing interpolation
system, the following happens:

In [2]: print 'sys.platform is: %(sys.platform)s' % locals()
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)

?

KeyError: sys.platform

I understand why this is the case, but it has always struck me as an unpleasant
aspect of string interpolation, since it often forces me to create throw-away
locals when I want to print data stored as attributes of an object.  It is easy
to end up with code that looks like:

_x,_y,_z = self.x, self.y,self.z  # throwaways for printing

print """\
The coordinates are: 
x  =   %(_x)s
y  =   %(_y)s
z  =   %(_z)s """ % locals()

Simply because trying to do %(self.x)s won't work.  This just feels ugly, IMHO.

I wonder if it would be possible to at least allow this kind of expansion
(without going to the full-blown possibilities of PEP 215) in pep-292.  I know
there are tricks to produce this kind of evaluation already (which I sometimes
use).  But since pep-292 is trying to introduce a standard interpolation
mechanism which is easy to use for the common cases, and object attributes are
such a pervasive part of the language, I hope this addition can be considered.

Best,

f

ps. Forgive me if the above is already allowed; if that is the case, it might be
worth pointing it out in an example in the PEP.

From fperez528 at yahoo.com  Wed Aug 11 14:16:59 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Wed Aug 11 14:17:08 2004
Subject: [Python-Dev] Re: [SPAM: 3.611] IPython, @,
	and option E from the wiki
References: <cf17ic$83o$1@sea.gmane.org>
	<200408090202.i7922diB005813@cosc353.cosc.canterbury.ac.nz>
Message-ID: <cfd2nu$n2g$1@sea.gmane.org>

Greg Ewing wrote:

> 3. In this kind of setup, using | instead of @ would be ok as well, I guess:
> Fernando Perez <fperez528@yahoo.com>:
> 
>> def bar(low,high):
>>    |accepts(int,int)
>>    |returns(float)
> 
> How about
> 
>   def bar(low,high):
>     ...accepts(int,int)
>     ...returns(float)

The ellipsis is alredy valid python syntax elsewhere:

In [7]: a=arange(3**3)

In [8]: a.shape=(3,3,3)

In [9]: a[:,...,:]
Out[9]:
array([[[ 0,  1,  2],
        [ 3,  4,  5],
        [ 6,  7,  8]],
       [[ 9, 10, 11],
        [12, 13, 14],
        [15, 16, 17]],
       [[18, 19, 20],
        [21, 22, 23],
        [24, 25, 26]]])

I don't think it would be a good idea to make it do this kind of double-duty.

best,

f

From jlgijsbers at planet.nl  Wed Aug 11 14:17:27 2004
From: jlgijsbers at planet.nl (Johannes Gijsbers)
Date: Wed Aug 11 14:20:55 2004
Subject: [Python-Dev] Checkin Privileges (was: Re: Third Bug Day outcome)
References: <20040809213820.GC11199@rogue.amk.ca>
	<2mllgn15cl.fsf@starship.python.net>
Message-ID: <loom.20040811T124517-239@post.gmane.org>

Michael Hudson <mwh <at> python.net> writes:

> Well apart from jlgijsbers who you mentioned in your post 
> 
> I don't know how much longer we should let him get away with not being
> a committer, though <wink>.
> 
> Cheers,
> mwh

Not very long <wink>. Being a committer would be great, though how much time I
can spend on Python really depends on the rest of my life. Consider this a
request for checkin privileges.

Some of you may have seen me on the wiki or the pydotorg list, but a short
introduction seems in order. 

As you've probably noticed, my name is Johannes Gijsbers. I run a small company
with a friend; we use Python whenever we can. Also, I've just finished the first
year of my philosophy bachelor. This year the plan is not to continue studying
philosophy, but to work for six months and then to travel around in South
America until my money runs out. I'll see what happens after that.

I discovered Python in March 2001 and haven't really looked back since. I admire
other languages (e.g. Lisp, Smalltalk) in some ways, but I can't use them for
real work. I've been helping out with the wiki and pydotorg for some time now
and the Python Bug Days sucked me into helping out with Python itself as well.

Besides programming, my main interests are literature and its relation with
philosophy, juggling, communities in general and wiki culture in particular.

Johannes

From adurdin at gmail.com  Wed Aug 11 14:21:57 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Wed Aug 11 14:21:59 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <cfd23o$lla$1@sea.gmane.org>
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
Message-ID: <59e9fd3a040811052152548b6b@mail.gmail.com>

On Wed, 11 Aug 2004 06:06:12 -0600, Fernando Perez <fperez528@yahoo.com> wrote:
> Barry Warsaw wrote:
> 
> > Raymond and I had a chance to talk about PEP 292 on bug day, and we've
> > exchanged a few private emails and implementations.  I think we've
> > finally knocked everything into shape for inclusion in Python 2.4's next
> > alpha.
> >
> > In the sandbox you will find the updated implementation, along with
> > updated tests and documentation (libstring.tex).
> >
> > The updated PEP is at:
> >
> > http://www.python.org/peps/pep-0292.html

Forgive me for being obtuse, but I can't see anything significantly
simpler in this PEP than the standard string interpolation, except for
the case where the identifier doesn't need braces:

template("Hello, $name, how are you?") % {name: "Jim"}
versus
"Hello, %(name)s, how are you?") % {name: "Jim"}

Apart from removing the need for the parentheses/braces in this case,
all it does is consistently remove the "s", and replace the % with $,
and then limiting the interpolated values to strings (or will it
automatically use str(x) for any non-string variable x?).
Is there any other benefit to this that I'm overlooking?
From barry at python.org  Wed Aug 11 15:50:35 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 11 15:50:21 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <2mr7qdzzpq.fsf@starship.python.net>
References: <1092177771.21839.54.camel@localhost>
	<2mr7qdzzpq.fsf@starship.python.net>
Message-ID: <1092232235.9941.15.camel@localhost>

On Wed, 2004-08-11 at 07:22, Michael Hudson wrote:

> It's a silly point, but my instant impression on reading
> 
>     This PEP describes a simpler string substitution feature, also
>     known as string interpolation.
> 
> is "simpler than *what*?".  Perhaps a better title for the PEP would
> just be "simple string interpolation".

As others have pointed out, that's described in the text of the PEP.

> Otherwise, I like it.

Cool!

> The only issue I have with putting it in string is that this makes it
> a touch harder to bundle up for use with scripts that work with older
> Python's (I take it the implementation is pure Python and works or can
> be fairly easily be made to work with 2.2?).  But I can live with
> 
> try:
>     from string import template
> except ImportError:
>     from string_compat import template

To be brutally honest (about our work, that is ;), the template class is
so damn simple you could stick its definition in the except clause.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040811/3c44549a/attachment.pgp
From barry at python.org  Wed Aug 11 15:55:37 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 11 15:55:23 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <cfd23o$lla$1@sea.gmane.org>
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
Message-ID: <1092232536.9945.21.camel@localhost>

On Wed, 2004-08-11 at 08:06, Fernando Perez wrote:

> If I understood things correctly, this proposal will not allow the evaluation of
> any kind of expression, including attributes.  With the existing interpolation
> system, the following happens:
> 
> In [2]: print 'sys.platform is: %(sys.platform)s' % locals()
> ---------------------------------------------------------------------------
> KeyError                                  Traceback (most recent call last)
> 
> ?
> 
> KeyError: sys.platform

You don't mean the 'existing' interpolation system, because this
obviously doesn't work in Python today, although with a fancy enough
implementation of 'locals()', it could.  That actually doesn't change
much with PEP 292.  I specifically designed it so that you could
subclass from template, so as to accept the extended placeholder syntax
(i.e. dots), and then implement a mapping-like class to do the lookups
in any namespace you want.  In fact, my earlier implementation contained
such a class, but it was deemed too complex to win the name 'simpler' so
it was left out of the library.  It's a trivial amount of code and my
own personal use case has a need for such a feature.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040811/ca79d64c/attachment-0001.pgp
From mcherm at mcherm.com  Wed Aug 11 15:57:41 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Aug 11 15:56:37 2004
Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2,
	and @decorators)
Message-ID: <1092232661.411a25d55a567@mcherm.com>

My patch (attached) accepts
>
>   import (os, sys)
>   from sys import (stdin, stdout, stderr)
>   import (os)
>   from sys import (*)
>
> but rejects
>
>   from (sys) import stdin
>   import (os), (sys)
>   import (os,)
>
> Should any of those be allowed? Anything that I missed?

Others have already mentioned that "from sys import xxx, xxx, xxx"
is the case that badly needs line wrapping. I would, however, like
to suggest that (if it's easy to do) you allow a trailing comma when
parenthesees are used. Of course this looks silly:

   from mymodule import (xxx, xxx,
           xxx, xxx,)

But in the most extreme case, like this:

   from myBigModule import (
           xxxxxxxx, xxxxxxxx, xxxxxxxx,
           xxxxxxxx, xxxxxxxx, xxxxxxxx,
           xxxxxxxx, xxxxxxxx, xxxxxxxx,
           xxxxxxxx, xxxxxxxx, xxxxxxxx,
           xxxxxxxx, xxxxxxxx, xxxxxxxx,
           )

it's awefully nice on cut-and-paste to allow the trailing comma. But
the most important reason for doing so is that Python already allows
(and disregards) an extra trailing comma in similar lists:

    >>> [1,2,3,]
    [1, 2, 3]
    >>> {1:1,2:2,3:3,}
    {1: 1, 2: 2, 3: 3}
    >>> def func(a,b,c,):
    ...     print a, b, c
    ...
    >>> func(1,2,3)
    1 2 3
    >>> (1,2,3,)
    (1, 2, 3)

-- Michael Chermside

PS: The length-one-tuple is a special case (and a bit of a wart)
 in that it MANDATES the use of the trailing comma. But that's
 got nothing to do with the basic principle.

From barry at python.org  Wed Aug 11 15:58:03 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 11 15:57:49 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <59e9fd3a040811052152548b6b@mail.gmail.com>
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<59e9fd3a040811052152548b6b@mail.gmail.com>
Message-ID: <1092232683.9945.26.camel@localhost>

On Wed, 2004-08-11 at 08:21, Andrew Durdin wrote:

> Forgive me for being obtuse, but I can't see anything significantly
> simpler in this PEP than the standard string interpolation, except for
> the case where the identifier doesn't need braces:
> 
> template("Hello, $name, how are you?") % {name: "Jim"}
> versus
> "Hello, %(name)s, how are you?") % {name: "Jim"}
> 
> Apart from removing the need for the parentheses/braces in this case,
> all it does is consistently remove the "s", and replace the % with $,
> and then limiting the interpolated values to strings (or will it
> automatically use str(x) for any non-string variable x?).
> Is there any other benefit to this that I'm overlooking?

The elimination of the trailing 's' is definitely a benefit, as is the
adoption of $-placeholders, since that tends to be the placeholder
introducing character in many other languages.  In terms of usability,
it's a win.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040811/9ab236ac/attachment.pgp
From mcherm at mcherm.com  Wed Aug 11 16:18:47 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Aug 11 16:17:46 2004
Subject: [Python-Dev] Re: Decimal type question [Prothon]
Message-ID: <1092233927.411a2ac77a160@mcherm.com>

Mark Hahn writes:
> I am more interested now in hearing about people's feelings
> on the idea of
> having the Decimal type "in-between" Int and Float.  Assume a decent
> Decimal implementation is used.

All right. I think that "in-between" Int and Float is a bad idea -- or
at least that it's a bad way to think about a decent design.

"in-between" suggests that Float has greater range than Decimal which
has greater range than Int... but in your design that simply isn't
true. Your Int, for instance, can represent values that neither Decimal
nor Float can handle.

So the "right" way (IMHO) to think about it is as three *different*
classes of numbers. There are Ints (can only handle integers, but able
to manage any of those), Decimals (can only handle certain rational
numbers but it happens to be those that humans often find useful; also
has an upper bound and a precision limit (which may vary with size
depending on what implementation you use)), and Floats (can only handle
certain rational numbers most of which are NOT the same as those handled
by Decimal; also has upper bounds and precision limits; runs real fast).

I would recomend that you AVOID "automatic" conversion (just like
Python does) but that you make sure that when a (pure) number can be
represented in two ways that they compare equal. So the following would
be true:
    Int(1) == Decimal(1.0) == Float(1.e0)
    Decimal(2.5) == Float(2.5e0)

This is pretty much what you are already doing... for instance,
    Int(1) / Int(3) ---> Decimal(0.3333333333333333)
because there is a "/" operator which accepts two Ints or Decimals and
returns a Decimal. I'd propose that there's also another "/" operator
which accepts two Floats and returns a Float. And that passing one
Float and one (Int or Decimal) to "/" results in a type error. The
cos() function would accept one number (of any type) and return a
Float.

There's also the question (and it's an important one!) of what literals
to use. I agree (and everyone else would too) that literals like "1234"
should create Ints. I also agree (as would everyone else) that literals
like "123.45e6" should create Floats. The contraversial bit is whether
literals like "123.45" should create Decimals... but I think that they
should. My reasons are (1) for newbies, matching what people expect is
more important than speed... and that's exactly why Decimal is better
than Float, and (2) if not, then what form WOULD you use for Decimal
literals? (I suppose there's always the trailing "d".)




From amk at amk.ca  Wed Aug 11 16:39:20 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Wed Aug 11 16:39:50 2004
Subject: [Python-Dev] Request for bug closures
In-Reply-To: <4119D7B8.8030507@iinet.net.au>
References: <4119D7B8.8030507@iinet.net.au>
Message-ID: <20040811143920.GB18194@rogue.amk.ca>

On Wed, Aug 11, 2004 at 01:24:24AM -0700, Nick Coghlan wrote:
> There are a couple of bugs which can probably be closed:
> #1006001 (Future warning in test_format.py):
> #1006740 ('Memory leak' in long running console window):

I've closed both of these.

--amk
From guido at python.org  Wed Aug 11 16:46:17 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug 11 16:46:23 2004
Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2,
	and @decorators)
In-Reply-To: Your message of "Wed, 11 Aug 2004 06:57:41 PDT."
	<1092232661.411a25d55a567@mcherm.com> 
References: <1092232661.411a25d55a567@mcherm.com> 
Message-ID: <200408111446.i7BEkHI27825@guido.python.org>

> Others have already mentioned that "from sys import xxx, xxx, xxx"
> is the case that badly needs line wrapping. I would, however, like
> to suggest that (if it's easy to do) you allow a trailing comma when
> parentheses are used. [...]

+1

--Guido van Rossum (home page: http://www.python.org/~guido/)
From aahz at pythoncraft.com  Wed Aug 11 16:52:14 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed Aug 11 16:52:22 2004
Subject: [Python-Dev] Checkin Privileges (was: Re: Third Bug Day outcome)
In-Reply-To: <loom.20040811T124517-239@post.gmane.org>
References: <20040809213820.GC11199@rogue.amk.ca>
	<2mllgn15cl.fsf@starship.python.net>
	<loom.20040811T124517-239@post.gmane.org>
Message-ID: <20040811145214.GA20280@panix.com>

On Wed, Aug 11, 2004, Johannes Gijsbers wrote:
>
> I discovered Python in March 2001 and haven't really looked back
> since. I admire other languages (e.g. Lisp, Smalltalk) in some ways,
> but I can't use them for real work. I've been helping out with the
> wiki and pydotorg for some time now and the Python Bug Days sucked me
> into helping out with Python itself as well.

For those who don't translate automatically, "help with pydotorg" means
that he already has checkin privs on creosote (the machine that runs
www.python.org).
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From fperez528 at yahoo.com  Wed Aug 11 16:54:27 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Wed Aug 11 16:54:37 2004
Subject: [Python-Dev] Re: Re: Update PEP 292
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<1092232536.9945.21.camel@localhost>
Message-ID: <cfdbv6$pv5$1@sea.gmane.org>

Barry Warsaw wrote:

> On Wed, 2004-08-11 at 08:06, Fernando Perez wrote:
> 
>> If I understood things correctly, this proposal will not allow the evaluation
>> of
>> any kind of expression, including attributes.  With the existing
>> interpolation system, the following happens:
>> 
>> In [2]: print 'sys.platform is: %(sys.platform)s' % locals()
>> ---------------------------------------------------------------------------
>> KeyError                                  Traceback (most recent call last)
>> 
>> ?
>> 
>> KeyError: sys.platform
> 
> You don't mean the 'existing' interpolation system, because this
> obviously doesn't work in Python today, although with a fancy enough
> implementation of 'locals()', it could.  That actually doesn't change
> much with PEP 292.  I specifically designed it so that you could
> subclass from template, so as to accept the extended placeholder syntax
> (i.e. dots), and then implement a mapping-like class to do the lookups
> in any namespace you want.  In fact, my earlier implementation contained
> such a class, but it was deemed too complex to win the name 'simpler' so
> it was left out of the library.  It's a trivial amount of code and my
> own personal use case has a need for such a feature.

I guess my post wasn't worded too clearly.  I did mean existing, in the sense
that the error above occurs with current python (it's cut-and-paste from python
2.2).  My point was precisely to illustrate this failure with a simple example,
to then ask whether the new scheme could be made, _as shipped_, to accept this
kind of expansion:

print template('sys.platform is: $sys.platform') % locals()

It may be trivial to extend pep-292 for this, but not having to do it in every
small utility you write/ship would be very useful, IMHO.  If there are no
technical reasons to prevent it, is the 'too complex' argument really that
strong (especially since you state it's a trivial amount of code)?

I hope this clarifies better my question.

Regards,

f

From skip at pobox.com  Wed Aug 11 16:55:23 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Aug 11 16:55:41 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <1092232235.9941.15.camel@localhost>
References: <1092177771.21839.54.camel@localhost>
	<2mr7qdzzpq.fsf@starship.python.net>
	<1092232235.9941.15.camel@localhost>
Message-ID: <16666.13147.274810.410339@montanaro.dyndns.org>


    Barry> On Wed, 2004-08-11 at 07:22, Michael Hudson wrote:
    >> It's a silly point, but my instant impression on reading
    >> 
    >> This PEP describes a simpler string substitution feature, also
    >> known as string interpolation.
    >> 
    >> is "simpler than *what*?".  Perhaps a better title for the PEP would
    >> just be "simple string interpolation".

    Barry> As others have pointed out, that's described in the text of the
    Barry> PEP.

    >> Otherwise, I like it.

    Barry> Cool!

    ...

I would personally like to thank Barry for stepping up to the plate and
providing a much-needed distraction from the D-word. ;-)

Skip
From skip at pobox.com  Wed Aug 11 16:55:41 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Aug 11 16:56:28 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <DAE618D9A063D6BD6E7D58DC@[192.168.1.247]>
References: <200408051636.i75Gaak03654@guido.python.org>
	<20040805174252.GA27820@burma.localdomain>
	<20040805112318.B13062@ActiveState.com>
	<20040805183444.GA29796@burma.localdomain>
	<20040805190549.GC29796@burma.localdomain>
	<4112A18D.5090801@v.loewis.de>
	<20040805215331.GB31288@burma.localdomain>
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost>
	<1f7befae040806090644be9c04@mail.gmail.com>
	<cf0gd1$smv$1@sea.gmane.org> <1091818207.7529.34.camel@localhost>
	<4113D5E1.6040006@interlink.com.au>
	<41141255.5010408@stackless.com> <41175442.4010902@livinglogic.de>
	<16664.50665.170916.600964@montanaro.dyndns.org>
	<DAE618D9A063D6BD6E7D58DC@[192.168.1.247]>
Message-ID: <16666.13165.619920.756435@montanaro.dyndns.org>


    Andrew> Aren't we really trying to recreate def as a multi-line lambda
    Andrew> that binds when it's in a context that is not expecting a
    Andrew> function argument in block form?  

I suppose you can look at it that way.  The output of a function definition
plus any decorator applications to it need not be a function though.  I
posted a simpleminded example of generating a greyscale ramp using a
decorator.  If I thought about it for more than a few seconds I could
probably come up with some other examples as well where the output wasn't a
function.  Some of them might even be useful. ;-)

Skip

From edcjones at erols.com  Wed Aug 11 17:05:02 2004
From: edcjones at erols.com (Edward C. Jones)
Date: Wed Aug 11 17:11:07 2004
Subject: [Python-Dev] Semantics of decorators?
Message-ID: <411A359E.3080309@erols.com>

In the development docs, Python Reference Manual, 7.5, it says:

A function definition may be wrapped by one or more decorator expressions.
Decorator expressions are evaluated when the function is defined, in the
scope that contains the function definition. The result must be a callable,
which is invoked with the function object as the only argument. The returned
value is bound to the function name instead of the function object. If there
are multiple decorators, they are applied in reverse order.

Does this completely describe the semantics of decorators?

From amk at amk.ca  Wed Aug 11 17:13:47 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Wed Aug 11 17:14:18 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <1092177771.21839.54.camel@localhost>
References: <1092177771.21839.54.camel@localhost>
Message-ID: <20040811151347.GA18312@rogue.amk.ca>

On Tue, Aug 10, 2004 at 06:42:51PM -0400, Barry Warsaw wrote:
> The updated PEP is at:
> http://www.python.org/peps/pep-0292.html

Are you sure that people aren't going to want to format numbers
as they do with % (e.g. %4.2f)?  

You should post the PEP to c.l.py.announce, too.  (Perhaps you have
and it's still sitting in the queue.)

--amk
From John.Marshall at ec.gc.ca  Wed Aug 11 17:16:57 2004
From: John.Marshall at ec.gc.ca (John Marshall)
Date: Wed Aug 11 17:17:07 2004
Subject: [Python-Dev] A decorator syntax not yet mentioned (I think!)
Message-ID: <1092237417.22674.12.camel@mango.cmc.ec.gc.ca>

How about the following, which I am almost positive
has not been suggested:
-----
class Klass:
    def __init__(self, name):
        self.name = name

    deco meth0:
        staticmethod
    def meth0(x):
        return x
    
    deco meth1:
        classmethod
    def meth1(cls):
        return cls

    deco sayhello:
        funcattrs(name='GvR', language='python')
        log(file='func.log')
    def sayhello(self):
        print 'hello python world'

-----
1) The decorators clearly apply to a specific method/function,
   therefore there is no need to do any stack pushing in memory
   until the method/function definition is done.
2) The decorators are "outside" of the method/function they
   decorate:
   a) which will please those who want the outside location
   b) will not be folded within the function
   c) folding on the decorators can be done so that the
      def is not obfuscated
   d) can be located anywhere in the code--but most likely
      before the "def ...()"
3) The sequence in which the decorators are applied is just
   like code would be--this is certainly intuitive given
   that we are writing code.

This approach could also be applied to classes in case
decorators should ever be extended to them:
-----
deco Klass:
    doc("This is a class to ...")
class Klass:
    :
    :
-----

Any comments?

John

From edcjones at erols.com  Wed Aug 11 17:11:28 2004
From: edcjones at erols.com (Edward C. Jones)
Date: Wed Aug 11 17:17:32 2004
Subject: [Python-Dev] Long term plans for Python?
Message-ID: <411A3720.8050301@erols.com>

What are the long term plans for changes in Python from 2.5 to 3.0 and 
beyond?

From s.percivall at chello.se  Wed Aug 11 17:23:12 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Wed Aug 11 17:23:15 2004
Subject: [Python-Dev] Semantics of decorators?
In-Reply-To: <411A359E.3080309@erols.com>
References: <411A359E.3080309@erols.com>
Message-ID: <5BEEA740-EBAA-11D8-8A1D-0003934AD54A@chello.se>

On 2004-08-11, at 17.05, Edward C. Jones wrote:
> In the development docs, Python Reference Manual, 7.5, it says:
>
> A function definition may be wrapped by one or more decorator 
> expressions.
> Decorator expressions are evaluated when the function is defined, in 
> the
> scope that contains the function definition. The result must be a 
> callable,
> which is invoked with the function object as the only argument. The 
> returned
> value is bound to the function name instead of the function object. If 
> there
> are multiple decorators, they are applied in reverse order.
>
> Does this completely describe the semantics of decorators?

"applied in reverse order" might be a bit ambiguous. Reverse order
to what? To their definition?, to their proximity to the function?

//Simon

From s.percivall at chello.se  Wed Aug 11 17:30:04 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Wed Aug 11 17:30:08 2004
Subject: [Python-Dev] A decorator syntax not yet mentioned (I think!)
In-Reply-To: <1092237417.22674.12.camel@mango.cmc.ec.gc.ca>
References: <1092237417.22674.12.camel@mango.cmc.ec.gc.ca>
Message-ID: <51A1B43B-EBAB-11D8-8A1D-0003934AD54A@chello.se>

On 2004-08-11, at 17.16, John Marshall wrote:
> How about the following, which I am almost positive
> has not been suggested:
> -----
> class Klass:
>     def __init__(self, name):
>         self.name = name
>
>     deco meth0:
>         staticmethod
>     def meth0(x):
>         return x
>
>     deco meth1:
>         classmethod
>     def meth1(cls):
>         return cls
>
>     deco sayhello:
>         funcattrs(name='GvR', language='python')
>         log(file='func.log')
>     def sayhello(self):
>         print 'hello python world'
>
> Any comments?

There is the problem that you have to repeat the name of the function
you define. Repetition of this kind leads to error. It also takes more
place and distracts more from the actual function than most of the
other syntax. Also, there is the "new keyword" problem, leading to
breakage if older code uses the "deco" name.

//Simon

From John.Marshall at ec.gc.ca  Wed Aug 11 17:52:26 2004
From: John.Marshall at ec.gc.ca (John Marshall)
Date: Wed Aug 11 17:52:33 2004
Subject: [Python-Dev] A decorator syntax not yet mentioned (I think!)
In-Reply-To: <492EA476-EBAB-11D8-87F8-000A95686CD8@redivi.com>
References: <1092237417.22674.12.camel@mango.cmc.ec.gc.ca> 
	<492EA476-EBAB-11D8-87F8-000A95686CD8@redivi.com>
Message-ID: <1092239547.22674.39.camel@mango.cmc.ec.gc.ca>

On Wed, 2004-08-11 at 15:29, Bob Ippolito wrote:
> On Aug 11, 2004, at 11:16 AM, John Marshall wrote:
> 
> > How about the following, which I am almost positive
> > has not been suggested:
> > -----
> > class Klass:
> >     def __init__(self, name):
> >         self.name = name
> >
> >     deco meth0:
> >         staticmethod
> >     def meth0(x):
> >         return x
> >
> >     deco meth1:
> >         classmethod
> >     def meth1(cls):
> >         return cls
> >
> >     deco sayhello:
> >         funcattrs(name='GvR', language='python')
> >         log(file='func.log')
> >     def sayhello(self):
> >         print 'hello python world'
> >
> > -----
> > 1) The decorators clearly apply to a specific method/function,
> >    therefore there is no need to do any stack pushing in memory
> >    until the method/function definition is done.
> 
> None of that code can actually execute until the method/function 
> definition is done, how is this different at all?
> 

I am not refering to actually executing the code but
addressing people's point that when they read through
the code, they need to keep all the decorator items in
memory until they come to the "def ...". One person
described the process as push each decorator onto
a stack (in his head) until he found the def.

> > 2) The decorators are "outside" of the method/function they
> >    decorate:
> >    a) which will please those who want the outside location
> >    b) will not be folded within the function
> 
> Doesn't a) imply b)?
> 

Sure. But this addresses two concerns explicitly.

> >    c) folding on the decorators can be done so that the
> >       def is not obfuscated
> 
> This could probably be done with the @syntax too if the editor was 
> smart enough

I imagine it is possible. But the point was to make
the signature clearly visible/prominent but also indicate
that there _are_ decorators associated with the method/function.

> 
> >    d) can be located anywhere in the code--but most likely
> >       before the "def ...()"
> 
> I don't think that is a bonus at all.
> 

I am not claiming this as a bonus. Rather, I am making it clear
that the mention of the method/function name does not require
any special/forced placement.

> > 3) The sequence in which the decorators are applied is just
> >    like code would be--this is certainly intuitive given
> >    that we are writing code.
> 
> That's not really obvious.
> 

I am not claiming that it is obvious but that it could be
understood that way (rather than leave it to interpretation,
I made it explicit as an option). Note how the sequence issue
has come up recently with respect to the @-syntax and the list
syntax, which apparently have been understood to apply in the
opposite order.

> > This approach could also be applied to classes in case
> > decorators should ever be extended to them:
> 
> So can any of the other proposals.

That may be, but to be able to do so this way makes it
an viable option.

> 
> > -----
> > deco Klass:
> >     doc("This is a class to ...")
> > class Klass:
> >     :
> >     :
> > -----
> >
> > Any comments?
> 
> Yeah, this is no less than twice as bad as any of the current decorator 
> proposals because you have to repeat the function name :)
> 

Personally, I might opt for something else, but having the
decorators outside have raised concerns for some. With this
approach the tagging makes things explicit, which is often
a good thing. The current @-syntax _implicitly_ identifies
the decorators with the method/function/class which people
have said they don't like.

Thanks for the comments.

John

> -bob


From pje at telecommunity.com  Wed Aug 11 18:22:04 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Aug 11 18:22:09 2004
Subject: [Python-Dev] Semantics of decorators?
In-Reply-To: <411A359E.3080309@erols.com>
Message-ID: <5.1.1.6.0.20040811121756.039ef9e0@mail.telecommunity.com>

At 11:05 AM 8/11/04 -0400, Edward C. Jones wrote:
>In the development docs, Python Reference Manual, 7.5, it says:
>
>A function definition may be wrapped by one or more decorator expressions.
>Decorator expressions are evaluated when the function is defined, in the
>scope that contains the function definition. The result must be a callable,
>which is invoked with the function object as the only argument. The returned
>value is bound to the function name instead of the function object. If there
>are multiple decorators, they are applied in reverse order.
>
>Does this completely describe the semantics of decorators?

It should also be noted that when the decorator expressions are applied, 
the function has not yet been bound to its name in the appropriate 
scope.  Thus, if there is a previous binding of that name in the scope, the 
decorator expression may be able to access it.

This aspect of the semantics is important for additive decorators like Ed 
Loper's 'property_getter' and 'property_setter', or my generic function 
decorators.  E.g.:

     @property_getter
     def foo(self):
         return self.__foo

     @property_setter
     def foo(self,value):
         self.__foo = value

     # etc.




>_______________________________________________
>Python-Dev mailing list
>Python-Dev@python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: 
>http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com

From nbastin at opnet.com  Wed Aug 11 19:48:42 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Wed Aug 11 19:49:05 2004
Subject: [Python-Dev] 2.3.4 on Win64?
Message-ID: <AF1D4B40-EBBE-11D8-AAEA-000D932927FE@opnet.com>

Has anybody built 2.3.4 on Win64, and if so, is there a preferred way?  
We built it by exporting the makefiles and twiddling with the linker 
options and pointer size in pyconfig.h, but that seems a bit kludgy.

--
Nick

From martin at v.loewis.de  Wed Aug 11 20:20:43 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 11 20:20:38 2004
Subject: [Python-Dev] 2.3.4 on Win64?
In-Reply-To: <AF1D4B40-EBBE-11D8-AAEA-000D932927FE@opnet.com>
References: <AF1D4B40-EBBE-11D8-AAEA-000D932927FE@opnet.com>
Message-ID: <411A637B.6080109@v.loewis.de>

Nick Bastin wrote:
> Has anybody built 2.3.4 on Win64, and if so, is there a preferred way?  
> We built it by exporting the makefiles and twiddling with the linker 
> options and pointer size in pyconfig.h, but that seems a bit kludgy.

I did (assuming you mean Itanium). I first tried it with that approach,
then I talked Peter Tr?ger into developing vsextcomp, which is now used
in the CVS. So unless you have a specific reason to use 2.3.4, I
recommend to use 2.4.

If I had to do it again, I would use a vsextcomp procedure now: Import
the VC6 project files into VC7, then, using vsextcomp, generate Itanium
targets for them.

Regards,
Martin

P.S. sf.net/projects/vsextcomp
From nbastin at opnet.com  Wed Aug 11 20:26:33 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Wed Aug 11 20:26:56 2004
Subject: [Python-Dev] 2.3.4 on Win64?
In-Reply-To: <411A637B.6080109@v.loewis.de>
References: <AF1D4B40-EBBE-11D8-AAEA-000D932927FE@opnet.com>
	<411A637B.6080109@v.loewis.de>
Message-ID: <F8C1D87A-EBC3-11D8-AAEA-000D932927FE@opnet.com>


On Aug 11, 2004, at 2:20 PM, Martin v. L?wis wrote:

> Nick Bastin wrote:
>> Has anybody built 2.3.4 on Win64, and if so, is there a preferred 
>> way?  We built it by exporting the makefiles and twiddling with the 
>> linker options and pointer size in pyconfig.h, but that seems a bit 
>> kludgy.
>
> I did (assuming you mean Itanium). I first tried it with that approach,
> then I talked Peter Tr?ger into developing vsextcomp, which is now used
> in the CVS. So unless you have a specific reason to use 2.3.4, I
> recommend to use 2.4.

I forgot to mention that the platform in question is AMD64, although we 
will be building on Itanium as well.  Using 2.4 is not an option at 
this point.

--
Nick
From fredrik at pythonware.com  Wed Aug 11 21:16:58 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed Aug 11 21:15:19 2004
Subject: [Python-Dev] Re: Re: Update PEP 292
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<1092232536.9945.21.camel@localhost>
Message-ID: <cfdr83$niv$1@sea.gmane.org>

Barry Warsaw wrote:

> I specifically designed it so that you could subclass from template, so as to
> accept the extended placeholder syntax (i.e. dots), and then implement a
> mapping-like class to do the lookups in any namespace you want.

on the other hand, to do that, you need to replace the pattern, and
make sure that you have the same number of groups, with the same
meaning.  it's doable, but it's rather fragile.

two suggestions:

- refactor the RE, and use lastindex to extract the name (see the link
  I posted earlier).  This would let users add any number of patterns,
  without having to rewrite the __mod__ method.  And indexing by
  lastindex should be faster than tuple unpacking, too.

- Consider changing the ${} to accept arbitrary patterns, instead of
  just Python-style identifiers.  \${([^}]+)} should do the trick.

- Consider using character classes instead of re.IGNORECASE (easier
  to read, and IIRC, slightly faster).

</F> 



From mark at prothon.org  Wed Aug 11 20:53:21 2004
From: mark at prothon.org (Mark Hahn)
Date: Wed Aug 11 21:20:39 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
References: <1092233927.411a2ac77a160@mcherm.com>
Message-ID: <xucoxifr6vfp.1klhrdy4enc6z.dlg@40tude.net>

Michael Chermside wrote:

>Mark Hahn writes:
>> I am more interested now in hearing about people's feelings on the
>> idea of having the Decimal type "in-between" Int and Float.  
>Assume a
>> decent Decimal implementation is used.
>
>All right. I think that "in-between" Int and Float is a bad
>idea -- or at least that it's a bad way to think about a decent design.
>
>"in-between" suggests that Float has greater range than
>Decimal which has greater range than Int... but in your design 
>that simply isn't true. Your Int, for instance, can represent 
>values that neither Decimal nor Float can handle.

What do you think of the newer idea of using the IBM floating decimal for
all numerics?  Tim Peters and others pushed this on me.  

Performance would be handled somewhat by automatically switching between
ints, bigints,  and decimal internally, but with no loss of precision
except when a divide or other such operation happens.  I would still have a
hardware based binary floating point type that you have to manually specify
if you want hardware performance for floats.

I am thinking of having all current constant formats default to the
"Decimal" type (0, 123, 123.456, 123e3) and have a new constant type using
the letter 'f' for the binary "Float" type (0f0, 123f0, 123.456f0, 123.f3).

From mzarate at uoguelph.ca  Tue Aug 10 18:55:19 2004
From: mzarate at uoguelph.ca (Martin Zarate)
Date: Wed Aug 11 21:43:06 2004
Subject: [Python-Dev] Another approach to decorators.
Message-ID: <1092156919.4118fdf7b71c4@webmail.uoguelph.ca>

Hello.  I've already thrown a few cents into this debate, and have read every 
post on it.  I've used Ruby, so I love being able to make blocks for whatever 
I want - which is why Python's decorator syntax interests me.  Its not as 
powerful as Ruby blocks, but its a close approach for some things.  I can see 
the problems are legion.  I think the fundamental difficulty is that Python is 
line-oriented, and decorators break this - they either make a gigantic line 
that the coder remakes as they see fit (very unpython) or a multi-line mess, 
which Python does not have for any statements.  Second is that Python is an 
extremely legible language, and most decorator syntaxes are not.

@ means nothing to an uninformed eye.  This violates the most important 
feature of Python (imho) which is that it is "runnable pseudocode".  A coder 
will see what "def" or "class" or "import" means just by looking at it.  @ is 
meaningless.

I submit that a reserved keyword would fit much better in the @ place.  
Second, that comma-based delimiting (like the global statement) would also be 
nicer, to avoid line-after-line of decoration.

For example, lets say our keyword is "remake" - a bad word, a better one 
should be used, but I'm tired of thinking of one.

remake functor(foo, bar)
remake staticmethod
def baz():
     pass

or, alternately,

remake functor(foo,bar), staticmethod
def baz():
      pass

but either way, the system doesn't look very Python, as you've a multi-line 
statement.

I submit that the most Python solution, that would also be legible (but not 
necessarily very pretty) would be to actually make the decorator a bona-fide 
block that you nest your class or function into.

remake functor(foo, bar), staticmethod:
     def baz():
           pass

This concretely shows the relationship of the original define statement to its 
wrapper objects.  The principle is simple - the remake block will close with 
only one object in its immediate namespace, and that object will be wrapped by 
each of its argument objects in turn, and place it in its surrounding 
namespace with the same name.  Thus, more general-purpose oddball approaches 
could be used, such as, lambdas or even rewrapping an external object.  For 
example, in order to import a C-based extension module as a member method in a 
class...

class foo(number):
      remake instancemethod:
            sin = math.sin

bar = foo()
num = bar.sin

or something like that.

other good keywords that could be used instead of remake: wrap, decorate, as, 
etc.  I'm sure you can all think of something better than @.
From barnesc at engr.orst.edu  Wed Aug 11 01:12:58 2004
From: barnesc at engr.orst.edu (barnesc@engr.orst.edu)
Date: Wed Aug 11 21:43:09 2004
Subject: [Python-Dev] Decorators after 'def' should be reconsidered
Message-ID: <1092179578.4119567af011d@webmail.oregonstate.edu>


>In the discussion on decorators months ago, solutions involving
>special syntax inside the block were ruled out early on.  Basically,
>you shouldn't have to peek inside the block to find out important
>external properties of the function.

>--Guido van Rossum (home page: http://www.python.org/~guido/)

I disagree.

The most important part of a function is the first def line.

If you place other stuff before the def line, you make it
harder to grep for the def line.


Compare:

-----------------------------------------------
Option A
-----------------------------------------------

"""
Hi,

Performs a binary operation.

Docstring.
"""
@staticmethod
@synchronized(lockvar)
def f(a, b):
    return a + b

"""
Now,

Performs a simple operation.
"""
@classmethod
def g(a):
    return a

-----------------------------------------------
Option B
-----------------------------------------------

def f(a, b):
    @staticmethod
    @synchronized(lockvar)
    """
    Hi,

    Performs a binary operation.

    Docstring.
    """
    return a + b

def g(a):
    @classmethod
    """
    Now,

    Performs a simple operation.
    """
    return a


Now, forget everything you've learned about Python.
Forget that @ symbols are ugly.

 - For which method is it visually easier to find the function def?

 - For which method is the code in the most logical order?

Note that Option A causes you to skim all the way through
the docstring and the decorators to find out which function
is being defined.  At this point, you have to start over
at the docstring, to find out what the function actually does.

Option A is especially painful when a large number of functions
are thrown into a module.  It is almost impossible to grep
efficiently for the function names.  Thus the code appears
disorganized, and messy.

It seems strange that Option B was "ruled out early on."
Option B is clearly more readable.

 - Connelly
From bokr at oz.net  Wed Aug 11 21:26:46 2004
From: bokr at oz.net (Bengt Richter)
Date: Wed Aug 11 21:43:13 2004
Subject: [Python-Dev] deco_fun_expr(= vs. @deco_fun_expr
Message-ID: <411a5943.561998230@news.oz.net>

Just wanted to make sure this got into python-dev as an alternative ;-)
Example using post-fixed '(=' instead of prefixed '@':

deco1(=
decomodule.deco2(=
def foo():
    pass

effect is same as with prefixed-@ syntax, i.e., foo = deco1(decomodule.deco2(foo))
(except either way there's no intermediate foo binding during evaluation, I gather?)

Advantages:
1. currently illegal syntax, UIAM
2. lets callable-returning expression have unchanged grammar up to new (= trailer part
2. doesn't use up '@' or interfere with tools that use it
3. expresses calling in the proper nested way (inner before outer). No confusion about "backwards."
4. Leaves the door open to callable-calls operating on other code-block-derived
   arguments than from def and its suite (e.g. class and suite). Leaves door open for explicit
    =) end-of-block-arg mark if that should turn out useful for demarcation of otherwise
    ambiguous block endings, for some future use.
   
Unfortunately vexing personal matters will demand priority attention for some time again :-(
Bye for now.

Regards,
Bengt Richter
From jack at performancedrivers.com  Wed Aug 11 21:44:44 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Wed Aug 11 21:44:49 2004
Subject: [Python-Dev] pep 318, Decorators for Functions,
	Methods and Classes
In-Reply-To: <20040811011116.GF23725@performancedrivers.com>
References: <A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org>
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
	<200408072105.i77L5Ea10965@guido.python.org>
	<20040811011116.GF23725@performancedrivers.com>
Message-ID: <20040811194444.GG23725@performancedrivers.com>

On Tue, Aug 10, 2004 at 09:11:16PM -0400, Jack Diederich wrote:
> On Sat, Aug 07, 2004 at 02:05:14PM -0700, Guido van Rossum wrote:
> > > I just wanted to say that I believe it should be allowed to decorate
> > > classes. There's not reason enough (reading the thread linked to by the
> > > PEP) to limit decorators this way. Like said several times in that
> > > thread, metaclasses are harder (to write and to understand) than
> > > decorators.
> > 
> > Are you volunteering to implement it?
> >  
> I'm game, I would prefer to use class decorators in most of the places
> I currently use metaclasses.

I have a mostly working patch for class decorators but it gets an XXX ambiguity
between

funcdef: [decorators] 'def' NAME parameters ':' suite
and
classdef: [decorators] 'class' NAME [ '(' testlist ')' ] ':' suite

It works as advertised with the pre-decorators funcdef definition (which removes the ambiguity).
Help?

-Jack

# --- quicktst.py ----
class Tom(object): pass

def decor(cls):
  print "decord'd", cls
  return Tom

@decor
class Dick(object):
  """ some classs """
  def bar(self):
    print "BAR!"
    return

@decor
class Harry:
  def bar(self):
    print "BAR!"
    return

print Dick
print Dick.__class__
print Harry
print Harry()
print Harry().bar()


./python
Python 2.4a2 (#13, Aug 11 2004, 15:25:00) 
[GCC 3.3.4 (Debian 1:3.3.4-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import quicktst 
decord'd <class 'tst.Dick'>
decord'd tst.Harry
<class 'tst.Tom'>
<type 'type'>
<class 'tst.Tom'>
<tst.Tom object at 0x402c23ec>
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "tst.py", line 25, in ?
    print Harry().bar()
AttributeError: 'Tom' object has no attribute 'bar'
>>> 
From python at rcn.com  Wed Aug 11 21:48:20 2004
From: python at rcn.com (Raymond Hettinger)
Date: Wed Aug 11 21:49:07 2004
Subject: [Python-Dev] Checkin Privileges (was: Re: Third Bug Day outcome)
In-Reply-To: <20040811145214.GA20280@panix.com>
Message-ID: <002f01c47fdc$290bfa40$e622c797@oemcomputer>

> On Wed, Aug 11, 2004, Johannes Gijsbers wrote:
> >
> > I discovered Python in March 2001 and haven't really looked back
> > since. I admire other languages (e.g. Lisp, Smalltalk) in some ways,
> > but I can't use them for real work. I've been helping out with the
> > wiki and pydotorg for some time now and the Python Bug Days sucked
me
> > into helping out with Python itself as well.
> 
> For those who don't translate automatically, "help with pydotorg"
means
> that he already has checkin privs on creosote (the machine that runs
> www.python.org).

Does that exempt him from the rule that the new guy has to fix bugs and
buy all the current committers a round of beer?


Raymond

From raymond.hettinger at verizon.net  Wed Aug 11 21:51:37 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed Aug 11 21:52:27 2004
Subject: [Python-Dev] Naming nit
Message-ID: <003401c47fdc$a0f722a0$e622c797@oemcomputer>

For PEP 292, should the class be Template() or template()?
 
 
Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040811/8649f842/attachment.htm
From pje at telecommunity.com  Wed Aug 11 21:54:23 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Aug 11 21:54:36 2004
Subject: [Python-Dev] pep 318, Decorators for Functions, Methods
	and Classes
In-Reply-To: <20040811194444.GG23725@performancedrivers.com>
References: <20040811011116.GF23725@performancedrivers.com>
	<A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org>
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
	<200408072105.i77L5Ea10965@guido.python.org>
	<20040811011116.GF23725@performancedrivers.com>
Message-ID: <5.1.1.6.0.20040811155219.02a56a00@mail.telecommunity.com>

At 03:44 PM 8/11/04 -0400, Jack Diederich wrote:
>I have a mostly working patch for class decorators but it gets an XXX 
>ambiguity
>between
>
>funcdef: [decorators] 'def' NAME parameters ':' suite
>and
>classdef: [decorators] 'class' NAME [ '(' testlist ')' ] ':' suite
>
>It works as advertised with the pre-decorators funcdef definition (which 
>removes the ambiguity).
>Help?


How about this?

     decorated_assignment: [decorators] (funcdef | classdef)
     classdef: 'class' NAME [ '(' testlist ')' ] ':' suite
     funcdef: 'def' NAME parameters ':' suite


From bac at OCF.Berkeley.EDU  Wed Aug 11 21:55:22 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Aug 11 21:55:35 2004
Subject: [Python-Dev] Naming nit
In-Reply-To: <003401c47fdc$a0f722a0$e622c797@oemcomputer>
References: <003401c47fdc$a0f722a0$e622c797@oemcomputer>
Message-ID: <411A79AA.4020505@ocf.berkeley.edu>

Raymond Hettinger wrote:

> For PEP 292, should the class be Template() or template()?
>  

Template as per PEP 8.

-Brett
From martin at v.loewis.de  Wed Aug 11 22:04:06 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 11 22:04:02 2004
Subject: [Python-Dev] 2.3.4 on Win64?
In-Reply-To: <F8C1D87A-EBC3-11D8-AAEA-000D932927FE@opnet.com>
References: <AF1D4B40-EBBE-11D8-AAEA-000D932927FE@opnet.com>	<411A637B.6080109@v.loewis.de>
	<F8C1D87A-EBC3-11D8-AAEA-000D932927FE@opnet.com>
Message-ID: <411A7BB6.4040909@v.loewis.de>

Nick Bastin wrote:
> I forgot to mention that the platform in question is AMD64, although we 
> will be building on Itanium as well.  Using 2.4 is not an option at this 
> point.

I'd still recommend vsextcomp. Copy the 2.4 project files to 2.3, and 
replace all occurrences of MS_ITANIUM with MS_OPTERON. Alternatively,
create new AMD64 configurations, and contribute that to Python 2.4,
then use the project files on 2.3.

Regards,
Martin
From bac at OCF.Berkeley.EDU  Wed Aug 11 22:03:54 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Aug 11 22:04:12 2004
Subject: [Python-Dev] Drop support for multiple decorators per line?	(was:
	Decorator order implemented backwards?)
In-Reply-To: <1092175256.21841.20.camel@localhost>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au>	<1092172040.1681.9.camel@localhost>	<B90C2588-EB13-11D8-8849-000A95686CD8@redivi.com>
	<1092175256.21841.20.camel@localhost>
Message-ID: <411A7BAA.80703@ocf.berkeley.edu>

Barry Warsaw wrote:

> On Tue, 2004-08-10 at 17:24, Bob Ippolito wrote:
> 
>>On Aug 10, 2004, at 5:07 PM, Mark Russell wrote:
>>
>>
>>>While I'm at it, do we want to drop support for multiple decorators on 
>>>a
>>>single line?  Nobody has spoken up for it, and in fact forcing
>>>one-per-line would simplify the grammar (as well as making it easier 
>>>for
>>>toy parsers to find decorators).
>>
>>+1
> 
> 
> +1
> 

+1 = +3 from me!  =)

trying-to-get-through-over-1,000-emails-for-the-next-summary-yr's, Brett

From martin at v.loewis.de  Wed Aug 11 22:11:37 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 11 22:11:32 2004
Subject: [Python-Dev] Another approach to decorators.
In-Reply-To: <1092156919.4118fdf7b71c4@webmail.uoguelph.ca>
References: <1092156919.4118fdf7b71c4@webmail.uoguelph.ca>
Message-ID: <411A7D79.1040302@v.loewis.de>

Martin Zarate wrote:
> @ means nothing to an uninformed eye.  This violates the most important 
> feature of Python (imho) which is that it is "runnable pseudocode".  A coder 
> will see what "def" or "class" or "import" means just by looking at it.  @ is 
> meaningless.

I think it is irrelevant that @ means nothing in this context. Python 
will still be runnable pseudo-code. Remember that the decorator will read

   @staticmethod

So the eye unfamiliar with Python may wonder what the @ means. As you 
suggest, it means nothing, and the reader will see the staticmethod 
after it, and guess (correctly): "this somehow make a static method".

> I submit that a reserved keyword would fit much better in the @ place.  

To the uninformed eye, this would be harmful. It may have a grasp what a 
static method is, and that you have to declare it. However, what does 
the additional keyword mean? The uninformed eye will try to put a 
meaning into it, but there is none. The unformed eye will never guess 
how static methods work in Python - that the method is actually an 
object, that "staticmethod" is actually a type, and this all is a 
constructor call, assigning to the name of the function.

> I'm sure you can all think of something better than @.

Actually, I can't. Your post just convinced me that a keyword can never
be better.

Regards,
Martin
From michel at dialnetwork.com  Wed Aug 11 22:37:11 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Wed Aug 11 22:42:15 2004
Subject: [Python-Dev] Another approach to decorators.
In-Reply-To: <20040811195454.9BD311E400A@bag.python.org>
References: <20040811195454.9BD311E400A@bag.python.org>
Message-ID: <20040811133711.5f1986b6.michel@dialnetwork.com>


> Message: 2
> Date: Tue, 10 Aug 2004 12:55:19 -0400
> From: Martin Zarate <mzarate@uoguelph.ca>
> Subject: [Python-Dev] Another approach to decorators.
> To: python-dev@python.org
> Message-ID: <1092156919.4118fdf7b71c4@webmail.uoguelph.ca>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> @ means nothing to an uninformed eye.  This violates the most important 
> feature of Python (imho) which is that it is "runnable pseudocode".  

Yes!

> I submit that the most Python solution, that would also be legible (but not 
> necessarily very pretty) would be to actually make the decorator a bona-fide 
> block that you nest your class or function into.
> 
> remake functor(foo, bar), staticmethod:
>      def baz():
>            pass
> 
> This concretely shows the relationship of the original define statement to its 
> wrapper objects.  The principle is simple - the remake block will close with 
> only one object in its immediate namespace

I like your idea a lot, buy why just one?  Your scheme of making decorators a
block could be applied to several methods in a class:

class Foo:

  decorate static:

    def static1(blah...):
        pass

    def static2(blah...):
        pass

To me, this idea looks more like Python than all the rest, and allows
you to consolidate related decorated methods and classes.  Nest them to
apply decorators "aquisition style":


class Foo:

  decorate static:

    def static1(blah...):
        pass

    decorate locksFile:

      def static2andLocks(blah...):  # both decorators appy
          pass

-Michel
From mcherm at mcherm.com  Wed Aug 11 22:53:31 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Aug 11 22:52:24 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
Message-ID: <1092257611.411a874b5094e@mcherm.com>

> the only reason programmers write loops as integers is
> because accumulating floats "doesn't work".  With Decimal, that
> is no longer the case.

That is completely untrue. Binary floating point does just as
good a job of handling integers as decimal floating point does.
For instance:

    >>> accumulator = 0.0
    >>> for i in range(25000):
    ...     accumulator += 1.0
    ...
    >>> print accumulator
    25000.0

Notice, perfect accuracy. Yet people don't tend to use floating
point for loops... why not?

I'd argue there are at LEAST two other good reasons. Performance
(floating point arithmatic is fast, but the equivalent integer
arithmatic is normally faster), and clarity of intent (using an
int type makes it clear that you intend only to use integers.

But lots of languages make other choices. The number of languages
that have a single numeric type is pretty large... perl comes to
mind, but I bet I could list 10 others without too much trouble.

-- Michael Chermside

From mcherm at mcherm.com  Wed Aug 11 22:55:39 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Aug 11 22:54:32 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
Message-ID: <1092257739.411a87cb46d65@mcherm.com>

> What do you think of the newer idea of using the IBM floating
> decimal for
> all numerics?  Tim Peters and others pushed this on me.
   [...]
> I am thinking of having all current constant formats default to the
> "Decimal" type (0, 123, 123.456, 123e3) and have a new
> constant type using
> the letter 'f' for the binary "Float" type (0f0, 123f0,
> 123.456f0, 123.f3).

+1

-- Michael Chermside

From mcherm at mcherm.com  Wed Aug 11 23:02:26 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Aug 11 23:01:18 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
Message-ID: <1092258146.411a896246d1a@mcherm.com>

> > Is there a plan for implementing a base class for int and long (like
> > basestring for str and unicode):
> >
> >  >>> issubclass(int, baseint) and issubclass(long, baseint)
> > True
> >
> > ?

> I think this would be a good idea; maybe the name should be
> baseinteger?

I would like to urge caution before making this change. Despite
what the PEP may say, I actually think that creating a 'baseint'
type is the WRONG design choice for the long term. I envision
an eventual Python which has just one type, called 'int'. The
fact that an efficient implementation is used when the ints
are small and an arbitrary-precision version when they get too
big would be hidden from the user by automatic promotion of
overflow. (By "hidden" I mean the user doesn't need to care, not
that they can't find out if they want to.) We are almost there
already, but if people start coding to 'baseinteger' it takes
us down a different path entirely. 'basestring' is a completely
different issue -- there will always be a need for both unicode
and 8-bit-strings as separate types.

-- Michael Chermside

From bac at OCF.Berkeley.EDU  Wed Aug 11 23:03:11 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Aug 11 23:03:23 2004
Subject: [Python-Dev] Another approach to decorators.
In-Reply-To: <20040811133711.5f1986b6.michel@dialnetwork.com>
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
Message-ID: <411A898F.3060004@ocf.berkeley.edu>

Michel Pelletier wrote:

>>Message: 2
>>Date: Tue, 10 Aug 2004 12:55:19 -0400
>>From: Martin Zarate <mzarate@uoguelph.ca>
>>Subject: [Python-Dev] Another approach to decorators.
>>To: python-dev@python.org
>>Message-ID: <1092156919.4118fdf7b71c4@webmail.uoguelph.ca>
>>Content-Type: text/plain; charset=ISO-8859-1
>>
>>@ means nothing to an uninformed eye.  This violates the most important 
>>feature of Python (imho) which is that it is "runnable pseudocode".  
> 
> 
> Yes!
> 
> 
>>I submit that the most Python solution, that would also be legible (but not 
>>necessarily very pretty) would be to actually make the decorator a bona-fide 
>>block that you nest your class or function into.
>>
>>remake functor(foo, bar), staticmethod:
>>     def baz():
>>           pass
>>
>>This concretely shows the relationship of the original define statement to its 
>>wrapper objects.  The principle is simple - the remake block will close with 
>>only one object in its immediate namespace
> 
> 
> I like your idea a lot, buy why just one?  Your scheme of making decorators a
> block could be applied to several methods in a class:
> 
> class Foo:
> 
>   decorate static:
> 
>     def static1(blah...):
>         pass
> 
>     def static2(blah...):
>         pass
> 
> To me, this idea looks more like Python than all the rest, and allows
> you to consolidate related decorated methods and classes.  Nest them to
> apply decorators "aquisition style":

But then how are you supposed to do multiple decorators for the same method?

I am with Martin that a keyword argument is not going to work here. 
Going that route leads to ``public static 
painInTheAssAndClutteredSyntax(That can) implements Lots, Of, Stuff, 
That throws Things, You, Can, Learn, From, The, Docs`` hell.

-Brett
From bac at OCF.Berkeley.EDU  Wed Aug 11 23:09:46 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Aug 11 23:09:57 2004
Subject: [Python-Dev] request: add keywords for static and class methods
	only [related to decorators discussion]
In-Reply-To: <4113BC2C.1010206@vanderbilt.edu>
References: <4113BC2C.1010206@vanderbilt.edu>
Message-ID: <411A8B1A.7000806@ocf.berkeley.edu>

Doug Holton wrote:

> After reading some posts here and the wiki page discussing the 
> decorators:  http://www.python.org/moin/PythonDecorators
> One idea that some people have suggested is to separate out standard 
> uses for decorators like function metadata, vs. built-in features of 
> python, like static methods and class methods.
> 
> So I'm suggesting, don't use decorators for static and class methods.
> 
> Perhaps for 2.5 or whenever, we should add keyword support for built-in 
> features, just like java:
> 
> These two keywords at least:
> 
> def static mymethod (arg1, arg2):
>     ....
> 
> def classmethod mymethod( cls, arg1, arg2 ):
>     ....
> 
> And then you have to decide if you want to support other java built-in 
> features and keywords like private, protected, public, synchronized.
> 

This is in no way meant to be taken personally, but I am going make the 
understated statement, "hell no", for this.  That means -Inf from me.  =)

We do not need more keywords for just static and class methods.  They 
don't get used enough to warrant that.

-Brett
From michel at dialnetwork.com  Wed Aug 11 23:28:02 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Wed Aug 11 23:25:42 2004
Subject: [Python-Dev] Another approach to decorators.
In-Reply-To: <20040811142354.661970e0.michel@dialnetwork.com>
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<411A898F.3060004@ocf.berkeley.edu>
	<20040811142354.661970e0.michel@dialnetwork.com>
Message-ID: <20040811142802.0386bd60.michel@dialnetwork.com>

On Wed, 11 Aug 2004 14:23:54 -0700
Michel Pelletier <michel@dialnetwork.com> wrote:

> 
> Put more than one of them in the same decorate block:
> 
> class ...
> 
>   decorate staticmethod, locksFile(arg1):
> 
>     def meth1(...)
>       ...
> 
>     def meth2(...)
>       ...

Oh, and also for many decorators that look better spanning several
lines, use the same syntax proposed for multi-line imports:

class ...

  decorate (staticmethod, 
            blah,
            blah,
            locksFile(arg1)):

    def meth1(...)
      ...

    def meth2(...)
      ...

-Michel
From michel at dialnetwork.com  Wed Aug 11 23:23:54 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Wed Aug 11 23:27:36 2004
Subject: [Python-Dev] Another approach to decorators.
In-Reply-To: <411A898F.3060004@ocf.berkeley.edu>
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<411A898F.3060004@ocf.berkeley.edu>
Message-ID: <20040811142354.661970e0.michel@dialnetwork.com>

On Wed, 11 Aug 2004 14:03:11 -0700
"Brett C." <bac@ocf.berkeley.edu> wrote:

I said:
> > 
> > class Foo:
> > 
> >   decorate static:
> > 
> >     def static1(blah...):
> >         pass
> > 
> >     def static2(blah...):
> >         pass
> > 
> > To me, this idea looks more like Python than all the rest, and allows
> > you to consolidate related decorated methods and classes.  Nest them to
> > apply decorators "aquisition style":
> 
> But then how are you supposed to do multiple decorators for the same method?

Put more than one of them in the same decorate block:

class ...

  decorate staticmethod, locksFile(arg1):

    def meth1(...)
      ...

    def meth2(...)
      ...

> I am with Martin that a keyword argument is not going to work here. 

I presume you mean MvL and not the orginal poster, Martin Zarate.

> Going that route leads to ``public static 
> painInTheAssAndClutteredSyntax(That can) implements Lots, Of, Stuff, 
> That throws Things, You, Can, Learn, From, The, Docs`` hell.

How does Martin Z's idea lead to this?

-Michel
From fredrik at pythonware.com  Thu Aug 12 00:29:02 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu Aug 12 00:30:22 2004
Subject: [Python-Dev] Re: Re: Re: pre-PEP: Complete,
	Structured Regular ExpressionGroup Matching
References: <411429FB.6080603@gradient.cis.upenn.edu>	<lkd1xijedzx.fsf@panix2.panix.com><brhn9guy.fsf@yahoo.co.uk>	<lkdzn56aljx.fsf@panix2.panix.com><4118C626.4070203@heneryd.com>
	<cfah8s$van$1@sea.gmane.org> <4118D5F7.1090709@heneryd.com>
Message-ID: <cfe6g7$nss$1@sea.gmane.org>

Erik Heneryd wrote:

> Yes, of course it's slow, as I recursivly compile subpatterns and try them out, but as things are, 
> it's probably much easier to implement it this way than using scanners, which I suspect is a lot 
> of work (given the structmatch interface)...

sure, which is why it's a good idea to design a better interface.

more on this later.

</F> 



From dave at boost-consulting.com  Thu Aug 12 00:38:12 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Aug 12 00:38:56 2004
Subject: [Python-Dev] Re: Another approach to decorators.
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
Message-ID: <un011l2qz.fsf@boost-consulting.com>

Michel Pelletier <michel@dialnetwork.com> writes:

>> From: Martin Zarate <mzarate@uoguelph.ca>
>
>> I submit that the most Python solution, that would also be legible (but not 
>> necessarily very pretty) would be to actually make the decorator a bona-fide 
>> block that you nest your class or function into.
>> 
>> remake functor(foo, bar), staticmethod:
>>      def baz():
>>            pass
>> 
>> This concretely shows the relationship of the original define statement to its 
>> wrapper objects.  The principle is simple - the remake block will close with 
>> only one object in its immediate namespace
>
> I like your idea a lot, buy why just one?  Your scheme of making decorators a
> block could be applied to several methods in a class:
>
> class Foo:
>
>   decorate static:
>
>     def static1(blah...):
>         pass
>
>     def static2(blah...):
>         pass
>
> To me, this idea looks more like Python than all the rest, and allows
> you to consolidate related decorated methods and classes.  Nest them to
> apply decorators "aquisition style":
>
>
> class Foo:
>
>   decorate static:
>
>     def static1(blah...):
>         pass
>
>     decorate locksFile:
>
>       def static2andLocks(blah...):  # both decorators appy
>           pass

Wow, Martin Z's idea (using your keyword) really went "thunk" for me.
What decorate does would be very much like what "class" does in some
ways.

I wonder if there's a way, using metaclasses, to get this syntax
today:

    class Foo:

      class d1(static):
        def static1(blah):
          pass

        class d2(locksFile):
          def static2andLocks(blah): # both apply
            pass

for suitably defined "static" and "locksFile" classes?

It's not beautiful, but it would be a good proof of the
generality/implementability of the decorator->class mapping.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From mcherm at mcherm.com  Thu Aug 12 00:46:59 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Aug 12 00:45:52 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
Message-ID: <1092264419.411aa1e38b2fd@mcherm.com>

I previously wrote:
> I envision an eventual Python which has just one type, called 'int'.

Oops, didn't mean to let the cat out of the bag about my evil plan to
make Python fast by turning it into a simple machine langauge.

Hmm... let's pretend I said "...just one integral numeric type..."
instead. That way the PSU won't be able to figure out that
From skip at pobox.com  Thu Aug 12 00:03:59 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 12 01:06:46 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <1092258146.411a896246d1a@mcherm.com>
References: <1092258146.411a896246d1a@mcherm.com>
Message-ID: <16666.38863.339305.573591@montanaro.dyndns.org>


    >> I think this would be a good idea; maybe the name should be
    >> baseinteger?

    Michael> I would like to urge caution before making this change. Despite
    Michael> what the PEP may say, I actually think that creating a
    Michael> 'baseint' type is the WRONG design choice for the long term.  I
    Michael> envision an eventual Python which has just one type, called
    Michael> 'int'.

I agree.  I made a suggestion that we consider the entire tree of numeric
types, but I had int/long unification in the back of my mind as well.  I
will take /F's suggestion and poke around the peps when I have some time,
but I see no pressing reason a base integer class, however it's spelled,
needs to be added for 2.4.

Skip
From ncoghlan at iinet.net.au  Thu Aug 12 01:26:38 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Aug 12 01:26:47 2004
Subject: [Python-Dev] Decorator keyword possibility?
Message-ID: <411AAB2E.1050200@iinet.net.au>

Someone recently suggested 'remake' as a possible replacement for '@'

I don't like that name because Python doesn't use the word 'make' for 
anything. However, it immediately prompted the thought of 'redef'

After all, decorator's effectively 'redefine' the original 'define' 
block, to mean something else.

And the old decorator syntax is *definitely* a 'redefinition' syntax.

However, the problem with that word is that you would expect it to come 
_after_ the definition (like the old syntax does). Which leads to the 
actual suggestion of a 'predefinition'.

That is, we're going to tell you about the definition before we actually 
get to it.

And so, on with the examples:

class demo(object):

   predef classmethod
   def foo(cls): pass

   predef staticmethod
   def bar(): pass

   predef accepts(int, int)
   predef returns(bool)
   predef someOtherDecorator(anarg, anotherarg, yetanotherarg)
   def foobar(self, apples, oranges):
     """Compares apples and oranges"""
     return apples < oranges


Regards,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From mcherm at mcherm.com  Thu Aug 12 01:31:12 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Aug 12 01:30:05 2004
Subject: [Python-Dev] Re: Re: Update PEP 292
Message-ID: <1092267072.411aac40ecd4a@mcherm.com>

Fredrik writes:
> - Consider changing the ${} to accept arbitrary patterns, instead of
>   just Python-style identifiers.  \${([^}]+)} should do the trick.

+1

And hey, I'd be +1 on releasing a less-simple template that allowed
arbitrary Python expressions (but no '}' characters) instead of just
identifiers. But that's another issue -- and my liking it doesn't
detract from my liking SIMPLE templates too.

As Barry said elsewhere... even if all it does is to replace '%()s'
with '${}' it's still an improvement.

-- Michael Chermside

From chris.cavalaria at free.fr  Thu Aug 12 01:04:43 2004
From: chris.cavalaria at free.fr (Christophe Cavalaria)
Date: Thu Aug 12 01:40:15 2004
Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered
References: <1092179578.4119567af011d@webmail.oregonstate.edu>
Message-ID: <cfe8mc$ujt$1@sea.gmane.org>

barnesc@engr.orst.edu wrote:

> -----------------------------------------------
> Option A
> -----------------------------------------------
> 
> """
> Hi,
> 
> Performs a binary operation.
> 
> Docstring.
> """
> @staticmethod
> @synchronized(lockvar)
> def f(a, b):
>     return a + b
> 
> """
> Now,
> 
> Performs a simple operation.
> """
> @classmethod
> def g(a):
>     return a
> 
> -----------------------------------------------
> Option B
> -----------------------------------------------
> 
> def f(a, b):
>     @staticmethod
>     @synchronized(lockvar)
>     """
>     Hi,
> 
>     Performs a binary operation.
> 
>     Docstring.
>     """
>     return a + b
> 
> def g(a):
>     @classmethod
>     """
>     Now,
> 
>     Performs a simple operation.
>     """
>     return a
> 
> 
> Now, forget everything you've learned about Python.
> Forget that @ symbols are ugly.
> 
>  - For which method is it visually easier to find the function def?

None of them. A good syntax coloring would even make it easier in fact. On
the second hand, the Option B makes it much harder to find the function
code once you've found the function def.

>  - For which method is the code in the most logical order?

Option A of course. Since the decorator can be seen as a function that takes
the defined function as it's first parameter, it's logical to place the
decorator before the definition.

@staticmethod
def f():
    pass

is a short version of

f = staticmethod(
    def f():
        pass
)

after all

> Note that Option A causes you to skim all the way through
> the docstring and the decorators to find out which function
> is being defined.  At this point, you have to start over
> at the docstring, to find out what the function actually does.

This is false of course, any good syntax coloring would give you a good
contrast between the function itself and the decorators. That way, it'll be
easy to find the first line that isn't colored like a decorator. On the
Option B, you'll have to identify 3 blocs of data : the function def, the
decorator bloc and the function body.

> Option A is especially painful when a large number of functions
> are thrown into a module.  It is almost impossible to grep
> efficiently for the function names.  Thus the code appears
> disorganized, and messy.

I don't understand how grep can be "confused" by the decorators before the
def. Are we talking about the same program ???



From andrew at indranet.co.nz  Thu Aug 12 03:21:33 2004
From: andrew at indranet.co.nz (Andrew McGregor)
Date: Thu Aug 12 03:22:54 2004
Subject: [Python-Dev] Re: Re: Call for defense of @decorators
In-Reply-To: <16666.13165.619920.756435@montanaro.dyndns.org>
References: <200408051636.i75Gaak03654@guido.python.org>       
	<20040805174252.GA27820@burma.localdomain>       
	<20040805112318.B13062@ActiveState.com>       
	<20040805183444.GA29796@burma.localdomain>       
	<20040805190549.GC29796@burma.localdomain>       
	<4112A18D.5090801@v.loewis.de>       
	<20040805215331.GB31288@burma.localdomain>       
	<ceuk0e$rut$1@sea.gmane.org> <1091806774.8414.63.camel@localhost> 
	<1f7befae040806090644be9c04@mail.gmail.com>       
	<cf0gd1$smv$1@sea.gmane.org> <1091818207.7529.34.camel@localhost> 
	<4113D5E1.6040006@interlink.com.au> <41141255.5010408@stackless.com>
	<41175442.4010902@livinglogic.de>       
	<16664.50665.170916.600964@montanaro.dyndns.org>       
	<DAE618D9A063D6BD6E7D58DC@[192.168.1.247]>
	<16666.13165.619920.756435@montanaro.dyndns.org>
Message-ID: <34A2826306EF5636150168FE@[192.168.1.248]>



--On Wednesday, 11 August 2004 9:55 a.m. -0500 Skip Montanaro 
<skip@pobox.com> wrote:

>
>     Andrew> Aren't we really trying to recreate def as a multi-line lambda
>     Andrew> that binds when it's in a context that is not expecting a
>     Andrew> function argument in block form?
>
> I suppose you can look at it that way.  The output of a function
> definition plus any decorator applications to it need not be a function
> though.  I posted a simpleminded example of generating a greyscale ramp
> using a decorator.  If I thought about it for more than a few seconds I
> could probably come up with some other examples as well where the output
> wasn't a function.  Some of them might even be useful. ;-)
>
> Skip
>
>

And that's exactly my point: (relatively) big one-time-use functions can be 
really useful, especially if the function definition is somewhat dependant 
on context.

Andrew


From greg at cosc.canterbury.ac.nz  Thu Aug 12 03:24:58 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug 12 03:25:05 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <59e9fd3a040811052152548b6b@mail.gmail.com>
Message-ID: <200408120124.i7C1OwMk011645@cosc353.cosc.canterbury.ac.nz>

> Apart from removing the need for the parentheses/braces in this case,
> all it does is consistently remove the "s",

I believe that just removing the 's' alone would be a
substantial benefit. It's easy to forget when writing,
and tends to get lost or confused with the surrounding
text when reading.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From bac at OCF.Berkeley.EDU  Thu Aug 12 03:36:27 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug 12 03:36:40 2004
Subject: [Python-Dev] Another approach to decorators.
In-Reply-To: <20040811142354.661970e0.michel@dialnetwork.com>
References: <20040811195454.9BD311E400A@bag.python.org>	<20040811133711.5f1986b6.michel@dialnetwork.com>	<411A898F.3060004@ocf.berkeley.edu>
	<20040811142354.661970e0.michel@dialnetwork.com>
Message-ID: <411AC99B.2040906@ocf.berkeley.edu>

Michel Pelletier wrote:
> On Wed, 11 Aug 2004 14:03:11 -0700
> "Brett C." <bac@ocf.berkeley.edu> wrote:
> 
> I said:
> 
>>>class Foo:
>>>
>>>  decorate static:
>>>
>>>    def static1(blah...):
>>>        pass
>>>
>>>    def static2(blah...):
>>>        pass
>>>
>>>To me, this idea looks more like Python than all the rest, and allows
>>>you to consolidate related decorated methods and classes.  Nest them to
>>>apply decorators "aquisition style":
>>
>>But then how are you supposed to do multiple decorators for the same method?
> 
> 
> Put more than one of them in the same decorate block:
> 
> class ...
> 
>   decorate staticmethod, locksFile(arg1):
> 
>     def meth1(...)
>       ...
> 
>     def meth2(...)
>       ...
> 
> 

I still don't like it.  It feels like I am classifying my methods based 
on the applied decorators instead of thinking of decorators as, well, 
decorators of methods.  I feel methods are the major thing and 
decorators are just extra fluff on them, not the other way around as 
this syntax says to me.  Changing the indentation delineates scope and I 
don't feel decorators truly represent a change in scope.

>>I am with Martin that a keyword argument is not going to work here. 
> 
> 
> I presume you mean MvL and not the orginal poster, Martin Zarate.
> 

Yep, sorry for not being clearer.

> 
>>Going that route leads to ``public static 
>>painInTheAssAndClutteredSyntax(That can) implements Lots, Of, Stuff, 
>>That throws Things, You, Can, Learn, From, The, Docs`` hell.
> 
> 
> How does Martin Z's idea lead to this?
> 

A keyword followed by a comma-separated list of what that keyword affects.

-Brett
From greg at cosc.canterbury.ac.nz  Thu Aug 12 03:56:42 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug 12 03:56:52 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <1092258146.411a896246d1a@mcherm.com>
Message-ID: <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>

Michael Chermside <mcherm@mcherm.com>:

> 'basestring' is a completely different issue -- there will always be
> a need for both unicode and 8-bit-strings as separate types.

I'm not so sure about that. There will certainly be a need for
something holding an arbitrary sequence of bytes, but it's not clear
that it needs to be called anything with 'string' in it.

I can enivisage a future in which 'string' means unicode string, and
byte sequences are called something else entirely, such as
'bytevector' or 'bytearray' (or maybe just 'array' :-).

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From eppstein at ics.uci.edu  Thu Aug 12 04:02:14 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Thu Aug 12 04:02:21 2004
Subject: [Python-Dev] Re: Another approach to decorators.
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<un011l2qz.fsf@boost-consulting.com>
Message-ID: <eppstein-8D4A8F.19021411082004@sea.gmane.org>

In article <un011l2qz.fsf@boost-consulting.com>,
 David Abrahams <dave@boost-consulting.com> wrote:

> > class Foo:
> >
> >   decorate static:
> >
> >     def static1(blah...):
> >         pass
> >
> >     decorate locksFile:
> >
> >       def static2andLocks(blah...):  # both decorators appy
> >           pass
> 
> Wow, Martin Z's idea (using your keyword) really went "thunk" for me.
> What decorate does would be very much like what "class" does in some
> ways.

class: (and other something: constructs) start a block that can contain 
any code.  Does this decorate keyword allow e.g. assignments as well as 
defs and other decorates?  Or loops?  If so what should it mean?  Is it 
like that locals() gets replaced by a special dictionary-like-object 
that calls the decorator whenever any of its contents gets set?

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From python at rcn.com  Thu Aug 12 04:21:29 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Aug 12 04:22:15 2004
Subject: [Python-Dev] Naming nit
In-Reply-To: <411A79AA.4020505@ocf.berkeley.edu>
Message-ID: <006a01c48013$13eab7a0$e622c797@oemcomputer>

[Raymond Hettinger]
> > For PEP 292, should the class be Template() or template()?
> >

[Brett]
> Template as per PEP 8.


Does anyone else want to chime in (I'm not the one who needs to be
convinced)?


Raymond

From skip at pobox.com  Thu Aug 12 04:37:47 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 12 04:38:01 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
Message-ID: <16666.55291.734287.170753@montanaro.dyndns.org>


    Greg> I can enivisage a future in which 'string' means unicode string,
    Greg> and byte sequences are called something else entirely, such as
    Greg> 'bytevector' or 'bytearray' (or maybe just 'array' :-).

Or just 'bytes'?

Skip
From skip at pobox.com  Thu Aug 12 04:50:09 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 12 04:50:20 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
Message-ID: <16666.56033.65067.241794@montanaro.dyndns.org>


    Greg> I can enivisage a future in which 'string' means unicode string,
    Greg> and byte sequences are called something else entirely, such as
    Greg> 'bytevector' or 'bytearray' (or maybe just 'array' :-).

After another couple minutes of thought:

    1. Maybe in 2.5 introduce a "bytes" builtin as a synonym for "str" and
       recommend its use when the intent is an arbitrary sequence of bytes.
       Add 'b' as a string literal prefix to generate a bytes object (which
       will really just be a string).  At the same time, start raising
       exceptions when non-ascii string literals are used without a coding
       cookie.  (Warnings are already issued for that.)

    2. In 2.6, make str a synonym for unicode, remove basestring (or also
       make it a synonym for unicode), make bytes its own distinct (mutable?
       immutable?) type, and have b"..." literals generate bytes objects.

Then again, maybe this can't happen until Py3K.  Before then we could do the
following though:

    1. Make bytes a synonuym for str.

    2. Warn about the use of bytes as a variable name.

    3. Introduce b"..." literals as a synonym for current string literals,
       and have them *not* generate warnings if non-ascii characters were
       used in them without a coding cookie.

PEP time?

Skip
From barnesc at engr.orst.edu  Thu Aug 12 04:38:25 2004
From: barnesc at engr.orst.edu (barnesc@engr.orst.edu)
Date: Thu Aug 12 04:51:14 2004
Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered
Message-ID: <1092278305.411ad82141c53@webmail.oregonstate.edu>


>> -----------------------------------------------
>> Option A
>> -----------------------------------------------
>> 
>> """
>> Hi,
>> 
>> Performs a binary operation.
>> 
>> Docstring.
>> """
>> @staticmethod
>> @synchronized(lockvar)
>> def f(a, b):
>>     return a + b
>> 
>> """
>> Now,
>> 
>> Performs a simple operation.
>> """
>> @classmethod
>> def g(a):
>>     return a
>> 
>> -----------------------------------------------
>> Option B
>> -----------------------------------------------
>> 
>> def f(a, b):
>>     @staticmethod
>>     @synchronized(lockvar)
>>     """
>>     Hi,
>> 
>>     Performs a binary operation.
>> 
>>     Docstring.
>>     """
>>     return a + b
>> 
>> def g(a):
>>     @classmethod
>>     """
>>     Now,
>> 
>>     Performs a simple operation.
>>     """
>>     return a
>> 
>> 
>> Now, forget everything you've learned about Python.
>> Forget that @ symbols are ugly.
>> 
>>  - For which method is it visually easier to find the function def?
>
>None of them. A good syntax coloring would even make it easier in fact. On
>the second hand, the Option B makes it much harder to find the function
>code once you've found the function def.

Option B makes it easier to find the function def, because the def is the
only line that is not indented.  I can scan straight down the page and
instantly find the two function 'def's.   For Option A, I have to scan
line by line.

>>  - For which method is the code in the most logical order?
>
>Option A of course. Since the decorator can be seen as a function that takes
>the defined function as it's first parameter, it's logical to place the
>decorator before the definition.


I consider the function def to be the most important line of the
entire function.

Thus it follows that it should be the first line.

Thinking further, it makes sense to define a function, then define
various properties of the function in indented blocks, since the
properties are 'contained' in the function.

I do agree with your assessment for decorators.  However, you
have to weigh the 'priority' of the decorator against the priority
of the def line.  In my mind, the def line wins, and so should be
first.

Again, of course, there's the advantage of being able to skim down
the left column (which is mostly whitespace) and locate function
entrance points quickly.  For me, this increases the perceived
organization of the code, much like labelled boxes help organize
the papers in my room (locate the box quickly, then sort through
it).

See:

http://oregonstate.edu/~barnesc/temp/option/optiona.html
http://oregonstate.edu/~barnesc/temp/option/optionb.html

Consider the difficulty of scanning such code, when there is
a long module, with nested functions inside a class.

>I don't understand how grep can be "confused" by the decorators before the
>def. Are we talking about the same program ???

I refer to searching done by the eye.

 - Connelly
From steven at zensaki.com  Thu Aug 12 03:20:20 2004
From: steven at zensaki.com (Steven Kah Hien Wong)
Date: Thu Aug 12 04:51:18 2004
Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence
Message-ID: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au>


Hi,

From:

    http://docs.python.org/tut/node11.html#SECTION0011510000000000000000

This paragraph (last two sentences):

    It is clear that indiscriminate use of multiple inheritance is a
    maintenance nightmare, given the reliance in Python on conventions
    to avoid accidental name conflicts. A well-known problem with
    multiple inheritance is a class derived from two classes that happen
    to have a common base class. While it is easy enough to figure out
    what happens in this case (the instance will have a single copy of
    ``instance variables'' or data attributes used by the common base
    class), it is not clear that these semantics are in any way useful.

Maybe I have misread, but I have interpreted that as meaning the two
children will share the same instance of the base class. So if one child
changes its parent, the other child will pick up that change, too.

But I did some quick tests, and it turns out this isn't so.

    class CommonBase:
        x = 0

    class ChildOne(CommonBase):
        None

    class ChildTwo(CommonBase):
        None

    class ChildOneAndTwo(ChildOne, ChildTwo):
        def run(self):
            ChildOne.x = 1
            print "one =", ChildOne.x
            print "two =", ChildTwo.x

    ChildOneAndTwo().run()

And the output was:

    $ python multi.py
    one = 1
    two = 0

According to the documentation, I thought it should be:

    one = 1
    two = 1

Mind you, I find the current behaviour is much more useful, so I am
saying the documentation is in error, not the implementation. :) Have I
simply misread? If so, maybe that paragraph should be rewritten to make
it more clear. With an example like the above, perhaps?

Steven Wong
From exarkun at divmod.com  Thu Aug 12 04:59:06 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Thu Aug 12 04:59:11 2004
Subject: [Python-Dev] Re: Another approach to decorators.
In-Reply-To: <eppstein-8D4A8F.19021411082004@sea.gmane.org>
References: <20040811195454.9BD311E400A@bag.python.org>	<20040811133711.5f1986b6.michel@dialnetwork.com>	<un011l2qz.fsf@boost-consulting.com>
	<eppstein-8D4A8F.19021411082004@sea.gmane.org>
Message-ID: <411ADCFA.8050305@divmod.com>

David Eppstein wrote:
> In article <un011l2qz.fsf@boost-consulting.com>,
>  David Abrahams <dave@boost-consulting.com> wrote:
> 
> 
>>>class Foo:
>>>
>>>  decorate static:
>>>
>>>    def static1(blah...):
>>>        pass
>>>
>>>    decorate locksFile:
>>>
>>>      def static2andLocks(blah...):  # both decorators appy
>>>          pass
>>
>>Wow, Martin Z's idea (using your keyword) really went "thunk" for me.
>>What decorate does would be very much like what "class" does in some
>>ways.
> 
> 
> class: (and other something: constructs) start a block that can contain 
> any code.  Does this decorate keyword allow e.g. assignments as well as 
> defs and other decorates?  Or loops?  If so what should it mean?  Is it 
> like that locals() gets replaced by a special dictionary-like-object 
> that calls the decorator whenever any of its contents gets set?
> 

   (I have no idea what the original poster intended, however....)

     'decorate' expression ':'
         suite

   could create a nested scope, the locals of which could be passed to 
whatever "expression" evaluates to (or if it is a tuple, do the looping 
thing, since people seem to like that).

   The call can return a dict with which the class dict is updated.

   For that matter, 'decorate' could even be dropped, to avoid 
introducing a new keyword.  I don't think

     expression ':'
         suite

   means anything in the current grammar.  This would mean staticmethod 
and classmethod would no longer work (unless we really want them to - 
they can do type checking on their argument and modify their behavior 
accordingly).  Replacement implementations would be simple.  In fact, 
making any current decorator work with this new mechanism is easy:

     def forall(decorator):
         def new_decorator(namespace):
             for k, v in namespace.items():
                 namespace[k] = decorator(v)
             return namespace
         return new_decorator

     class Foo(object):
         forall(staticmethod):
             def bar(x):
                 print 'spam', x

   Many decorators would probably be used with something like forall, or 
simply support dicts directly.  Of course, many would want to work some 
other way.

   For the PyObjC crowd, I think this would remove a major use of 
metaclasses (as people have argued that metaclasses are hard to 
understand and best avoided for simple decoration, this seems to be a 
good thing).  I can also think of some terrible, terrible metaclass 
hackery in my own code that this would obviate the need for.

   This covers a use case James Knight mentioned as well.  As he pointed 
out, '@' and other proposals only decorate functions (and maybe classes, 
soon).  The example he gave, I believe, was:

     @public
     x = 1

   This doesn't work, but:

     public:
         x = 1

   would.  Now, swallow that scream of horror (I did, barely ;). 
"public" here is intended to be defined as something like:

     def public(namespace):
         __all__.extend(namespace.keys())
         return namespace

   I can see the potential for confusion to new Python programmers here, 
though, if they come from certain other languages.  Perhaps leaving the 
keyword prefix would be best after all.

   Another possibility would be to pass a code object for the suite, 
instead of a namespace.  I am inclined to prefer this, since it more 
closely resembles the other construct which can create a nested scope. 
It adds the burden of exec'ing the code to the decorator, though.  This 
isn't a problem if something like "forall" above is included in the 
stdlib, since the common case is dealt with already (thus people 
desiring class or static methods need not know or care about it), and 
more advanced users will value the power it affords over the minor 
additional effort required to use it.

   Jp
From guido at python.org  Thu Aug 12 05:00:30 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 05:00:40 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
In-Reply-To: Your message of "Wed, 11 Aug 2004 13:53:31 PDT."
	<1092257611.411a874b5094e@mcherm.com> 
References: <1092257611.411a874b5094e@mcherm.com> 
Message-ID: <200408120300.i7C30Uk28852@guido.python.org>

> For instance:
> 
>     >>> accumulator = 0.0
>     >>> for i in range(25000):
>     ...     accumulator += 1.0
>     ...
>     >>> print accumulator
>     25000.0
> 
> Notice, perfect accuracy. Yet people don't tend to use floating
> point for loops... why not?

Because the generalization to using fractions doesn't work.  Try the
same with adding 0.1, 25000 times.  Better not be tempted.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug 12 05:03:11 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 05:03:19 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: Your message of "Wed, 11 Aug 2004 14:02:26 PDT."
	<1092258146.411a896246d1a@mcherm.com> 
References: <1092258146.411a896246d1a@mcherm.com> 
Message-ID: <200408120303.i7C33BG28874@guido.python.org>

> I would like to urge caution before making this change. Despite
> what the PEP may say, I actually think that creating a 'baseint'
> type is the WRONG design choice for the long term. I envision
> an eventual Python which has just one type, called 'int'. The
> fact that an efficient implementation is used when the ints
> are small and an arbitrary-precision version when they get too
> big would be hidden from the user by automatic promotion of
> overflow. (By "hidden" I mean the user doesn't need to care, not
> that they can't find out if they want to.) We are almost there
> already, but if people start coding to 'baseinteger' it takes
> us down a different path entirely. 'basestring' is a completely
> different issue -- there will always be a need for both unicode
> and 8-bit-strings as separate types.

Not so sure.  I expect that, like Jython and IronPython, Python 3000
will use unicode for strings, and have a separate mutable byte array
for 8-bit bytes.

In Python 3000 I expect that indeed somehow the existence of long is
completely hidden from the user, but that's a long time away, and
until then baseinteger might be a better solution than requiring
people to write isinstance(x, (int, long)).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From exarkun at divmod.com  Thu Aug 12 05:07:21 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Thu Aug 12 05:07:26 2004
Subject: [Python-Dev] Semantics of decorators?
In-Reply-To: <411A359E.3080309@erols.com>
References: <411A359E.3080309@erols.com>
Message-ID: <411ADEE9.4000105@divmod.com>

Edward C. Jones wrote:
> In the development docs, Python Reference Manual, 7.5, it says:
> 
> A function definition may be wrapped by one or more decorator expressions.
> Decorator expressions are evaluated when the function is defined, in the
> scope that contains the function definition. The result must be a callable,
> which is invoked with the function object as the only argument. The 
> returned
> value is bound to the function name instead of the function object. If 
> there
> are multiple decorators, they are applied in reverse order.
> 
> Does this completely describe the semantics of decorators?
> 

   One thing it omits is what "decorator expressions" means.  How about 
including a note regarding the kind of expressions which are acceptable?

   Jp
From bac at OCF.Berkeley.EDU  Thu Aug 12 05:47:31 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug 12 05:47:43 2004
Subject: [Python-Dev] status of statistics.py?
In-Reply-To: <2mwu0c476t.fsf@starship.python.net>
References: <2mwu0c476t.fsf@starship.python.net>
Message-ID: <411AE853.10404@ocf.berkeley.edu>

Michael Hudson wrote:

> I see statistics.py is still in nondist/sandbox/.  Is there any deep
> reason for this?  I see the file ends with
> 
> XXX = """
>     Other possible functions include data groupers for
>     binning, counting, and splitting into equivalence classes.
> """
> 
> but that hardly seems a blocker.  I'd like to see the module in 2.4.
> 

Just so that it is documented here and the people not at the last Bug 
Day know, Michael asked Raymond about this.  Raymond said that he didn't 
really see a need for the module anymore since most of the important 
functionality had already made it into the stdlib in one form or 
another.  He also said that time would be better spent on a calculator 
module that could take on any functionality that was needed that 
'statistics' provided.

-Brett
From aahz at pythoncraft.com  Thu Aug 12 05:54:55 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Aug 12 05:54:59 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <200408120124.i7C1OwMk011645@cosc353.cosc.canterbury.ac.nz>
References: <59e9fd3a040811052152548b6b@mail.gmail.com>
	<200408120124.i7C1OwMk011645@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040812035455.GA12140@panix.com>

On Thu, Aug 12, 2004, Greg Ewing wrote:
>
>> Apart from removing the need for the parentheses/braces in this case,
>> all it does is consistently remove the "s",
> 
> I believe that just removing the 's' alone would be a substantial
> benefit. It's easy to forget when writing, and tends to get lost or
> confused with the surrounding text when reading.

I'll say.  I've been doing a lot of work with interpolated strings
lately, and even though I've been bitten several times, I keep making
the same mistake.  Maybe I'm just stupid, though...
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From tim.hochberg at ieee.org  Thu Aug 12 06:08:37 2004
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu Aug 12 06:11:02 2004
Subject: [Python-Dev] Re: Another approach to decorators.
In-Reply-To: <411ADCFA.8050305@divmod.com>
References: <20040811195454.9BD311E400A@bag.python.org>	<20040811133711.5f1986b6.michel@dialnetwork.com>	<un011l2qz.fsf@boost-consulting.com>	<eppstein-8D4A8F.19021411082004@sea.gmane.org>
	<411ADCFA.8050305@divmod.com>
Message-ID: <411AED45.1070300@ieee.org>

Jp Calderone wrote:
> David Eppstein wrote:
> 
>> In article <un011l2qz.fsf@boost-consulting.com>,
>>  David Abrahams <dave@boost-consulting.com> wrote:
>>
>>
>>>> class Foo:
>>>>
>>>>  decorate static:
>>>>
>>>>    def static1(blah...):
>>>>        pass
>>>>
>>>>    decorate locksFile:
>>>>
>>>>      def static2andLocks(blah...):  # both decorators appy
>>>>          pass
>>>
>>>
>>> Wow, Martin Z's idea (using your keyword) really went "thunk" for me.
>>> What decorate does would be very much like what "class" does in some
>>> ways.
>>
>>
>>
>> class: (and other something: constructs) start a block that can 
>> contain any code.  Does this decorate keyword allow e.g. assignments 
>> as well as defs and other decorates?  Or loops?  If so what should it 
>> mean?  Is it like that locals() gets replaced by a special 
>> dictionary-like-object that calls the decorator whenever any of its 
>> contents gets set?
>>
> 
>   (I have no idea what the original poster intended, however....)
> 
>     'decorate' expression ':'
>         suite
> 
>   could create a nested scope, the locals of which could be passed to 
> whatever "expression" evaluates to (or if it is a tuple, do the looping 
> thing, since people seem to like that).
> 
>   The call can return a dict with which the class dict is updated.

FWIW, I proposed something very similar to this on python-list 
(http://mail.python.org/pipermail/python-list/2004-August/233262.html). 
It didn't get a very excited reception there. I think deafening silence 
is the phrase I'd use. But perhaps it'll inspire some new thoughts on 
your proposal.

The gist of my proposal was that:

     'def' name 'as' expression ':'
          suite

would bind name to the result of expression where expression was 
evaluated in the scope of suite, after suite was executed. So, to reuse 
the example below:

      class Foo(object):
          def bar as staticmethod(bar):
              def bar(x):
                  print 'spam', x

For several more examples, see the original post.

-tim

> 
>   For that matter, 'decorate' could even be dropped, to avoid 
> introducing a new keyword.  I don't think
> 
>     expression ':'
>         suite
> 
>   means anything in the current grammar.  This would mean staticmethod 
> and classmethod would no longer work (unless we really want them to - 
> they can do type checking on their argument and modify their behavior 
> accordingly).  Replacement implementations would be simple.  In fact, 
> making any current decorator work with this new mechanism is easy:
> 
>     def forall(decorator):
>         def new_decorator(namespace):
>             for k, v in namespace.items():
>                 namespace[k] = decorator(v)
>             return namespace
>         return new_decorator
> 
>     class Foo(object):
>         forall(staticmethod):
>             def bar(x):
>                 print 'spam', x
> 
>   Many decorators would probably be used with something like forall, or 
> simply support dicts directly.  Of course, many would want to work some 
> other way.
> 
>   For the PyObjC crowd, I think this would remove a major use of 
> metaclasses (as people have argued that metaclasses are hard to 
> understand and best avoided for simple decoration, this seems to be a 
> good thing).  I can also think of some terrible, terrible metaclass 
> hackery in my own code that this would obviate the need for.
> 
>   This covers a use case James Knight mentioned as well.  As he pointed 
> out, '@' and other proposals only decorate functions (and maybe classes, 
> soon).  The example he gave, I believe, was:
> 
>     @public
>     x = 1
> 
>   This doesn't work, but:
> 
>     public:
>         x = 1
> 
>   would.  Now, swallow that scream of horror (I did, barely ;). "public" 
> here is intended to be defined as something like:
> 
>     def public(namespace):
>         __all__.extend(namespace.keys())
>         return namespace
> 
>   I can see the potential for confusion to new Python programmers here, 
> though, if they come from certain other languages.  Perhaps leaving the 
> keyword prefix would be best after all.
> 
>   Another possibility would be to pass a code object for the suite, 
> instead of a namespace.  I am inclined to prefer this, since it more 
> closely resembles the other construct which can create a nested scope. 
> It adds the burden of exec'ing the code to the decorator, though.  This 
> isn't a problem if something like "forall" above is included in the 
> stdlib, since the common case is dealt with already (thus people 
> desiring class or static methods need not know or care about it), and 
> more advanced users will value the power it affords over the minor 
> additional effort required to use it.
> 
>   Jp
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org 
> 
> 

From guido at python.org  Thu Aug 12 06:14:53 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 06:15:01 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: Your message of "Wed, 11 Aug 2004 21:50:09 CDT."
	<16666.56033.65067.241794@montanaro.dyndns.org> 
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz> 
	<16666.56033.65067.241794@montanaro.dyndns.org> 
Message-ID: <200408120414.i7C4Er429003@guido.python.org>

>     1. Make bytes a synonuym for str.

Hmm...  I worry that a simple alias would just encourage confused
usage, since the compiler won't check.  I'd rather see bytes an alias
for a bytes array as defined by the array module.

>     2. Warn about the use of bytes as a variable name.

Is this really needed?  Builtins don't byte variable names.

>     3. Introduce b"..." literals as a synonym for current string
>        literals, and have them *not* generate warnings if non-ascii
>        characters were used in them without a coding cookie.

I expecet all sorts of problems with that, such as what it would mean
if Unicode or multibyte characters are used in the source.

Do we really need byte array literals at all?  I don't expect there to
be much of a demand.  Rather, byte arrays would eventually be returned
by the read() method when a file is opened in binary mode.  (Isn't
this roughly how Java does this?)  We could start doing this
relatively soon if we used a new mode character ("B" anyone?).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug 12 06:17:07 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 06:17:15 2004
Subject: [Python-Dev] Naming nit
In-Reply-To: Your message of "Wed, 11 Aug 2004 22:21:29 EDT."
	<006a01c48013$13eab7a0$e622c797@oemcomputer> 
References: <006a01c48013$13eab7a0$e622c797@oemcomputer> 
Message-ID: <200408120417.i7C4H7529029@guido.python.org>

> [Raymond Hettinger]
> > > For PEP 292, should the class be Template() or template()?
> > >
> 
> [Brett]
> > Template as per PEP 8.
> 
> Does anyone else want to chime in (I'm not the one who needs to be
> convinced)?

Who is?

In general, classes imported from modules are Capitalized (CamelCase,
really).  Only builtins aren't.  (Counter-examples in the standard
library are mostly historic relics.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug 12 06:26:30 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 06:26:38 2004
Subject: [Python-Dev] Another approach to decorators.
In-Reply-To: Your message of "Wed, 11 Aug 2004 14:28:02 PDT."
	<20040811142802.0386bd60.michel@dialnetwork.com> 
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<411A898F.3060004@ocf.berkeley.edu>
	<20040811142354.661970e0.michel@dialnetwork.com> 
	<20040811142802.0386bd60.michel@dialnetwork.com> 
Message-ID: <200408120426.i7C4QU829083@guido.python.org>

[Various proposals suggesting the use of a single decorator block to
decorate multiple method definitions]

I'm not keen on this.  I expect that beyond classmethod and
staticmethod there aren't many use cases for repeating the same
decorator exactly multiple times.  It also requires indenting the
method, which I've already argued against.  And lastly it removes the
decorator from the method definition again, thereby defeating the
readability goals: when you look at a def in isolation, you may forget
to look several pages up for a decorator.

I think this is too far-fetched to consider as an alternative to the
humble @decorator.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From michel at dialnetwork.com  Thu Aug 12 06:18:02 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Thu Aug 12 06:28:05 2004
Subject: [Python-Dev] An indentation-based approach to decorators (was
 Another aproach to decorators.)
In-Reply-To: <411AC99B.2040906@ocf.berkeley.edu>
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<411A898F.3060004@ocf.berkeley.edu>
	<20040811142354.661970e0.michel@dialnetwork.com>
	<411AC99B.2040906@ocf.berkeley.edu>
Message-ID: <20040811211802.55a7276e.michel@dialnetwork.com>


On Wed, 11 Aug 2004 18:36:27 -0700
"Brett C." <bac@ocf.berkeley.edu> wrote:

> >>But then how are you supposed to do multiple decorators for the same method?
> > 
> > 
> > Put more than one of them in the same decorate block:
> > 
> > class ...
> > 
> >   decorate staticmethod, locksFile(arg1):
> > 
> >     def meth1(...)
> >       ...
> > 
> >     def meth2(...)
> >       ...
> 
> I still don't like it.  It feels like I am classifying my methods based 
> on the applied decorators instead of thinking of decorators as, well, 
> decorators of methods. 

True, it is whole 'nother pair o' dimes, a different way of looking at
it than has been discussed so far.  But by decorating a whole
indentation block, new ideas can surface beyond decorating methods.

An indentation based decoration syntax can implement quite a few new
features by decorating a whole block including, but not limited to,
methods inside that block. This could possible implement several PEPs
that are really their own subset of a generic block decoration syntax.
Consider the followign decoration blocks: 

  # PEP 319  (disclaimer: my PEP)

  def foo():
   
    # asynchronous code

    decorate synchronized:
      # sychronous code

    # back to async

  # PEP 310

  def foo():  # undecorated
   
    decorate with(expr):
      # acquired block

      def bar():   # decorated
        ...

    # released


  # PEP 316 (method inside)

    decorate inv(expr), pre(expr):
      # invariant expressions

      def foo():  # decorated method
	...   

(although I'm a little fuzzy on that last one)

Possible more, what about __future__ features?

  decorate nested_scopes:
    ...

that's a bit of a stretch, but the point is decorated blocks can
introduce different language features or semantics on a per-block basis,
explicitly.  

Imagine Guido decided nested scopes were a bad idea, and decided to get
in the time machine and change the __future__ so the feature would never
be in the language. Those who chose to go back to the __future__ would
have to stay there, leaving that brilliant turd of an import in all
their code.

Decorated blocks would allow you to experiment explicitly with new
language features or semantics without having to guarantee that they
will be in the __future__.  They never even need to be considered for
the __future__, and if they were, the decoration would then become a
noop.

> I feel methods are the major thing and 
> decorators are just extra fluff on them, not the other way around as 
> this syntax says to me. 
>
> Changing the indentation delineates scope and I 
> don't feel decorators truly represent a change in scope.

I'm no language lawyer, but do all changes in indentation represent
changes in scope?  What about if, for, while, try, etc?

To me, methods are not the "major thing", it's indentation. Of all the
other people I've met who have heard of Python but never used it, almost
all of them know nothing about the langauge but for these facts: it's
based on indentation and few people go back once they get a taste of it.

Decorators preceding a method definition on their own line is the first
thing in Python I can think of that breaks wide of Python's indentation
style, which is what has made me so uncomfortable with it from the
beginning.

-Michel
From mkc at mathdogs.com  Thu Aug 12 06:25:38 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Thu Aug 12 07:10:34 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<lkdwu0bcz7r.fsf@panix2.panix.com>
	<87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp>
	<lkdacx37oxr.fsf@panix2.panix.com>
	<873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <lkdpt5x3rul.fsf@panix2.panix.com>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
> I didn't have that in mind, but I can live with it.  Regexp doesn't
> compare well even with that when you consider error-checking and
> program maintenance.

It's true that regexp matching is pretty much a binary thing--either it does
match or it doesn't.  For human generated files, this isn't so good, but my
domain of interest is machine-generated files, where I "know" that the pattern
will match.  If the match fails in this case, I'm happy enough to just detect
it and bail, possibly with an error message that says "looks like there's a
problem somewhere around char x in line y".

Re maintenance, yeah regexp is pretty terse and ugly.  Generally, though, I'd
rather deal with a reasonably well-considered 80 char regexp than 100 lines of
code that does the same thing.

> It's not obvious to me how to make grammar rules pretty in Python, but
> implementing an SLR parser-generator should be at most a couple of
> days' work to get something you could live with.

There are several of these packages available and I'm all in favor of their
use, particularly for problems of sufficient complexity.  These are currently
hindered, though, by the fact that none have been elected for inclusion in the
standard library.

Furthermore, since we have 're', and it's not going away, I'd really like to
fix this repeated match deficiency, one way or another.

> I agree it would be useful.  I think that a parser generator would be
> equally useful for this, as easy to learn as regexps are (assuming
> somebody comes up with a tasteful way to express grammar rules), and
> more powerful with even a basic implementation.

I will be +1 in favor if someone wants to pursue it.

> BTW, do you have a sample implementation for re.structmatch?  Fredrik
> seems to have some doubt about ease of implementation.

Erik Heneryd posted an intriguing Python prototype, which I'm still looking
at.

>     Mike> I agree that, like list comprehensions (for example), it
>     Mike> needs to be applied with good judgement.
> 
> I don't see the analogy to list comprehensions.

Just that although list comprehensions are cool, you can still make an
unreadable mess with them, as I've found myself doing occasionally.  (nesting
complex comprehensions four levels deep, etc.)

> My objection is that it throws away a lot of structure, and therefore
> is liable to return the _wrong_ parse, or simply an error with no hint
> as to where the data is malformed.

Hmm.  Regarding the lack of error diagnosis, I'm not too concerned about this,
for the reason I mention above.  When 're.structmatch' does fail, though, it
returns a "farthest match position", which will usually be of some aid, I
would think.

Regarding getting the parse wrong, sure, this could happen.  Users will have
to be judicious.  The answer here is to write RE's with some care, realize
that some matches may require further checking after being returned by
re.structmatch, and further realize that some parsing problems are too complex
for this method and require a grammar-based approach instead.  This doesn't
really seem worse than the situation for re.match, though, to me.

> Because they're all too often not "multiple matches".  They're matches
> for different objects that happen to match the same regular
> expression, and the notation for the parser should express that
> without requiring comments.  re.structmatch _encourages_ a style where
> users deliberately throw away structure; your /etc/group parser is an
> example.  (See below.)

Hmm.  You're quibbling with my example.  Yes, I could have written the
expression to "know" that there were only four fields and that the third was a
number, etc.  I chose not to for pedagogical purposes, but it could easily
have been done the other way.  And in practice the programmer can choose the
specificity of the match depending on exactly what they're trying to
accomplish.  Is this really different than what would happen with grammars?

> Furthermore, since this is going to be recursive, users are going to
> end up with "list of list of ..." nested to arbitrary depth, depending
> on how complex the regexp is.  Isn't that what we have Lisp for? <wink>

The shape of the result follows directly from the shape of the grouping in the
RE, yes.  This is construed as a benefit.  And yes, Lisp's backtick (? it's
been a while) expressions were part of what inspired me on this.

> I suppose you would be satisfied if you could represent
> 
> file := file line
> line := group ':' passwd ':' number ':' any '\n'
> group := '[A-Za-z0-9]+'
> passwd := '[A-Za-z0-9]+'
> number := '[0-9]+'
> any := '.*'
> 
> by
> 
> '(([A-Za-z0-9]+:)*.*\n)*'

This isn't really fair.  If you know that that's what you want to parse, then
you could use a more appropriate re.structmatch equivalent like

    p = r'''(?xm)           # VERBOSE, MULTILINE
            (
              (?P<group> [A-Za-z0-9]+)
              (?P<passwd> [A-Za-z0-9]+)
              (?P<number> [0-9]+)
              (?P<any> .*)
              \n
            )*
            \Z              # assert that the entire
                            #   input was matched
         '''

which will return the submatches tagged by those group names.

> improving 'passwd' to check for the characteristic signature of a
> crypted password or a "no password authentication" flag would be
> annoying with re.structmatch, while it's trivial with a parser.

If this signature can be expressed as an RE, then I suspect it would be easy
enough to add.  If not, then probably you'd have to check with code after the
fact, even if you're using a grammar parser.  Or am I missing something?

> I grant that a full-blown parser generator is overkill for your target
> set of applications.  But I stick with my judgement that regexps as
> they are are an attractive nuisance, and re.structmatch would make the
> situation worse in proportion to its usage.  I hope your proposal will
> be superseded either by a scanner package, or even by a parser package.

I think you're saying that you just don't care for regexps, period.  This
reminds me of jwz's quip ("now you have two problems"), and I have some
sympathy for this point of view, but practically speaking 're' is here and
it's not going away any decade soon.  Given that, I'd like to fix this
multiple match deficiency.

I'm in favor of the scanner package, but it doesn't cover the same problem
space as re.structmatch, so I don't see it as a replacement.  A parser package
*might* be, but I don't see any sign that that's coming any time soon.  (I'd
be happy enough to be wrong about this.)

Cheers,
Mike

From martin at v.loewis.de  Thu Aug 12 07:10:44 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 12 07:10:44 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <200408120414.i7C4Er429003@guido.python.org>
References: <1092258146.411a896246d1a@mcherm.com>	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
Message-ID: <411AFBD4.5080903@v.loewis.de>

Guido van Rossum wrote:
> Do we really need byte array literals at all?  I don't expect there to
> be much of a demand.  

I'm not so sure. For example, in httplib, in the line

   h.putrequest('GET', selector)

I would claim that 'GET' is a byte string, not a character string:
it is sent as-is onto the wire, which is a byte-oriented wire.

Now, it would also "work" if it were a character string, which
then gets converted to a byte string using the system default
encoding - but I believe that an application that relies on the
system default encoding is somewhat broken: Explicit is better
than implicit.

> Rather, byte arrays would eventually be returned
> by the read() method when a file is opened in binary mode.  (Isn't
> this roughly how Java does this?)

Java also supports byte arrays in the source, although they are
difficult to type:

    byte[] request = {'G', 'E', 'T'};

As for reading from streams: Java has multiple reader APIs; some
return byte strings, some character strings.

Regards,
Martin
From martin at v.loewis.de  Thu Aug 12 07:15:30 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 12 07:15:40 2004
Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence
In-Reply-To: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au>
References: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au>
Message-ID: <411AFCF2.1050208@v.loewis.de>

Steven Kah Hien Wong wrote:
> Mind you, I find the current behaviour is much more useful, so I am
> saying the documentation is in error, not the implementation. :) Have I
> simply misread? 

Yes. You are confusing instance attributes and class attributes.

> If so, maybe that paragraph should be rewritten to make
> it more clear. With an example like the above, perhaps?

Can you provide a patch? Please upload to sf.net/projects/python.

Regards,
Martin
From adurdin at gmail.com  Thu Aug 12 07:16:32 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Thu Aug 12 07:16:37 2004
Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered
In-Reply-To: <cfe8mc$ujt$1@sea.gmane.org>
References: <1092179578.4119567af011d@webmail.oregonstate.edu>
	<cfe8mc$ujt$1@sea.gmane.org>
Message-ID: <59e9fd3a0408112216345f70b9@mail.gmail.com>

On Thu, 12 Aug 2004 01:04:43 +0200, Christophe Cavalaria
<chris.cavalaria@free.fr> wrote:
> barnesc@engr.orst.edu wrote:
> >
> >  - For which method is it visually easier to find the function def?
> 
> None of them. A good syntax coloring would even make it easier in fact.

One very nice thing with Python to date is that it is very easy to
read even without syntax coloring--I think that's a feature worth
keeping.

> On the second hand, the Option B makes it much harder to find the function
> code once you've found the function def.

Barely harder at all than the usual function with a docstring -- you
just look below the docstring.

> >  - For which method is the code in the most logical order?
> 
> Option A of course. Since the decorator can be seen as a function that takes
> the defined function as it's first parameter, it's logical to place the
> decorator before the definition.
> 
> @staticmethod
> def f():
>     pass
> 
> is a short version of
> 
> f = staticmethod(
>     def f():
>         pass
> )

Except that in actual fact it is a short version of

def f():
    pass
f = staticmethod(f)

> > Note that Option A causes you to skim all the way through
> > the docstring and the decorators to find out which function
> > is being defined.  At this point, you have to start over
> > at the docstring, to find out what the function actually does.
> 
> This is false of course, any good syntax coloring would give you a good
> contrast between the function itself and the decorators. That way, it'll be
> easy to find the first line that isn't colored like a decorator. On the
> Option B, you'll have to identify 3 blocs of data : the function def, the
> decorator bloc and the function body.

Relying on syntax coloring is a Bad Thing. :)
From tim.peters at gmail.com  Thu Aug 12 07:19:00 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 12 07:19:03 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1092232683.9945.26.camel@localhost>
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost>
Message-ID: <1f7befae040811221916a955fd@mail.gmail.com>

[Barry Warsaw]
> The elimination of the trailing 's' is definitely a benefit, as is the
> adoption of $-placeholders, since that tends to be the placeholder
> introducing character in many other languages.  In terms of usability,
> it's a win.

Allowing to skip the {} brackets when they aren't needed is also a win
-- and I expect the most common case.

I do object to this part:

     If the $ character appears at the end of the line, or is followed
     by any other character than those described above, it is treated
     as if it had been escaped, appearing in the resulting string
     unchanged.

There's already a facility to escape $, and it's dead easy to use.  $
isn't a frequently needed character either in most apps.  So escaping
$ "by magic" too is, I think, more likely to make typing errors harder
to find than to do anyone real good.

Better to reserve that magic for the Sloppy^H^H^H^H^H^HSafeTemplate() class.
From greg at cosc.canterbury.ac.nz  Thu Aug 12 07:19:58 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug 12 07:20:08 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <411AFBD4.5080903@v.loewis.de>
Message-ID: <200408120519.i7C5JwQV011984@cosc353.cosc.canterbury.ac.nz>

Martin:

> Now, it would also "work" if it were a character string, which
> then gets converted to a byte string using the system default
> encoding

Or the encoding associated with the file object, which
could be set to ascii, either by default or explicitly.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Thu Aug 12 07:22:21 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug 12 07:22:28 2004
Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered
In-Reply-To: <59e9fd3a0408112216345f70b9@mail.gmail.com>
Message-ID: <200408120522.i7C5MLl8011994@cosc353.cosc.canterbury.ac.nz>

Andrew Durdin <adurdin@gmail.com>:

> One very nice thing with Python to date is that it is very easy to
> read even without syntax coloring--I think that's a feature worth
> keeping.

So do I. Currently I edit Python on my Mac with BBEdit Lite,
without syntax colouring, and I haven't missed it. I wouldn't
enjoy starting to miss it now.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From guido at python.org  Thu Aug 12 07:39:52 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 07:40:00 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: Your message of "Thu, 12 Aug 2004 01:19:00 EDT."
	<1f7befae040811221916a955fd@mail.gmail.com> 
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost> 
	<1f7befae040811221916a955fd@mail.gmail.com> 
Message-ID: <200408120539.i7C5dqL29226@guido.python.org>

This PEP reminded me of the "boo" language (http://boo.codehaus.org/),
a new scripting language for the CLI strongly inspired by Python
(especially the Python 3000 variant :-).

It has string interpolation like this
(http://boo.codehaus.org/String+Interpolation):

print("Now is ${date.Now}.")
print("Tomorrow will be ${date.Now + 1d}")

They require use of {...} after the $, and you don't need to escape
the $ if not followed by {.

Just another design variant (of course they may change it all tomorrow
:-).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug 12 07:45:04 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 07:45:12 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: Your message of "Thu, 12 Aug 2004 07:10:44 +0200."
	<411AFBD4.5080903@v.loewis.de> 
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org> 
	<411AFBD4.5080903@v.loewis.de> 
Message-ID: <200408120545.i7C5j4W29250@guido.python.org>

> Guido van Rossum wrote:
> > Do we really need byte array literals at all?  I don't expect there to
> > be much of a demand.  
> 
> I'm not so sure. For example, in httplib, in the line
> 
>    h.putrequest('GET', selector)
> 
> I would claim that 'GET' is a byte string, not a character string:
> it is sent as-is onto the wire, which is a byte-oriented wire.
> 
> Now, it would also "work" if it were a character string, which
> then gets converted to a byte string using the system default
> encoding - but I believe that an application that relies on the
> system default encoding is somewhat broken: Explicit is better
> than implicit.

Alternatively, we could postulate that the stream to which the string
is written determines the encoding.  This would still be explicit.

Anyway, if we really do have enough use cases for byte array literals,
we might add them.  I still think that would be confusing though,
because byte arrays are most useful if they are mutable: and then we'd
have mutable literals -- blechhhh!

--Guido van Rossum (home page: http://www.python.org/~guido/)
From jcarlson at uci.edu  Thu Aug 12 07:41:42 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu Aug 12 07:47:48 2004
Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence
In-Reply-To: <411AFCF2.1050208@v.loewis.de>
References: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au>
	<411AFCF2.1050208@v.loewis.de>
Message-ID: <20040811224013.BDE8.JCARLSON@uci.edu>

> > If so, maybe that paragraph should be rewritten to make
> > it more clear. With an example like the above, perhaps?
> 
> Can you provide a patch? Please upload to sf.net/projects/python.

Perhaps the following example (or a variant) should be included in the
tutorial as an example.

>>> class a:
...     x = 0
...
>>> class b(a):
...     x = 1
...
>>> class c(a):
...     pass
...
>>> class d(a,b):
...     pass
...
>>> class e(b,a):
...     pass
...
>>> d().x
0
>>> e().x
1


 - Josiah

From rnd at onego.ru  Thu Aug 12 07:51:13 2004
From: rnd at onego.ru (Roman Suzi)
Date: Thu Aug 12 07:51:24 2004
Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered
In-Reply-To: <200408120522.i7C5MLl8011994@cosc353.cosc.canterbury.ac.nz>
References: <200408120522.i7C5MLl8011994@cosc353.cosc.canterbury.ac.nz>
Message-ID: <Pine.LNX.4.58.0408120947310.4253@suzi.com.onego.ru>

On Thu, 12 Aug 2004, Greg Ewing wrote:

> Andrew Durdin <adurdin@gmail.com>:
>
> > One very nice thing with Python to date is that it is very easy to
> > read even without syntax coloring--I think that's a feature worth
> > keeping.
>
> So do I. Currently I edit Python on my Mac with BBEdit Lite,
> without syntax colouring, and I haven't missed it. I wouldn't
> enjoy starting to miss it now.
>
> Greg Ewing, Computer Science Dept, +--------------------------------------+

The only useful sense for syntax colouring right now is to
highlight reserved keywords and built-in names. There are so many
of them, that it is already hard to find a name for a variable ;-)

Probably, Python should have an option to protect built-in names.

Sincerely yours, Roman A.Suzi
-- 
 - Petrozavodsk - Karelia - Russia - mailto:rnd@onego.ru -

From adurdin at gmail.com  Thu Aug 12 08:01:10 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Thu Aug 12 08:01:12 2004
Subject: [Python-Dev] Similarity of docstrings and decorators
Message-ID: <59e9fd3a040811230159e61ff3@mail.gmail.com>

It recently struck me that both decorators and docstrings are
syntactic sugar for code that would otherwise follow the function
definition, vide:

@staticmethod
def foo():
    """Frobs the baz"""
    pass

versus

def foo():
    pass
foo.__doc__ = """Frobs the baz"""
foo = staticmethod(foo)

This suggests two things to me: firstly, that the proposed decorator
syntaxes which place the decorators after the def, beside the
docstring, are the most consistent with existing syntax (that for
docstrings); and secondly, that this suggests that it might be better
to have an explicit function "header" section where both decorators
and docstrings will reside. For a conceptual example (not a syntax
proposal):

def foo():
    header:
        staticmethod
        """Frobs the baz"""

    pass

I realise there would be some issues with this--backward compatibility
with existing docstrings (which could probably be resolved by a
gradual deprecation), and Guido's concern about having to "peek inside
the block"--but I thought it worth mentioning this generalised
conception of both features, that highlights their similarity.

(As for peeking inside the function, if a separate header area existed
for the "important
external properties", then it, together with the signature (which is
arguably the most important part of a function definition) would
comprise all that you would need to know externally.)
From adurdin at gmail.com  Thu Aug 12 08:05:56 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Thu Aug 12 08:06:00 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <200408120539.i7C5dqL29226@guido.python.org>
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost> 
	<1f7befae040811221916a955fd@mail.gmail.com>
	<200408120539.i7C5dqL29226@guido.python.org>
Message-ID: <59e9fd3a0408112305fa1b667@mail.gmail.com>

One thing the PEP doesn't make clear: do these templates only
interpolate string variables, or will they handle anything with a
__str__ method?

e.g., will the following raise an exception:

print Template("I am ${height} centimetres tall.") % {height: 183}
From greg at cosc.canterbury.ac.nz  Thu Aug 12 08:11:53 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug 12 08:11:59 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <200408120545.i7C5j4W29250@guido.python.org>
Message-ID: <200408120611.i7C6Brqt012067@cosc353.cosc.canterbury.ac.nz>

Guido:

> Anyway, if we really do have enough use cases for byte array
> literals, we might add them.  I still think that would be confusing
> though, because byte arrays are most useful if they are mutable: and
> then we'd have mutable literals -- blechhhh!

Perhaps the constructor for a byte array could accept a string
argument as long as it contained only ascii characters?

   h.putrequest(bytes('GET'), selector)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From tim.peters at gmail.com  Thu Aug 12 08:25:21 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 12 08:25:29 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae0408101258431039a1@mail.gmail.com>
References: <000001c47e88$81ef1d40$5635c797@oemcomputer>
	<1f7befae0408092348583f0953@mail.gmail.com>
	<2mvffqzum9.fsf@starship.python.net>
	<1f7befae0408101258431039a1@mail.gmail.com>
Message-ID: <1f7befae04081123253f08bb71@mail.gmail.com>

Here's some puzzling evidence (WinXP Pro SP1).  The test driver, for
convenience:

"""
import compiler, sys

f = open('../Lib/test/test_parser.py')
guts = f.read()
f.close()

def ouch(n):
    if n == 0:
        return compiler.compile(guts, "<string>", "exec")
    else:
        return ouch(n-1)

for n in range(0, 50):
    try:
        ouch(n)
        msg = 'ok'
    except KeyboardInterrupt:
        raise
    except Exception, msg:
        msg = str(sys.exc_info()[0]) + ' ' + str(msg)
    print n, msg
"""

The loop starts at 0 this time, and I'm only looking at a debug-build Python.

The stack size for an .exe can be specified to 4-byte granularity. 
Here's the largest stack size at which that produces no output at all:

C:\Code\python\PCbuild>editbin /stack:1310720 python_d.exe
Microsoft (R) COFF Binary File Editor Version 6.00.8447
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.

C:\Code\python\PCbuild>python_d temp.py

C:\Code\python\PCbuild>echo %ERRORLEVEL%
128


Add 4 measly bytes, and it's a world of difference:

C:\Code\python\PCbuild>editbin /stack:1310724 python_d.exe
Microsoft (R) COFF Binary File Editor Version 6.00.8447
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.



C:\Code\python\PCbuild>python_d temp.py
0 ok
1 ok
...
41 ok
42 ok

C:\Code\python\PCbuild>echo %ERRORLEVEL%
128

So it still vanishes early, but gets thru the entire
compiler.compile() business + 42 additional stacked Python calls! 
That's awfully impressive for four bytes.  That suggests "the problem"
isn't in detecting Python-level recursion.

Under the debugger, it dies with an access violation at that point. 
Alas, the C-level Python call stack is no longer anywhere in evidence
then.  Instead it's 7 levels deep in ntdll.dll, and is "in the middle"
of four consecutive Pentium PUSH instructions.  That's evidence that
the stack has been blown <wink>.

The debugger Output window does show ntdll.dll suffering a Stack
Overflow exception.  It then shows a variety of Access Violations in
ntdll.dll trying to read and write various locations, presumably in a
failing attempt to report the stack overflow.
From martin at v.loewis.de  Thu Aug 12 08:26:40 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 12 08:26:34 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <200408120545.i7C5j4W29250@guido.python.org>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org>
Message-ID: <411B0DA0.4030109@v.loewis.de>

Guido van Rossum wrote:
> Anyway, if we really do have enough use cases for byte array literals,
> we might add them.  I still think that would be confusing though,
> because byte arrays are most useful if they are mutable: and then we'd
> have mutable literals -- blechhhh!

I see. How would you like byte array displays then? This is the approach
taken in the other languages: Everytime the array display is executed, a
new array is created. There is then no problem with that being mutable.

Of course, if the syntax is too similar to string literals, people might
be tricked into believing they are actually literals. Perhaps

   bytes('G','E','T')

would be sufficient, or even

   bytes("GET")

which would implicitly convert each character to Latin-1.

Regards,
Martin
From martin at v.loewis.de  Thu Aug 12 08:29:00 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 12 08:28:56 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <200408120611.i7C6Brqt012067@cosc353.cosc.canterbury.ac.nz>
References: <200408120611.i7C6Brqt012067@cosc353.cosc.canterbury.ac.nz>
Message-ID: <411B0E2C.5000608@v.loewis.de>

Greg Ewing wrote:
> Perhaps the constructor for a byte array could accept a string
> argument as long as it contained only ascii characters?
> 
>    h.putrequest(bytes('GET'), selector)

That is probably most reasonable. It would be harmless to extend it
to Latin-1, which would allow to represent all byte values in a
string literal - even using the appropriate \x escapes.

Regards,
Martin
From ncoghlan at iinet.net.au  Thu Aug 12 08:49:11 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Aug 12 08:49:23 2004
Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence
In-Reply-To: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au>
References: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au>
Message-ID: <411B12E7.6060808@iinet.net.au>

Steven Kah Hien Wong wrote:

> Maybe I have misread, but I have interpreted that as meaning the two
> children will share the same instance of the base class. So if one child
> changes its parent, the other child will pick up that change, too.

This interpretation seems correct to me.

> But I did some quick tests, and it turns out this isn't so.

It *is* so if you actually change the common parent, rather than one of 
the siblings (see modified example below).

In your original example, the new 'x' was stored inside ChildOne's class 
dictionary, and so ChildTwo was not affected. To see the effect of the 
shared parent, you need to actually change the value of 'x' that is 
stored in CommonBase (as happens in the example below).

=====================
      class CommonBase:
          x = 0

      class ChildOne(CommonBase):
          def foo(self): CommonBase.x = 1  ##### Change here

      class ChildTwo(CommonBase):
          pass

      class ChildOneAndTwo(ChildOne, ChildTwo):
          def run(self):
              ChildOne.foo(self)  ##### Change Here
              print "one =", ChildOne.x
              print "two =", ChildTwo.x

      ChildOneAndTwo().run()

=====================
Output is now:
      $ python multi.py
      one = 1
      two = 1
=====================

In relation to Martin's post, the text is really talking about a problem 
with 'instance' variables in the presence of multiple inheritance:

=====================
class Base(object):
   def __init__(self):
     super(Base, self).__init__()
     self.x = 0
     print "Base initialised"

   def run(self): print "I am", self.x

class One(Base):
   def __init__(self):
     super(One, self).__init__()
     self.x = 1
     print "One initialised"


class Two(Base):
   def __init__(self):
     super(Two, self).__init__()
     self.x = 2
     print "Two initialised"

class OneTwo(One, Two):
   def __init__(self):
     super(OneTwo, self).__init__()

OneTwo().run()
=====================
Output is:
   Base initialised
   Two initialised
   One initialised
   I am 1
=====================

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From lists at hlabs.spb.ru  Thu Aug 12 12:58:11 2004
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Thu Aug 12 08:52:10 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <16666.38863.339305.573591@montanaro.dyndns.org>
References: <1092258146.411a896246d1a@mcherm.com>
	<16666.38863.339305.573591@montanaro.dyndns.org>
Message-ID: <411B4D43.70308@hlabs.spb.ru>

Skip Montanaro wrote:
>     >> I think this would be a good idea; maybe the name should be
>     >> baseinteger?
> 
>     Michael> I would like to urge caution before making this change. Despite
>     Michael> what the PEP may say, I actually think that creating a
>     Michael> 'baseint' type is the WRONG design choice for the long term.  I
>     Michael> envision an eventual Python which has just one type, called
>     Michael> 'int'.
> 
> I agree.  I made a suggestion that we consider the entire tree of numeric
> types,

Is it a good time to consider the entrie tree of ALL types then?

For example:

object
     (Tree of numeric types as suggested by Gareth)
     number
        complex
        real
           (Decimal)
           float
           rational
              fraction
              integer
                 int
                     bool
                 long
     sequence
         buffer
         basestring
             str
             unicode
         list
         tuple
         mapping
            dict
     set
     frozenset
     file

> but I had int/long unification in the back of my mind as well.  I
> will take /F's suggestion and poke around the peps when I have some time,
> but I see no pressing reason a base integer class, however it's spelled,
> needs to be added for 2.4.

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru
From ncoghlan at iinet.net.au  Thu Aug 12 09:10:55 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Aug 12 09:11:05 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <411B4D43.70308@hlabs.spb.ru>
References: <1092258146.411a896246d1a@mcherm.com>	<16666.38863.339305.573591@montanaro.dyndns.org>
	<411B4D43.70308@hlabs.spb.ru>
Message-ID: <411B17FF.7060607@iinet.net.au>

Dmitry Vasiliev wrote:

> Is it a good time to consider the entrie tree of ALL types then?

Please, no. One of the best things about Python is that, for most 
purposes, you can get the appropriate reaction from the standard library 
simply by providing the relevant magic methods (e.g. __iter__ if you 
want to be treated like a sequence).

Normal class inheritance means you are inheriting the *whole* interface, 
as well as the implementation of any parts you don't override.

So 'sequence' for instance, couldn't have any methods or do much of 
anything, without causing potentially undesirable behaviour.

About the only exception that makes any sense to me is for exceptions, 
where using inheritance for classification seems to be universal 
practice in all languages with exceptions (since 'catch X' generally 
means catch X, or any of its subclasses). (Bad pun noted, and not avoided).

Localised type hierarchies where it is useful, OK. Inheriting from 
object as a way to say 'new-style class please', OK. But please, please, 
please, lets keep interface introspection the dominant way of 
determining what can be done with a particular object. Creating a 'grand 
hierarchy of everything' runs directly counter to that goal.

Regards,
Nick.


-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From sdm7g at mac.com  Thu Aug 12 09:15:50 2004
From: sdm7g at mac.com (Steven Majewski)
Date: Thu Aug 12 09:16:01 2004
Subject: [Python-Dev] CGI module [was: New Subscriber]
In-Reply-To: <20040507043316.GB45563@unixbeast.com>
References: <20040507043316.GB45563@unixbeast.com>
Message-ID: <70C26872-EC2F-11D8-B23F-000A957D4F2A@mac.com>

[ OK. This is an old thread, but it's a new concern for me.
   I was researching to see what the current state of things was
   before making any public comments. I though I might as well provide
   this background to Dave Hastings, and also ask anyone involved in 
those
   earlier threads if there has been any more recent work on this that
   I might have missed. It looks like there was a flurry of discussion
   about this last fall but nothing seems to have come of it. ]

On May 7, 2004, at 12:33 AM, Dave Hastings wrote:

>  [ ... ] The project that was decided upon, with the advice of the 
> Professor, was to make a CGIpm python module that reflected CGI.pm for 
> perl.  We noticed that there is a current cgi module for python and 
> even a cgipm module.  These modules are very useful, but did not 
> include all the functions that CGI.pm for perl provides.  It turns out 
> that my Professor really likes python and would have loved to see a 
> CGIpm python module for python that was like the CGI.pm module for 
> perl. [ ... ]


Dave,

	It's been a few years since I needed to do any cgi programming in 
Python.
	When I recently had to dig into it, I was SHOCKED to discover what a 
mess
	the current cgi module is. ( My second shock was when I looked at the 
source
	to try to figure things out, and discovered my name there at the top! 
;-(

	It seems to be one of those things that just grew wild and didn't get 
much pruning.
	Mike McLay at NIST did the first version.
	I made some changes, but tried to keep backwards compatibility.
	Someone else came along and created a new set of interfaces, which 
both added
	and lost some functionality, but kept the older versions around for 
compatibility.
	( I couldn't seem to get the new, improved version to do what I 
wanted, so I
	  had to fall back on using the old compatability version! )

	None of them seem very nice to use.
	Probably one of the few places where Python is 'uglier' than perl.
	And searching various Python archives, I find a history of similar 
complaints.


Among the "Deferred, Abandoned, Withdrawn, and Rejected PEPs" is:

PEP 222 (Dec 2000):  http://www.python.org/peps/pep-0222.html

which first proposes dumping the cgi module and starting over again:

>  The cgi.py module would be deprecated.  (XXX A new module or
>     package name hasn't been chosen yet: 'web'?  'cgilib'?)

... and a partial outline of a proposed interface.

( along with: Extended HTTP functionality and WEBDAV:
	http://www.python.org/peps/pep-0268.html )


In the Sept. 2003 Python-Dev archives:
See the thread that begins with Subject: "[Python-Dev] Improving the 
CGI module":
http://mail.python.org/pipermail/python-dev/2003-September/038128.html


And the next month, a similar discussion picked up in the Web-SIG
mailing list. See Web-SIG October 2003 Archives:
http://mail.python.org/pipermail/web-sig/2003-October/thread.html

See the many messages concerning "Form field dictionaries" , as well
as several other threads in that months archive.


As far as I can tell, all of those threads died out without any actual
code being produced. However, they do give a good enumeration of what's
wrong with the current cgi module and what's needed in a new module.

Please let me know if there is any more current history I've missed
in my search. ( I've been out of the python-dev loop for a while! )

I guess  the Web-SIG is the best place for any further discussion.
http://mail.python.org/mailman/listinfo/web-sig



Dave: I'm not sure that following CGI.pm is the best possible design,
but I'ld like to see what you're planning. If nothing else is happening
on this front, I may be able to take a swing at it when I return from my
vacation.



-- Steve Majewski

From mwh at python.net  Thu Aug 12 12:33:48 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug 12 12:33:50 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae04081123253f08bb71@mail.gmail.com> (Tim Peters's
	message of "Thu, 12 Aug 2004 02:25:21 -0400")
References: <000001c47e88$81ef1d40$5635c797@oemcomputer>
	<1f7befae0408092348583f0953@mail.gmail.com>
	<2mvffqzum9.fsf@starship.python.net>
	<1f7befae0408101258431039a1@mail.gmail.com>
	<1f7befae04081123253f08bb71@mail.gmail.com>
Message-ID: <2mekmczlv7.fsf@starship.python.net>

Tim Peters <tim.peters@gmail.com> writes:

> Here's some puzzling evidence (WinXP Pro SP1).  The test driver, for
> convenience:
>
> """
> import compiler, sys
>
> f = open('../Lib/test/test_parser.py')
> guts = f.read()
> f.close()
>
> def ouch(n):
>     if n == 0:
>         return compiler.compile(guts, "<string>", "exec")
>     else:
>         return ouch(n-1)
>
> for n in range(0, 50):
>     try:
>         ouch(n)
>         msg = 'ok'
>     except KeyboardInterrupt:
>         raise
>     except Exception, msg:
>         msg = str(sys.exc_info()[0]) + ' ' + str(msg)
>     print n, msg
> """
>
> The loop starts at 0 this time, and I'm only looking at a debug-build Python.
>
> The stack size for an .exe can be specified to 4-byte granularity. 
> Here's the largest stack size at which that produces no output at all:
>
> C:\Code\python\PCbuild>editbin /stack:1310720 python_d.exe
> Microsoft (R) COFF Binary File Editor Version 6.00.8447
> Copyright (C) Microsoft Corp 1992-1998. All rights reserved.
>
> C:\Code\python\PCbuild>python_d temp.py
>
> C:\Code\python\PCbuild>echo %ERRORLEVEL%
> 128
>
>
> Add 4 measly bytes, and it's a world of difference:
>
> C:\Code\python\PCbuild>editbin /stack:1310724 python_d.exe
> Microsoft (R) COFF Binary File Editor Version 6.00.8447
> Copyright (C) Microsoft Corp 1992-1998. All rights reserved.
>
>
>
> C:\Code\python\PCbuild>python_d temp.py
> 0 ok
> 1 ok
> ...
> 41 ok
> 42 ok
>
> C:\Code\python\PCbuild>echo %ERRORLEVEL%
> 128
>
> So it still vanishes early, but gets thru the entire
> compiler.compile() business + 42 additional stacked Python calls! 
> That's awfully impressive for four bytes.  That suggests "the problem"
> isn't in detecting Python-level recursion.

Has the failure mode of alloca() changed?  I take it you're building
with VC++ 7.1?  What happens for a VC++ 6 build?

Hmm, a moment with msdn suggests that there's been no significant
changes here, although the documentation is for _alloca(), and Python
calls alloca().  That can't make any difference, can it?

It still smells like a tool change to me.

Cheers,
mwh

-- 
58. Fools ignore complexity. Pragmatists suffer it. Some can avoid
    it. Geniuses remove it.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html
From mwh at python.net  Thu Aug 12 12:39:35 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug 12 12:39:36 2004
Subject: [Python-Dev] pep 318, Decorators for Functions, Methods and
	Classes
In-Reply-To: <5.1.1.6.0.20040811155219.02a56a00@mail.telecommunity.com>
	(Phillip J. Eby's message of "Wed, 11 Aug 2004 15:54:23 -0400")
References: <20040811011116.GF23725@performancedrivers.com>
	<A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org>
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
	<200408072105.i77L5Ea10965@guido.python.org>
	<20040811011116.GF23725@performancedrivers.com>
	<5.1.1.6.0.20040811155219.02a56a00@mail.telecommunity.com>
Message-ID: <2macx0zllk.fsf@starship.python.net>

"Phillip J. Eby" <pje@telecommunity.com> writes:

> At 03:44 PM 8/11/04 -0400, Jack Diederich wrote:
>> I have a mostly working patch for class decorators but it gets an
>> XXX ambiguity
>>between
>>
>>funcdef: [decorators] 'def' NAME parameters ':' suite
>>and
>>classdef: [decorators] 'class' NAME [ '(' testlist ')' ] ':' suite
>>
>> It works as advertised with the pre-decorators funcdef definition
>> (which removes the ambiguity).
>>Help?
>
>
> How about this?
>
>      decorated_assignment: [decorators] (funcdef | classdef)

decorated_definition or something would be better surely?

>      classdef: 'class' NAME [ '(' testlist ')' ] ':' suite
>      funcdef: 'def' NAME parameters ':' suite

But yes, the answer to this problem is "left factoring", which google
can tell you all about...

Cheers,
mwh

-- 
  For their next act, they'll no doubt be buying a firewall 
  running under NT, which makes about as much sense as 
  building a prison out of meringue.                     -- -:Tanuki:-
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
From erik at heneryd.com  Thu Aug 12 12:44:26 2004
From: erik at heneryd.com (Erik Heneryd)
Date: Thu Aug 12 12:44:32 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete, Structured Regular
	Expression Group Matching
In-Reply-To: <lkdpt5x3rul.fsf@panix2.panix.com>
References: <lkdekmlvv8k.fsf@panix2.panix.com>	<lkdwu0bcz7r.fsf@panix2.panix.com>	<87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp>	<lkdacx37oxr.fsf@panix2.panix.com>	<873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp>
	<lkdpt5x3rul.fsf@panix2.panix.com>
Message-ID: <411B4A0A.1030201@heneryd.com>

Mike Coleman wrote:
> "Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Re maintenance, yeah regexp is pretty terse and ugly.  Generally, though, I'd
> rather deal with a reasonably well-considered 80 char regexp than 100 lines of
> code that does the same thing.

ditto


>>It's not obvious to me how to make grammar rules pretty in Python, but
>>implementing an SLR parser-generator should be at most a couple of
>>days' work to get something you could live with.
> 
> 
> There are several of these packages available and I'm all in favor of their
> use, particularly for problems of sufficient complexity.  These are currently
> hindered, though, by the fact that none have been elected for inclusion in the
> standard library.
> 
> Furthermore, since we have 're', and it's not going away, I'd really like to
> fix this repeated match deficiency, one way or another.

Well, I guess that if you want structmatch into the stdlib you'll have 
to show that it's better than it's alternatives.  Including those parser 
packages.


>>BTW, do you have a sample implementation for re.structmatch?  Fredrik
>>seems to have some doubt about ease of implementation.
> 
> 
> Erik Heneryd posted an intriguing Python prototype, which I'm still looking
> at.

You'd still have to do a real implementation.  If it can't be done 
without rewriting a whole lot of code, that would be a problem.


>>My objection is that it throws away a lot of structure, and therefore
>>is liable to return the _wrong_ parse, or simply an error with no hint
>>as to where the data is malformed.
> 
> 
> Hmm.  Regarding the lack of error diagnosis, I'm not too concerned about this,
> for the reason I mention above.  When 're.structmatch' does fail, though, it
> returns a "farthest match position", which will usually be of some aid, I
> would think.
> 
> Regarding getting the parse wrong, sure, this could happen.  Users will have
> to be judicious.  The answer here is to write RE's with some care, realize
> that some matches may require further checking after being returned by
> re.structmatch, and further realize that some parsing problems are too complex
> for this method and require a grammar-based approach instead.  This doesn't
> really seem worse than the situation for re.match, though, to me.

Hmm... think this is the wrong approach.  Your PEP is not just about 
"structured matching", it tries to deal with a couple of issues and I 
think it would be better to address them separately, one by one:

* Parsing/scanning - this is mostly what's been discussed so far...

* Capturing repeated groups - IMO nice-to-have (tm) but not something I 
would lose sleep over.  Hard to do.

* Partial matches - would be great for debugging more complex regexes. 
Why not a general re.RAISE flag raising an exception on failure?

* ... ?


Erik
From mak at trisoft.com.pl  Thu Aug 12 13:08:45 2004
From: mak at trisoft.com.pl (Grzegorz Makarewicz)
Date: Thu Aug 12 13:08:46 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae04081123253f08bb71@mail.gmail.com>
References: <000001c47e88$81ef1d40$5635c797@oemcomputer>	<1f7befae0408092348583f0953@mail.gmail.com>	<2mvffqzum9.fsf@starship.python.net>	<1f7befae0408101258431039a1@mail.gmail.com>
	<1f7befae04081123253f08bb71@mail.gmail.com>
Message-ID: <411B4FBD.9090100@trisoft.com.pl>

Tim Peters wrote:
> 
> So it still vanishes early, but gets thru the entire
> compiler.compile() business + 42 additional stacked Python calls! 
> That's awfully impressive for four bytes.  That suggests "the problem"
> isn't in detecting Python-level recursion.
> 
> Under the debugger, it dies with an access violation at that point. 
> Alas, the C-level Python call stack is no longer anywhere in evidence
> then.  Instead it's 7 levels deep in ntdll.dll, and is "in the middle"
> of four consecutive Pentium PUSH instructions.  That's evidence that
> the stack has been blown <wink>.
> 
> The debugger Output window does show ntdll.dll suffering a Stack
> Overflow exception.  It then shows a variety of Access Violations in
> ntdll.dll trying to read and write various locations, presumably in a
> failing attempt to report the stack overflow.

I have copied stack check into PyObject_Malloc and after X calls to 
PyEval_EvalFrame, fast_function python dies in malloc(msvcrt) and 
HeapAlloc (ntdll) as described by Tim.

After this patch most ocurring exception is MemoryError, but for
141, 171, 201 and 231 python raises SyntaxError - always witout traceback :(

mak

relevant part from obmalloc.c:

PyObject_Malloc(size_t nbytes)
{
     block *bp;
     poolp pool;
     poolp next;
     uint size;

#ifdef USE_STACKCHECK
     if (PyOS_CheckStack()) {
         return NULL;
     }
#endif

From arigo at tunes.org  Thu Aug 12 11:53:13 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu Aug 12 13:44:12 2004
Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence
In-Reply-To: <20040811224013.BDE8.JCARLSON@uci.edu>
References: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au>
	<411AFCF2.1050208@v.loewis.de>
	<20040811224013.BDE8.JCARLSON@uci.edu>
Message-ID: <20040812095313.GA5276@vicky.ecs.soton.ac.uk>

Hello,

On Wed, Aug 11, 2004 at 10:41:42PM -0700, Josiah Carlson wrote:
> Perhaps the following example (or a variant) should be included in the
> tutorial as an example.

Including this example would be confusing for two reasons.

One is that it doesn't answer the original concern, which was not about class
attributes but instance attributes, e.g.:

class point:
  def move_to_the_right(self):
    x, y = self.pos
    self.pos = x+1, y

class myfile:
  def seek(self, newpos):
    self.pos = newpos
  def read(self):
    return self.data[self.pos:]

In this example, a class inheriting from both 'point' and 'myfile' will be in
trouble because the same instance attribute 'pos' is used with different
meanings by the methods of the base classes.

Another concern with your example is that some of the behavior it relies upon
is specific to old-style classes (which might be dropped at some time long in
the future).  Try to do the same with new-style classes:

class a(object):    # forces new-style classes
  pass

class b(a):
  pass

class d(a,b):
TypeError: Cannot create a consistent method resultion
order (MRO) for bases b, a

A saner example along those lines would be something like:

class a:
  pass

class b(a):
  x = 1

class c(a):
  x = 2

class d(b,c):
  pass

class e(c,b):
  pass

d().x   # 1
e().x   # 2


A bient?t,

Armin.
From gmccaughan at synaptics-uk.com  Thu Aug 12 14:06:58 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Thu Aug 12 14:07:30 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <1092258146.411a896246d1a@mcherm.com>
References: <1092258146.411a896246d1a@mcherm.com>
Message-ID: <200408121306.58446.gmccaughan@synaptics-uk.com>

On Wednesday 2004-08-11 22:02, Michael Chermside wrote:

> I would like to urge caution before making this change. Despite
> what the PEP may say, I actually think that creating a 'baseint'
> type is the WRONG design choice for the long term. I envision
> an eventual Python which has just one type, called 'int'. The
> fact that an efficient implementation is used when the ints
> are small and an arbitrary-precision version when they get too
> big would be hidden from the user by automatic promotion of
> overflow. (By "hidden" I mean the user doesn't need to care, not
> that they can't find out if they want to.) We are almost there
> already, but if people start coding to 'baseinteger' it takes
> us down a different path entirely. 'basestring' is a completely
> different issue -- there will always be a need for both unicode
> and 8-bit-strings as separate types.

This is why "integer" is a better name than "baseinteger".
For now it can be the common supertype of int and long.
In the future, it can be the name of the single integer
type.

-- 
g

From john at grulic.org.ar  Thu Aug 12 14:47:38 2004
From: john at grulic.org.ar (John Lenton)
Date: Thu Aug 12 14:47:45 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
In-Reply-To: <1092257611.411a874b5094e@mcherm.com>
References: <1092257611.411a874b5094e@mcherm.com>
Message-ID: <20040812124738.GC8843@grulic.org.ar>

On Wed, Aug 11, 2004 at 01:53:31PM -0700, Michael Chermside wrote:
> 
> But lots of languages make other choices. The number of languages
> that have a single numeric type is pretty large... perl comes to
> mind, but I bet I could list 10 others without too much trouble.

perl doesn't have a single numeric type. Depending on how deep you
look into the issue, it has none (only a scalar type), or several
(scalars that are assigned integers are integers until they are
promoted to float when used floatishly).

-- 
John Lenton (john@grulic.org.ar) -- Random fortune:
What's so funny?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040812/a81e7074/attachment.pgp
From mal at egenix.com  Thu Aug 12 15:15:37 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 12 15:15:41 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <411920E4.9020404@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>
	<4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de>
Message-ID: <411B6D79.2040503@egenix.com>

Hi Walter,

I don't have time to comment on this this week, I'll respond
next week.

Overall, I don't like the idea of adding extra
APIs breaking the existing codec API. I believe that we can
extend stream codecs to also work in a feed mode without
breaking the API.

Walter D?rwald wrote:
> OK, here a my current thoughts on the codec problem:
> 
> The optimal solution (ignoring backwards compatibility)
> would look like this: codecs.lookup() would return the
> following stuff (this could be done by replacing the
> 4 entry tuple with a real object):
> 
> * decode: The stateless decoding function
> * encode: The stateless encocing function
> * chunkdecoder: The stateful chunk decoder
> * chunkencoder: The stateful chunk encoder
> * streamreader: The stateful stream decoder
> * streamwriter: The stateful stream encoder
> 
> The functions and classes look like this:
> 
> 
> Stateless decoder:
> decode(input, errors='strict'):
>     Function that decodes the (str) input object and returns
>     a (unicode) output object. The decoder must decode the
>     complete input without any remaining undecoded bytes.
> 
> Stateless encoder:
> encode(input, errors='strict'):
>     Function that encodes the complete (unicode) input object and
>     returns a (str) output object.
> 
> Stateful chunk decoder:
> chunkdecoder(errors='strict'):
>     A factory function that returns a stateful decoder with the
>     following method:
> 
>     decode(input, final=False):
>         Decodes a chunk of input and return the decoded unicode
>         object. This method can be called multiple times and
>         the state of the decoder will be kept between calls.
>         This includes trailing incomplete byte sequences
>         that will be retained until the next call to decode().
>         When the argument final is true, this is the last call
>         to decode() and trailing incomplete byte sequences will
>         not be retained, but a UnicodeError will be raised.
> 
> Stateful chunk encoder:
> chunkencoder(errors='strict'):
>     A factory function that returns a stateful encoder
>     with the following method:
>     encode(input, final=False):
>         Encodes a chunk of input and returns the encoded
>         str object. When final is true this is the last
>         call to encode().
> 
> Stateful stream decoder:
> streamreader(stream, errors='strict'):
>     A factory function that returns a stateful decoder
>     for reading from the byte stream stream, with the
>     following methods:
> 
>     read(size=-1, chars=-1, final=False):
>         Read unicode characters from the stream. When data
>         is read from the stream it should be done in chunks of
>         size bytes. If size == -1 all the remaining data
>         from the stream is read. chars specifies the number
>         of characters to read from the stream. read() may return
>         less then chars characters if there's not enough data
>         available in the byte stream. If chars == -1 as much
>         characters are read as are available in the stream.
>         Transient errors are ignored and trailing incomplete
>         byte sequence are retained when final is false. Otherwise
>         a UnicodeError is raised in the case of incomplete byte
>         sequences.
>     readline(size=-1):
>             ...
>     next():
>             ...
>     __iter__():
>             ...
> 
> Stateful stream encoder:
> streamwriter(stream, errors='strict'):
>     A factory function that returns a stateful encoder
>     for writing unicode data to the byte stream stream,
>     with the following methods:
> 
>     write(data, final=False):
>         Encodes the unicode object data and writes it
>         to the stream. If final is true this is the last
>         call to write().
>     writelines(data):
>         ...
> 
> 
> I know that this is quite a departure from the current API, and
> I'm not sure if we can get all of the functionality without
> sacrificing backwards compatibility.
> 
> I don't particularly care about the "bytes consumed" return value
> from the stateless codec. The codec should always have returned only
> the encoded/decoded object, but I guess fixing this would break too
> much code. And users who are only interested in the stateless
> functionality will probably use unicode.encode/str.decode anyway.
> 
> For the stateful API it would be possible to combine the chunk and
> stream decoder/encode into one class with the following methods
> (for the decoder):
> 
>     __init__(stream, errors='strict'):
>         Like the current StreamReader constructor, but stream may be
>         None, if only the chunk API is used.
>     decode(input, final=False):
>         Like the current StreamReader (i.e. it returns a (unicode, int)
>         tuple.) This does not keep the remaining bytes in a buffer.
>         This is the job of the caller.
>     feed(input, final=False):
>         Decodes input and returns a decoded unicode object. This method
>         calls decode() internally and manages the byte buffer.
>     read(size=-1, chars=-1, final=False):
>     readline(size=-1):
>     next():
>     __iter__():
>         See above.
> 
> As before implementers of decoders only need to implement decode().
> 
> To be able to support the final argument the decoding functions
> in _codecsmodule.c could get an additional argument. With this
> they could be used for the stateless codecs too and we can reduce
> the number of functions again.
> 
> Unfortunately adding the final argument breaks all of the current
> codecs, but dropping the final argument requires one of two
> changes:
> 1) When the input stream is exhausted, the bytes read are parsed
>    as if final=True. That's the way the CJK codecs currently
>    handle it, but unfortunately this doesn't work with the feed
>    decoder.
> 2) Simply ignore any remaing undecoded bytes at the end of the
>    stream.
> 
> If we really have to drop the final argument, I'd prefer 2).
> 
> I've uploaded a second version of the patch. It implements
> the final argument, adds the feed() method to StreamReader and
> again merges the duplicate decoding functions in the codecs
> module. Note that the patch isn't really finished (the final
> argument isn't completely supported in the encoders and the
> CJK and escape codecs are unchanged), but it should be sufficient
> as a base for discussion.
> 
> Bye,
>    Walter D?rwald
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/mal%40egenix.com

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 12 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From amk at amk.ca  Thu Aug 12 15:27:06 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Thu Aug 12 15:27:39 2004
Subject: [Python-Dev] Third Bug Day outcome
In-Reply-To: <A128D751272CD411BC9200508BC2194D053C7790@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D053C7790@escpl.tcp.com.ar>
Message-ID: <20040812132706.GF21744@rogue.amk.ca>

On Mon, Aug 09, 2004 at 07:19:04PM -0300, Batista, Facundo wrote:
> Maybe for the next Bug Day should be two "IRC hosts", one in America and
> other in Europe. so, the'll be 6 hs each (top) and the whole day will be of
> 12 hours.

That would be a good idea, if a volunteer in Europe can be found.

--amk

From amk at amk.ca  Thu Aug 12 15:36:55 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Thu Aug 12 15:37:28 2004
Subject: [Python-Dev] Asyncore .set_reuse_addr() on Windows
In-Reply-To: <20040810132612.GB28542@unpythonic.net>
References: <20040807142327.GA2529@rogue.amk.ca>
	<20040810132612.GB28542@unpythonic.net>
Message-ID: <20040812133655.GG21744@rogue.amk.ca>

On Tue, Aug 10, 2004 at 08:26:13AM -0500, Jeff Epler wrote:
> SO_EXCLUSIVEADDRUSE sounds like the exact opposite of SO_REUSEADDR,
> according to a page I found when trying to find out whether any other
> platform has SO_EXCLUSEIVEADDRUSE:
>     http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/using_so_exclusiveaddruse.asp

Hm; this change came from patch #982681; I'm not a Windows person
either, and just assumed the patch was correct for Winsock's
semantics.  Unless someone with Windows experience can suggest
otherwise, I'll back out this change.

--amk

From jimjjewett at yahoo.com  Thu Aug 12 15:39:52 2004
From: jimjjewett at yahoo.com (Jim J Jewett)
Date: Thu Aug 12 15:39:56 2004
Subject: [Python-Dev] Re: Re: Decimal type question [Prothon]
Message-ID: <20040812133952.69390.qmail@web50702.mail.yahoo.com>

Mark Hahn:

> What do you think of the newer idea of using
> the IBM floating decimal for all numerics?  
> Tim Peters and others pushed this on me.  

In theory, it is a great idea, and removes
a lot of useless clutter.

C often has signed and unsigned versions of 
char, short, int, long, and long long.  

Python has just int and long.  The "missing" 
types aren't really missed.

For Python itself, the unification probably
shouldn't happen until 3.0, because of all
the programs that use an "integer" as a bitfield,
and because performance won't really be equal.

For a new language -- yes, you might as well
get it right from the start.

-jJ



		
__________________________________
Do you Yahoo!?
Yahoo! Mail is new and improved - Check it out!
http://promotions.yahoo.com/new_mail
From guido at python.org  Thu Aug 12 16:20:39 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 16:20:43 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: Your message of "Thu, 12 Aug 2004 18:11:53 +1200."
	<200408120611.i7C6Brqt012067@cosc353.cosc.canterbury.ac.nz> 
References: <200408120611.i7C6Brqt012067@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408121420.i7CEKdo30140@guido.python.org>

> Guido:
> > Anyway, if we really do have enough use cases for byte array
> > literals, we might add them.  I still think that would be confusing
> > though, because byte arrays are most useful if they are mutable: and
> > then we'd have mutable literals -- blechhhh!

Greg:
> Perhaps the constructor for a byte array could accept a string
> argument as long as it contained only ascii characters?
> 
>    h.putrequest(bytes('GET'), selector)

Yeah, but that's what Martin called depending on the default encoding.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug 12 16:30:06 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 16:30:12 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: Your message of "Thu, 12 Aug 2004 08:26:40 +0200."
	<411B0DA0.4030109@v.loewis.de> 
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org> 
	<411B0DA0.4030109@v.loewis.de> 
Message-ID: <200408121430.i7CEU6F30168@guido.python.org>

> Guido van Rossum wrote:
> > Anyway, if we really do have enough use cases for byte array
> > literals, we might add them.  I still think that would be
> > confusing though, because byte arrays are most useful if they are
> > mutable: and then we'd have mutable literals -- blechhhh!

Martin:
> I see. How would you like byte array displays then?

Not as characters (not by default anyway), because more often than not
they will contain binary or encoded gibberish!

> This is the approach taken in the other languages: Everytime the
> array display is executed, a new array is created. There is then no
> problem with that being mutable.

The downside of that is that then for performance reasons you might
end up having to move bytes literals out of expressions if they are in
fact used read-only (which the compiler can't know but the user can).

> Of course, if the syntax is too similar to string literals, people
> might be tricked into believing they are actually literals. Perhaps
> 
>    bytes('G','E','T')
> 
> would be sufficient, or even
> 
>    bytes("GET")
> 
> which would implicitly convert each character to Latin-1.

The first form would also need an encoding, since Python doesn't have
character literals!  I don't think we should use Latin-1, for the same
reasons that the default encoding is ASCII.  Better would be to have a
second argument that's the encoding, so you can write

    bytes(u"<chinese text>", "utf-8")

Hm, u"<chinese text>".encode("utf-8") should probably return a bytes
array, and that might be sufficient.  Perhaps bytes should by default
be considered as arrays of tiny unsigned ints, so we could use

    bytes(map(ord, "GET"))

and it would display itself as

    bytes([71, 69, 84])

Very long ones should probably use ellipses rather than print a
million numbers.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug 12 16:44:05 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 16:44:09 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: Your message of "Thu, 12 Aug 2004 13:06:58 BST."
	<200408121306.58446.gmccaughan@synaptics-uk.com> 
References: <1092258146.411a896246d1a@mcherm.com>  
	<200408121306.58446.gmccaughan@synaptics-uk.com> 
Message-ID: <200408121444.i7CEi5c30221@guido.python.org>

> This is why "integer" is a better name than "baseinteger".
> For now it can be the common supertype of int and long.
> In the future, it can be the name of the single integer
> type.

No, that will be int, of course!

Like 'basestring', 'baseinteger' is intentionally cumbersome, because
it is only the base class of all *built-in* integral types.  Note that
UserString is *not* subclassing basestring, and likewise if you wrote
an integer-like class from scratch it should not inherit from
baseinteger.  Code testing for these types is interested in knowing
whether something is a member of one of the *built-in* types, which is
often needed because other built-in operations (e.g. many extension
modules) only handle the built-in types.

If you want to test for integer-like or string-like behavior, you
won't be able to use isinstance(), but instead you'll have to check
for the presence of certain methods.

I know this is not easy in the case of integers, but I don't want to
start requiring inheritance from a marker base type now.  Python is
built on duck typing.  (Google for it.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From barry at python.org  Thu Aug 12 16:54:13 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Aug 12 16:53:57 2004
Subject: [Python-Dev] Naming nit
In-Reply-To: <411A79AA.4020505@ocf.berkeley.edu>
References: <003401c47fdc$a0f722a0$e622c797@oemcomputer>
	<411A79AA.4020505@ocf.berkeley.edu>
Message-ID: <1092322453.8672.67.camel@localhost>

On Wed, 2004-08-11 at 15:55, Brett C. wrote:
> Raymond Hettinger wrote:
> 
> > For PEP 292, should the class be Template() or template()?
> >  
> 
> Template as per PEP 8.

Ok, I'll change that when I make my next round of mods to PEP 292.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040812/67d0d9fc/attachment.pgp
From pje at telecommunity.com  Thu Aug 12 17:00:15 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug 12 17:00:22 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <200408120545.i7C5j4W29250@guido.python.org>
References: <Your message of "Thu,
	12 Aug 2004 07:10:44 +0200." <411AFBD4.5080903@v.loewis.de>
	<1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
Message-ID: <5.1.1.6.0.20040812105836.0321fec0@mail.telecommunity.com>

At 10:45 PM 8/11/04 -0700, Guido van Rossum wrote:

>Anyway, if we really do have enough use cases for byte array literals,
>we might add them.  I still think that would be confusing though,
>because byte arrays are most useful if they are mutable: and then we'd
>have mutable literals -- blechhhh!

Not if they work like list or dictionary "literals".  That is, if they're 
just short for 'array("B","literal here")'.

From jack at performancedrivers.com  Thu Aug 12 17:03:54 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Thu Aug 12 17:03:58 2004
Subject: [Python-Dev] @decorator now w/ class support [was: pep 318,
	Decorators for Functions, Methods and Classes]
In-Reply-To: <2macx0zllk.fsf@starship.python.net>
References: <20040811011116.GF23725@performancedrivers.com>
	<A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org>
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
	<200408072105.i77L5Ea10965@guido.python.org>
	<20040811011116.GF23725@performancedrivers.com>
	<5.1.1.6.0.20040811155219.02a56a00@mail.telecommunity.com>
	<2macx0zllk.fsf@starship.python.net>
Message-ID: <20040812150354.GN23725@performancedrivers.com>

On Thu, Aug 12, 2004 at 11:39:35AM +0100, Michael Hudson wrote:
> "Phillip J. Eby" <pje@telecommunity.com> writes:
> 
> > At 03:44 PM 8/11/04 -0400, Jack Diederich wrote:
> >> I have a mostly working patch for class decorators but it gets an
> >> XXX ambiguity
> >>between
> >>
> >>funcdef: [decorators] 'def' NAME parameters ':' suite
> >>and
> >>classdef: [decorators] 'class' NAME [ '(' testlist ')' ] ':' suite
> >>
> >> It works as advertised with the pre-decorators funcdef definition
> >> (which removes the ambiguity).
> >>Help?
> >
> >
> > How about this?
> >
> >      decorated_assignment: [decorators] (funcdef | classdef)
> 
> decorated_definition or something would be better surely?

I ended up going with 'decorated_thing'
http://python.org/sf/1007991

decorator code moved out of the function code and will decorate
both functions and classes.

-Jack
From tim.peters at gmail.com  Thu Aug 12 17:23:23 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 12 17:23:27 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <2mekmczlv7.fsf@starship.python.net>
References: <000001c47e88$81ef1d40$5635c797@oemcomputer>
	<1f7befae0408092348583f0953@mail.gmail.com>
	<2mvffqzum9.fsf@starship.python.net>
	<1f7befae0408101258431039a1@mail.gmail.com>
	<1f7befae04081123253f08bb71@mail.gmail.com>
	<2mekmczlv7.fsf@starship.python.net>
Message-ID: <1f7befae04081208237cd173ef@mail.gmail.com>

[Michael Hudson <mwh@python.net]
> Has the failure mode of alloca() changed?

No, and the Windows stack-check code works fine.  I showed results
before from boosting the amount of padding the Windows stack-check
code checks for, and it if checks for 20x more  (which is ridiculously
large) padding than it checks for now, it reliably generates Python
stack-overflow MemoryErrors.

Indeed, the KeyError exceptions were traced specifically to this:  a
stack-overflow MemoryError, due to the Windows stack-check code,
getting wiped out by lookdict (whose caller took lookdict's NULL
return as meaning the key wasn't present in the dict -- although it
actually was).

> I take it you're building with VC++ 7.1?

Right.

> What happens for a VC++ 6 build?

Raymond reported on that earlier.  Appeared to be the same as I saw in
a release build.  He didn't report on a debug build.  He's running
WinME, so a "hard" failure may look quite different for him.

> Hmm, a moment with msdn suggests that there's been no significant
> changes here, although the documentation is for _alloca(), and Python
> calls alloca().  That can't make any difference, can it?

Right, no difference.

> It still smells like a tool change to me.

Not to me.  Although it does smell.
From skip at pobox.com  Thu Aug 12 16:29:29 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 12 17:33:16 2004
Subject: [Python-Dev] adding a bytes sequence type to Python
In-Reply-To: <411B0DA0.4030109@v.loewis.de>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org>
	<411B0DA0.4030109@v.loewis.de>
Message-ID: <16667.32457.28553.468008@montanaro.dyndns.org>


    Martin>    bytes("GET")

    Martin> which would implicitly convert each character to Latin-1.

That's why I think a special literal is necessary.  There'd be no unicode
foolishness involved. ;-)  They'd just be raw uninterpreted bytes.

Skip
From skip at pobox.com  Thu Aug 12 16:27:34 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 12 17:33:20 2004
Subject: [Python-Dev] adding a bytes sequence type to Python
In-Reply-To: <200408120545.i7C5j4W29250@guido.python.org>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org>
Message-ID: <16667.32342.640469.913208@montanaro.dyndns.org>


    Guido> Anyway, if we really do have enough use cases for byte array
    Guido> literals, we might add them.  I still think that would be
    Guido> confusing though, because byte arrays are most useful if they are
    Guido> mutable: and then we'd have mutable literals -- blechhhh!

Today I can initialize mutable objects from immutable strings:

    >>> print list("abc")
    ['a', 'b', 'c']
    >>> print set("abc")
    set(['a', 'c', 'b'])

I see no reason that mutable bytes objects couldn't be created from
otherwise immutable sequences either.  Would it be a problem to ensure that

    a = b"abc"
    b = b"abc"
    print a is b

prints False?

The main difference as I see it is that byte literals would be completely
devoid of any sort of interpretation as unicode sequences.  It would be nice
if this was possible:

    # -*- coding: utf-8 -*-
    b = b"?"

though that would probably wreak havoc with editors and hex escapes would
have to be used in most situations.

Skip
From skip at pobox.com  Thu Aug 12 16:09:05 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 12 17:33:27 2004
Subject: [Python-Dev] adding a bytes sequence type to Python
In-Reply-To: <200408120414.i7C4Er429003@guido.python.org>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
Message-ID: <16667.31233.264061.518559@montanaro.dyndns.org>


(changing the subject - sorry i didn't do it in my first two messages...)

    >> 1. Make bytes a synonuym for str.

    Guido> Hmm...  I worry that a simple alias would just encourage confused
    Guido> usage, since the compiler won't check.  I'd rather see bytes an
    Guido> alias for a bytes array as defined by the array module.

You're right.  This could probably be added now ("now" being 2.5) with
little or no problem.  My thought was to get the name in there quickly
(could be done in 2.4) with some supporting documentation so people could
begin modifying their code.

    >> 2. Warn about the use of bytes as a variable name.

    Guido> Is this really needed?  Builtins don't byte variable names.

I suppose this could be dispensed with.  Let pychecker handle it.

    >> 3. Introduce b"..." literals as a synonym for current string
    >> literals, and have them *not* generate warnings if non-ascii
    >> characters were used in them without a coding cookie.

    Guido> I expecet all sorts of problems with that, such as what it would
    Guido> mean if Unicode or multibyte characters are used in the source.

My intent in proposing b"..." literals was that they would be allowed in any
source file.  Their contents would not be interpreted in any way.  One
simple use case: identifying the magic number of a binary file type of some
sort.  That might well be a constant to programmers manipulating that sort
of file and have nothing to do with Unicode at all.

    Guido> Do we really need byte array literals at all?  

I think so.  Martin already pointed out an example where a string literal is
used today for a sequences of bytes that's put out on the wire as-is.  It's
just convenient that the protocol was developed in such a way that most of
its meta-data is plain ASCII.

Skip
From skip at pobox.com  Thu Aug 12 16:12:27 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 12 17:33:31 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <411AFBD4.5080903@v.loewis.de>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
Message-ID: <16667.31435.566032.294560@montanaro.dyndns.org>


    Martin> Java also supports byte arrays in the source, although they are
    Martin> difficult to type:

    Martin>     byte[] request = {'G', 'E', 'T'};

Seems to me that b"GET" would be more Pythonic given existing Python string
literals.

    Martin> As for reading from streams: Java has multiple reader APIs; some
    Martin> return byte strings, some character strings.

I think Guido's proposed 'B' might make sense here.  OTOH, today's 'b' might
work as well, though a switch of that magnitude could probably not be made
until 3.0 if bytes objects are not synonyms for strings.

Skip
From python at rcn.com  Thu Aug 12 17:46:58 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Aug 12 17:47:16 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae04081208237cd173ef@mail.gmail.com>
Message-ID: <002d01c48083$9a89a6c0$5229c797@oemcomputer>

> > What happens for a VC++ 6 build?
> 
> Raymond reported on that earlier.  Appeared to be the same as I saw in
> a release build.  He didn't report on a debug build.  He's running
> WinME, so a "hard" failure may look quite different for him.

On a debug build with VC++ 6 on WinMe, the script faults immediately and
the program terminates with no useful messages of any sort.



Raymond

From guido at python.org  Thu Aug 12 18:10:48 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 18:10:55 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: Your message of "Thu, 12 Aug 2004 09:09:05 CDT."
	<16667.31233.264061.518559@montanaro.dyndns.org> 
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org> 
	<16667.31233.264061.518559@montanaro.dyndns.org> 
Message-ID: <200408121610.i7CGAmX30436@guido.python.org>

>     >> 1. Make bytes a synonuym for str.
> 
>     Guido> Hmm...  I worry that a simple alias would just encourage confused
>     Guido> usage, since the compiler won't check.  I'd rather see bytes an
>     Guido> alias for a bytes array as defined by the array module.
> 
> You're right.  This could probably be added now ("now" being 2.5) with
> little or no problem.  My thought was to get the name in there quickly
> (could be done in 2.4) with some supporting documentation so people could
> begin modifying their code.

I think very few people would do so until the semantics of bytes were
clearer.  Let's just put it in meaning byte array when we're ready.

>     >> 2. Warn about the use of bytes as a variable name.
> 
>     Guido> Is this really needed?  Builtins don't byte variable names.
> 
> I suppose this could be dispensed with.  Let pychecker handle it.
> 
>     >> 3. Introduce b"..." literals as a synonym for current string
>     >> literals, and have them *not* generate warnings if non-ascii
>     >> characters were used in them without a coding cookie.
> 
>     Guido> I expecet all sorts of problems with that, such as what it would
>     Guido> mean if Unicode or multibyte characters are used in the source.
> 
> My intent in proposing b"..." literals was that they would be
> allowed in any source file.  Their contents would not be interpreted
> in any way.

But they would be manipulated if they were non-ASCII and the source
file was converted to a different encoding.  Better be safe and only
allow printable ASCII and hex escapes there.

> One simple use case: identifying the magic number of a
> binary file type of some sort.  That might well be a constant to
> programmers manipulating that sort of file and have nothing to do
> with Unicode at all.

Not a very strong use case, this could easily be done using just hex.

>     Guido> Do we really need byte array literals at all?  
> 
> I think so.  Martin already pointed out an example where a string
> literal is used today for a sequences of bytes that's put out on the
> wire as-is.  It's just convenient that the protocol was developed in
> such a way that most of its meta-data is plain ASCII.

See my response to that.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Thu Aug 12 18:17:18 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 12 18:17:21 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
Message-ID: <1f7befae040812091754035bcb@mail.gmail.com>

More clues.

As expected, commenting out the tests in test_parser.py that contain
deeply nested tuples makes the problems go away.

A surprise:  adding code to every function in parsermodule.c to do
Py_FatalError() if the Windows stack-check fails doesn't make any
difference.  So it's not C recursion in parsermodule.c that causes
this (although I bet it *could*, it simply doesn't in this test case).

Not a surprise:   Grzegorz Makarewicz's patch to call the stack-check
code in pymalloc catches the problem "in time".  But we could put the
same patch anywhere in Python that gets called frequently, and catch
it in time.  That's too expensive to bear.

The Windows stack-check code just isn't called often enough, and the
recursion limit is "too large" now to live without a reliable stack
check.  Py_EnterRecursiveCall() in fact rarely calls the Windows
stack-check code.  At the time the Windows stack-check code first
*would* have complained (had it been called then), the tstate
recursion depth was 694, and _Py_CheckRecursionLimit was 973.

The funky way this code works, that means we can do ~150 more levels
of Python call before the Windows stack-check code is called again,
and that's too late.  We only checked that there's still room for 2K
pointers on the stack, and 150 Python-level calls means about 450
C-level calls in this test:  the stack at this point is hundreds (&
hundreds) of repetitions of

PyEval_EvalFrame ->
fast_function ->
call_function ->
PyEval_Frame -> etc etc etc

and PyEval_EvalFrame C frames are fat enough on their own for 150 of
them to blow what remains of the stack.

The easiest workaround for Windows is still to boost the stack size. 
I'm going to do that if nobody screams.

Looks like nobody has an explanation yet for why 2.3.4 consistently
yielded MemoryError but 2.4a2 mixes those with spurious KeyError and
SyntaxError exceptions.  That could be coincidence in this specific
test driver, though -- lookdict has always wiped out exceptions.

where's-stackless-when-you-need-it-ly y'rs  - tim
From janssen at parc.com  Thu Aug 12 18:23:10 2004
From: janssen at parc.com (Bill Janssen)
Date: Thu Aug 12 18:23:31 2004
Subject: [Python-Dev] CGI module [was: New Subscriber] 
In-Reply-To: Your message of "Thu, 12 Aug 2004 00:15:50 PDT."
	<70C26872-EC2F-11D8-B23F-000A957D4F2A@mac.com> 
Message-ID: <04Aug12.092313pdt."58612"@synergy1.parc.xerox.com>

Steve Majewski writes:
> As far as I can tell, all of those threads died out without any actual
> code being produced. However, they do give a good enumeration of what's
> wrong with the current cgi module and what's needed in a new module.
> 
> I guess  the Web-SIG is the best place for any further discussion.
> http://mail.python.org/mailman/listinfo/web-sig

As Steve notes, we've discussed this subject quite a bit.  The next
step is to form some more structured proposals, in the form of PEP
submissions.  These can then properly be considered on c.l.p. and
python-dev.  If folks want to contribute, they can review the past
discussion, pick an area to write a PEP on (it would be polite to
announce that on Web-SIG), write a proposal, and continue on with
the process.

This is starting to happen.  Those on Web-SIG will see that Phillip
Eby has submitted a pre-PEP on a Web Server Gateway Interface (WSGI),
which is being discussed right now.
[http://mail.python.org/pipermail/web-sig/2004-August/000518.html]

Remember that having working code sometimes helps people evaluate a
PEP proposal :-).

Bill
From gmccaughan at synaptics-uk.com  Thu Aug 12 18:23:33 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Thu Aug 12 18:24:05 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <200408121444.i7CEi5c30221@guido.python.org>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408121306.58446.gmccaughan@synaptics-uk.com>
	<200408121444.i7CEi5c30221@guido.python.org>
Message-ID: <200408121723.33597.gmccaughan@synaptics-uk.com>

[I said:]
>> This is why "integer" is a better name than "baseinteger".
>> For now it can be the common supertype of int and long.
>> In the future, it can be the name of the single integer
>> type.

[Guido:]
> No, that will be int, of course!

Dang. I suppose it has to be, for hysterical raisins.

> Like 'basestring', 'baseinteger' is intentionally cumbersome, because
> it is only the base class of all *built-in* integral types.  Note that
> UserString is *not* subclassing basestring, and likewise if you wrote
> an integer-like class from scratch it should not inherit from
> baseinteger.  Code testing for these types is interested in knowing
> whether something is a member of one of the *built-in* types, which is
> often needed because other built-in operations (e.g. many extension
> modules) only handle the built-in types.
> 
> If you want to test for integer-like or string-like behavior, you
> won't be able to use isinstance(), but instead you'll have to check
> for the presence of certain methods.
> 
> I know this is not easy in the case of integers, but I don't want to
> start requiring inheritance from a marker base type now.  Python is
> built on duck typing.  (Google for it.)

I don't think this is a good reason for rejecting "integer"
as the name of the common supertype of int and long. You'd use
isinstance(x,integer) to check whether x is an integer of
built-in type, just as you currently use isinstance(x,float)
to check whether x is a floating-point number of built-in
type. I see no reason why a cumbersome name is much advantage.

I am, however, convinced by the other argument: All the steps
towards int/long unification that have been taken so far assume
that the endpoint is having a single type called "int", and
that would be derailed by changing the target name to "integer".
Not to mention that there's any amount of code out there that
uses "int" for conversions and the like, which there's no reason
to break.

I can't get away from the feeling that it would, in some possibly
over-abstract sense, have been neater to introduce "integer" as
the supertype and encourage people to *change* from using "int"
(and "long") to using "integer", so that the meanings of "int" and
"long" never change in the unification process. But, even if I
could convince you, it's too late for that :-).

By the way, I wasn't at any point proposing that inheritance
from a "marker" type should be required for anything, and I'm
not sure why you thought I were. I expect I was unclear, and
I'm sorry about that.

-- 
g

From janssen at parc.com  Thu Aug 12 18:25:49 2004
From: janssen at parc.com (Bill Janssen)
Date: Thu Aug 12 18:26:32 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint 
In-Reply-To: Your message of "Thu, 12 Aug 2004 07:30:06 PDT."
	<200408121430.i7CEU6F30168@guido.python.org> 
Message-ID: <04Aug12.092558pdt."58612"@synergy1.parc.xerox.com>

> Hm, u"<chinese text>".encode("utf-8") should probably return a bytes
> array, and that might be sufficient.

+1.

Bill
From python at discworld.dyndns.org  Thu Aug 12 18:38:38 2004
From: python at discworld.dyndns.org (Charles Cazabon)
Date: Thu Aug 12 18:31:11 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae040812091754035bcb@mail.gmail.com>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
Message-ID: <20040812163838.GA9343@discworld.dyndns.org>

Tim Peters <tim.peters@gmail.com> wrote:
> 
> The Windows stack-check code just isn't called often enough, and the
> recursion limit is "too large" now to live without a reliable stack
> check.
[...]
> The easiest workaround for Windows is still to boost the stack size. 
> I'm going to do that if nobody screams.

Would reducing the recursion limit (for Windows) be a reasonable approach?

Charles

P.S. Please don't cc: me on list messages.
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python@discworld.dyndns.org>
GPL'ed software available at:     http://www.qcc.ca/~charlesc/software/
-----------------------------------------------------------------------
From l at lrowe.co.uk  Thu Aug 12 18:31:10 2004
From: l at lrowe.co.uk (Laurence Rowe)
Date: Thu Aug 12 18:40:32 2004
Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered
In-Reply-To: <1092179578.4119567af011d@webmail.oregonstate.edu>
References: <1092179578.4119567af011d@webmail.oregonstate.edu>
Message-ID: <cfg60f$978$1@sea.gmane.org>

I must say I like this idea a lot. It seems much more in keeping with 
the rest of the python block structure. You could even think of 
decorators as bullet points. Now if only someone hadn't already found a 
use for '*' character...

For those of us who like code-folding editors, this option also allows 
them to continue folding the whole code block down to one line.

- Laurence

> -----------------------------------------------
> Option B
> -----------------------------------------------
> 
> def f(a, b):
>     @staticmethod
>     @synchronized(lockvar)
>     """
>     Hi,
> 
>     Performs a binary operation.
> 
>     Docstring.
>     """
>     return a + b

>  - Connelly

From mwh at python.net  Thu Aug 12 19:36:30 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug 12 19:36:32 2004
Subject: [Python-Dev] Third Bug Day outcome
In-Reply-To: <20040812132706.GF21744@rogue.amk.ca> (A. M. Kuchling's message
	of "Thu, 12 Aug 2004 09:27:06 -0400")
References: <A128D751272CD411BC9200508BC2194D053C7790@escpl.tcp.com.ar>
	<20040812132706.GF21744@rogue.amk.ca>
Message-ID: <2mllgkxnq9.fsf@starship.python.net>

"A.M. Kuchling" <amk@amk.ca> writes:

> On Mon, Aug 09, 2004 at 07:19:04PM -0300, Batista, Facundo wrote:
>> Maybe for the next Bug Day should be two "IRC hosts", one in America and
>> other in Europe. so, the'll be 6 hs each (top) and the whole day will be of
>> 12 hours.
>
> That would be a good idea, if a volunteer in Europe can be found.

I'm game, but have quite a tendency to be away at weekends, so it'll
depend if I'm around for any given bug day.

Cheers,
mwh

-- 
  MARVIN:  Oh dear, I think you'll find reality's on the blink again.
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 12
From mwh at python.net  Thu Aug 12 19:40:03 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug 12 19:40:10 2004
Subject: [Python-Dev] adding a bytes sequence type to Python
In-Reply-To: <16667.31233.264061.518559@montanaro.dyndns.org> (Skip
	Montanaro's message of "Thu, 12 Aug 2004 09:09:05 -0500")
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<16667.31233.264061.518559@montanaro.dyndns.org>
Message-ID: <2mekmcxnkc.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

> My intent in proposing b"..." literals was that they would be allowed in any
> source file.  Their contents would not be interpreted in any way.

I think this is a bad idea.  If a coding cookie says a file is in
utf-8, then the file really has to be valid utf-8 data, for
implementation sanity and not freaking out editors.  The use-case for
including arbitrary chunks of binary data in a source file seems
stretched in the extreme.

Cheers,
mwh

-- 
  My hat is lined with tinfoil for protection in the unlikely event
  that the droid gets his PowerPoint presentation working.
                               -- Alan W. Frame, alt.sysadmin.recovery
From tim.peters at gmail.com  Thu Aug 12 19:54:01 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 12 19:54:08 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <20040812163838.GA9343@discworld.dyndns.org>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
	<20040812163838.GA9343@discworld.dyndns.org>
Message-ID: <1f7befae04081210541b570143@mail.gmail.com>

[Charles Cazabon]
> Would reducing the recursion limit (for Windows) be a reasonable
> approach?

Not if that means Windows can't run the standard test suite.

Boosting the stack size on Windows is a VM reservation operation, BTW
-- it doesn't actually increase the RAM needed, unless the stack
actually needs to grow that big.
From s.percivall at chello.se  Thu Aug 12 19:56:14 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Thu Aug 12 19:56:18 2004
Subject: [Python-Dev] @decorator now w/ class support [was: pep 318,
	Decorators for Functions, Methods and Classes]
In-Reply-To: <20040812150354.GN23725@performancedrivers.com>
References: <20040811011116.GF23725@performancedrivers.com>
	<A56789C2-E7BE-11D8-8E6E-0003934AD54A@chello.se>
	<200408061555.i76FtV707448@guido.python.org>
	<4113BACA.6040801@ieee.org>
	<F563499F-E89B-11D8-8B30-0003934AD54A@chello.se>
	<200408072105.i77L5Ea10965@guido.python.org>
	<20040811011116.GF23725@performancedrivers.com>
	<5.1.1.6.0.20040811155219.02a56a00@mail.telecommunity.com>
	<2macx0zllk.fsf@starship.python.net>
	<20040812150354.GN23725@performancedrivers.com>
Message-ID: <E7682895-EC88-11D8-A8D7-0003934AD54A@chello.se>

On 2004-08-12, at 17.03, Jack Diederich wrote:
> I ended up going with 'decorated_thing'
> http://python.org/sf/1007991
>
> decorator code moved out of the function code and will decorate
> both functions and classes.

That's wonderful! +1 for inclusion from me, when the patch is final, 
etc.

//Simon

From dave at boost-consulting.com  Thu Aug 12 20:05:04 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Aug 12 20:05:47 2004
Subject: [Python-Dev] Re: Another approach to decorators.
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<un011l2qz.fsf@boost-consulting.com>
	<eppstein-8D4A8F.19021411082004@sea.gmane.org>
Message-ID: <u8yckfd0v.fsf@boost-consulting.com>

David Eppstein <eppstein@ics.uci.edu> writes:

> In article <un011l2qz.fsf@boost-consulting.com>,
>  David Abrahams <dave@boost-consulting.com> wrote:
>
>> > class Foo:
>> >
>> >   decorate static:
>> >
>> >     def static1(blah...):
>> >         pass
>> >
>> >     decorate locksFile:
>> >
>> >       def static2andLocks(blah...):  # both decorators appy
>> >           pass
>> 
>> Wow, Martin Z's idea (using your keyword) really went "thunk" for me.
>> What decorate does would be very much like what "class" does in some
>> ways.
>
> class: (and other something: constructs) start a block that can contain 
> any code.  Does this decorate keyword allow e.g. assignments as well as 
> defs and other decorates?  Or loops?  

Why not?  class: starts a block that establishes a new dict as the
execution namespace and then passes the result off to the metaclass.
decorate might do the same.

> If so what should it mean?  

Whatever the user wants, I guess?

> Is it like that locals() gets replaced by a special
> dictionary-like-object that calls the decorator whenever any of its
> contents gets set?

I was thinking it calls the decorator afterwards, just like class
calls the metaclass afterwards.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From dave at boost-consulting.com  Thu Aug 12 20:07:45 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Aug 12 20:10:20 2004
Subject: [Python-Dev] Re: Another approach to decorators.
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<un011l2qz.fsf@boost-consulting.com>
	<eppstein-8D4A8F.19021411082004@sea.gmane.org>
	<411ADCFA.8050305@divmod.com>
Message-ID: <u3c2sfcwe.fsf@boost-consulting.com>

Jp Calderone <exarkun@divmod.com> writes:

> David Eppstein wrote:
>> In article <un011l2qz.fsf@boost-consulting.com>,
>>  David Abrahams <dave@boost-consulting.com> wrote:
>> 
>>>>class Foo:
>>>>
>>>>  decorate static:
>>>>
>>>>    def static1(blah...):
>>>>        pass
>>>>
>>>>    decorate locksFile:
>>>>
>>>>      def static2andLocks(blah...):  # both decorators appy
>>>>          pass
>>>
>>>Wow, Martin Z's idea (using your keyword) really went "thunk" for me.
>>>What decorate does would be very much like what "class" does in some
>>>ways.
>> class: (and other something: constructs) start a block that can
>> contain any code.  Does this decorate keyword allow e.g. assignments
>> as well as defs and other decorates?  Or loops?  If so what should
>> it mean?  Is it like that locals() gets replaced by a special
>> dictionary-like-object that calls the decorator whenever any of its
>> contents gets set?
>> 
>
>    (I have no idea what the original poster intended, however....)
>
>      'decorate' expression ':'
>          suite
>
>    could create a nested scope, the locals of which could be passed to
>    whatever "expression" evaluates to (or if it is a tuple, do the
>    looping thing, since people seem to like that).
>
>    The call can return a dict with which the class dict is updated.

That's almost exactly what I intended.


<snip>

>    This covers a use case James Knight mentioned as well.  As he
>    pointed out, '@' and other proposals only decorate functions (and
>    maybe classes, soon).  The example he gave, I believe, was:
>
>      @public
>      x = 1
>
>    This doesn't work, but:
>
>      public:
>          x = 1
>
>    would.  Now, swallow that scream of horror (I did, barely
>    ;). 

No screams; I love it.


-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From fredrik at pythonware.com  Thu Aug 12 20:18:27 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu Aug 12 20:16:48 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com><lkdwu0bcz7r.fsf@panix2.panix.com><87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp><lkdacx37oxr.fsf@panix2.panix.com><873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp>
	<lkdpt5x3rul.fsf@panix2.panix.com>
Message-ID: <cfgc6b$pt4$1@sea.gmane.org>

Mike Coleman wrote:

> Re maintenance, yeah regexp is pretty terse and ugly.  Generally, though, I'd
> rather deal with a reasonably well-considered 80 char regexp than 100 lines of
> code that does the same thing.

well, the examples in your PEP can be written as:

    data = [line[:-1].split(":") for line in open(filename)]

and

    import ConfigParser

    c = ConfigParser.ConfigParser()
    c.read(filename)

    data = []
    for section in c.sections():
        data.append((section, c.items(section)))

both of which are shorter than your structparse examples.

and most of the one-liners in your pre-PEP can be handled with a
combination of "match" and "finditer".  here's a 16-line helper that
parses strings matching the "a(b)*c" pattern into a prefix/list/tail tuple.

    import re

    def parse(string, pat1, pat2):
        """Parse a string having the form pat1(pat2)*"""
        m = re.match(pat1, string)
        i = m.end()
        a = m.group(1)
        b = []
        for m in re.compile(pat2 + "|.").finditer(string, i):
            try:
                token = m.group(m.lastindex)
            except IndexError:
                break
            b.append(token)
            i = m.end()
        return a, b, string[i:]

>>> parse("hello 1 2 3 4 # 5", "(\w+)", "\s*(\d+)")
('hello', ['1', '2', '3', '4'], ' # 5')

tweak as necessary.

</F> 



From dave at boost-consulting.com  Thu Aug 12 20:10:37 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Aug 12 20:20:21 2004
Subject: [Python-Dev] Re: Another approach to decorators.
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<411A898F.3060004@ocf.berkeley.edu>
	<20040811142354.661970e0.michel@dialnetwork.com>
	<20040811142802.0386bd60.michel@dialnetwork.com>
	<200408120426.i7C4QU829083@guido.python.org>
Message-ID: <uwu04dy76.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

> [Various proposals suggesting the use of a single decorator block to
> decorate multiple method definitions]
>
> I'm not keen on this.  I expect that beyond classmethod and
> staticmethod there aren't many use cases for repeating the same
> decorator exactly multiple times.  It also requires indenting the
> method, which I've already argued against.  And lastly it removes the
> decorator from the method definition again, thereby defeating the
> readability goals: when you look at a def in isolation, you may forget
> to look several pages up for a decorator.
>
> I think this is too far-fetched to consider as an alternative to the
> humble @decorator.


Maybe, but if you're still serious about improving support for
domain-specific embedded languages in Python you ought to give the
ideas another look.  If a reasonable syntax for decorators falls out
of a more general and useful mechanism, so much the better.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From martin at v.loewis.de  Thu Aug 12 20:21:00 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 12 20:20:56 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <16667.32457.28553.468008@montanaro.dyndns.org>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org>
	<411B0DA0.4030109@v.loewis.de>
	<16667.32457.28553.468008@montanaro.dyndns.org>
Message-ID: <411BB50C.8000804@v.loewis.de>

Skip Montanaro wrote:
> That's why I think a special literal is necessary.  There'd be no unicode
> foolishness involved. ;-)  They'd just be raw uninterpreted bytes.

But you'd spell them b"GET", no? If so, which numeric value has "G"?

Regards,
Martin
From arigo at tunes.org  Thu Aug 12 20:55:21 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu Aug 12 21:00:24 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae040812091754035bcb@mail.gmail.com>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
Message-ID: <20040812185521.GA2277@vicky.ecs.soton.ac.uk>

Hello Tim,

On Thu, Aug 12, 2004 at 12:17:18PM -0400, Tim Peters wrote:
> Looks like nobody has an explanation yet for why 2.3.4 consistently
> yielded MemoryError but 2.4a2 mixes those with spurious KeyError and
> SyntaxError exceptions.

The reason is that Py_EnterRecursiveCall() was only introduced in 2.4.  
Comparisons didn't throw RuntimeErrors that easily in 2.3.


Armin
From skip at pobox.com  Thu Aug 12 21:14:35 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 12 21:14:46 2004
Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered
In-Reply-To: <cfg60f$978$1@sea.gmane.org>
References: <1092179578.4119567af011d@webmail.oregonstate.edu>
	<cfg60f$978$1@sea.gmane.org>
Message-ID: <16667.49563.652742.680876@montanaro.dyndns.org>


    Laurence> I must say I like this idea a lot. It seems much more in
    Laurence> keeping with the rest of the python block structure. You could
    Laurence> even think of decorators as bullet points. Now if only someone
    Laurence> hadn't already found a use for '*' character...

I think '*' is syntactically available for this use.  It's not used as a
unary operator I don't believe.  The *args thing isn't available in the
context where decorators would be used.

That also has the nice side effect of turning @ back over to tools like leo
and ipython.  Unlike '|' it's not likely to be mistaken for other commonly
used characters.  Also, a unary '*' or '**' already have some special
meaning when used in and around function definitions, so seeing

    *deco
    def func(...):
        ...

or

    def func(...):
        *deco
        ...

might be a small hint to readers of the code that deco is somehow involved
in the function's definition.

Skip
From skip at pobox.com  Thu Aug 12 21:19:22 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 12 21:19:30 2004
Subject: [Python-Dev] adding a bytes sequence type to Python
In-Reply-To: <200408121430.i7CEU6F30168@guido.python.org>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org>
	<411B0DA0.4030109@v.loewis.de>
	<200408121430.i7CEU6F30168@guido.python.org>
Message-ID: <16667.49850.156147.876146@montanaro.dyndns.org>


    >> This is the approach taken in the other languages: Everytime the
    >> array display is executed, a new array is created. There is then no
    >> problem with that being mutable.

    Guido> The downside of that is that then for performance reasons you
    Guido> might end up having to move bytes literals out of expressions if
    Guido> they are in fact used read-only (which the compiler can't know
    Guido> but the user can).

Wouldn't the compiler be able to tell it was to be treated specially if it
saw b"GET"?  In that case, the code generated for

    x = b"GET"

would be something like

    LOAD_CONST     "GET"
    LOAD_NAME      bytes
    CALL_FUNCTION  1
    STORE_FAST     x

Skip
From tim.peters at gmail.com  Thu Aug 12 21:41:37 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 12 21:41:53 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <20040812185521.GA2277@vicky.ecs.soton.ac.uk>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
	<20040812185521.GA2277@vicky.ecs.soton.ac.uk>
Message-ID: <1f7befae04081212414007274f@mail.gmail.com>

[Tim]
>> Looks like nobody has an explanation yet for why 2.3.4 consistently
>> yielded MemoryError but 2.4a2 mixes those with spurious KeyError and
>> SyntaxError exceptions.

[Armin]
> The reason is that Py_EnterRecursiveCall() was only introduced in 2.4.
> Comparisons didn't throw RuntimeErrors that easily in 2.3.

Doh!  Of course.  Recursive compares aren't implicated in the test
programs that failed here.  Under 2.3.4, all the compares return
normally because they're not stack-checking at all, and the program
"gets to" recurse deeper then, until a "recursion depth exceeded"
exception gets thrown.  But in 2.4 it's a crap shoot whether a
comparison or a recursive Python call notices first that we're nearing
the end of the stack.  The recursive Python calls make real stack
demands in these tests, but the comparisons are just int-vs-int and
string-vs-string, so the stack-check in compare now is "almost always"
a nuisance check in these tests.

Sounds like a good idea not to run out to stack <wink>.
From barry at python.org  Thu Aug 12 21:47:41 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Aug 12 21:47:25 2004
Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered
In-Reply-To: <16667.49563.652742.680876@montanaro.dyndns.org>
References: <1092179578.4119567af011d@webmail.oregonstate.edu>
	<cfg60f$978$1@sea.gmane.org>
	<16667.49563.652742.680876@montanaro.dyndns.org>
Message-ID: <1092340061.8681.109.camel@localhost>

On Thu, 2004-08-12 at 15:14, Skip Montanaro wrote:

> I think '*' is syntactically available for this use.

That was one of the characters I tried back when I was looking for @
alternatives.  For some reason I can't explain, this one just rubs me
the wrong way.

i'll-let-tim-psychoanalyze-the-reasons-why-ly y'rs,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040812/5f6b69ed/attachment.pgp
From skip at pobox.com  Thu Aug 12 22:40:58 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 12 22:41:06 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <411BB50C.8000804@v.loewis.de>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org>
	<411B0DA0.4030109@v.loewis.de>
	<16667.32457.28553.468008@montanaro.dyndns.org>
	<411BB50C.8000804@v.loewis.de>
Message-ID: <16667.54746.535997.120878@montanaro.dyndns.org>


    >> That's why I think a special literal is necessary.  There'd be no
    >> unicode foolishness involved. ;-) They'd just be raw uninterpreted
    >> bytes.

    Martin> But you'd spell them b"GET", no? If so, which numeric value has
    Martin> "G"?

Good point...

From arigo at tunes.org  Thu Aug 12 22:44:31 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu Aug 12 22:49:10 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae04081212414007274f@mail.gmail.com>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
	<20040812185521.GA2277@vicky.ecs.soton.ac.uk>
	<1f7befae04081212414007274f@mail.gmail.com>
Message-ID: <20040812204431.GA31884@vicky.ecs.soton.ac.uk>

Hello,

On Thu, Aug 12, 2004 at 03:41:37PM -0400, Tim Peters wrote:
> Sounds like a good idea not to run out to stack <wink>.

I don't remember if it was mentioned here, but maybe we'd better check
directly whether the C stack is too large rather than (or possibly in addition
to) using a limit on the number of Python iterations.

This is not really ANSI C, but I can't think of a setting in which the
following trick would fail to be a good indication of the amount of stack
space consumed: save a pointer to a local variable "early", e.g. in
Py_Initialize(); then in any other function call, the distance between this
pointer and a pointer to some local equals the amount of stack space consumed
by Python minus a few bytes.

If this sounds too much of a hack, the (usual) recursion limit could be kept
to limit nested Python frames; but all C-level recursions (including
eval_frame) could additionally use the above trick.  Its advantage is that it
is an extremely fast check.

If the total stack size is difficult to know in advance, we can still use
PyOS_CheckStack(), but more efficiently and less randomly than now, by
maintaining a "high tide" pointer that records how much stack we are sure we
have, and calling PyOS_CheckStack() only to attempt to push the "high tide"
mark further.

While I'm in a hacking mood, there might be a way to prevent PyErr_Clear() to
clear away "asynchronuous" exceptions like RuntimeError, MemoryError, and
KeyboardInterrupt: for these exceptions, let's just store them away instead of
clearing them, and re-raise them at the next occasion.  The "next occasion"
would have to be defined more precisely, but there is probably a way to ensure
that it will at least be before the next non-trivial opcode operation starts.  
It would overwrite an exception set later, like those spurious KeyError we get
for dict lookups.  It might be a better-than-nothing quick fix to all these
PyErr_Clear() all around the code base.


A bient?t,

Armin.
From guido at python.org  Thu Aug 12 22:56:07 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 22:56:13 2004
Subject: [Python-Dev] adding a bytes sequence type to Python
In-Reply-To: Your message of "Thu, 12 Aug 2004 14:19:22 CDT."
	<16667.49850.156147.876146@montanaro.dyndns.org> 
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org>
	<411B0DA0.4030109@v.loewis.de>
	<200408121430.i7CEU6F30168@guido.python.org> 
	<16667.49850.156147.876146@montanaro.dyndns.org> 
Message-ID: <200408122056.i7CKu7n30960@guido.python.org>

> Wouldn't the compiler be able to tell it was to be treated specially if it
> saw b"GET"?  In that case, the code generated for
> 
>     x = b"GET"
> 
> would be something like
> 
>     LOAD_CONST     "GET"
>     LOAD_NAME      bytes
>     CALL_FUNCTION  1
>     STORE_FAST     x

This is actually an illustration of what I meant: a performance-aware
person might want to move that CALL_FUNCTION out of an inner loop if
they knew the result was never modified inside the loop.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Aug 12 22:53:27 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 12 22:57:56 2004
Subject: [Python-Dev] Re: Another approach to decorators.
In-Reply-To: Your message of "Thu, 12 Aug 2004 14:10:37 EDT."
	<uwu04dy76.fsf@boost-consulting.com> 
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<411A898F.3060004@ocf.berkeley.edu>
	<20040811142354.661970e0.michel@dialnetwork.com>
	<20040811142802.0386bd60.michel@dialnetwork.com>
	<200408120426.i7C4QU829083@guido.python.org> 
	<uwu04dy76.fsf@boost-consulting.com> 
Message-ID: <200408122053.i7CKrRg30941@guido.python.org>

> > I think this is too far-fetched to consider as an alternative to the
> > humble @decorator.
> 
> Maybe, but if you're still serious about improving support for
> domain-specific embedded languages in Python you ought to give the
> ideas another look.  If a reasonable syntax for decorators falls out
> of a more general and useful mechanism, so much the better.

How does

    decorate <expr>, ...:
       <block>

improve support for domain-specific embedded languages?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From python at rcn.com  Thu Aug 12 23:12:10 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Aug 12 23:12:27 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <1092177771.21839.54.camel@localhost>
Message-ID: <007301c480b1$0867bec0$5229c797@oemcomputer>

[Barry]
> Now for the controversial bit <wink>.
> 
> I still think it's worthwhile to turn the string module into a
package.
> The sandbox has an implementation of this that is completely backward
> compatible.  I like it because it segregates the deprecated functions
> into a separate sub-module and makes the whole shebang easier to
> maintain.  It also provides an existing namespace for future
expansion,
> as has been discussed before.
> 
> Please note: PEP 292 is not dependent on the string module -> package
> reorganization, so it isn't documented in that PEP.  I don't think
it's
> worth writing a PEP just for this library reorganization.  Take a look
> at the sandbox if you're interested.
> 
>
http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/stri
ng
> /
> 
> As far as this aspect of the changes go, I would like BDFL
pronouncement
> on it.  I'll abide by Guido's decision here.  If he nixes the re-org,
> then I propose to put the template and safe_template classes in
> Lib/string.py.  The libstring.tex changes will still be relevant.

After more thought, I would like to voice a strong -1 on the packaging
of string.

At the outset, I felt that it introduced unnecessary complexity.  Adding
a new directory, an __init__.py, and two modules just to add a few lines
of code is overkill.  Package logic should be reserved for organizing
voluminous code like the email package.  Arguably, the logging package
should have been just a single module -- putting it in a package just
made it more difficult to read and maintain.

To test the existing sandbox code, I moved it into the library and found
a circular import issue:  importing the string package, causes
template.py to be imported, which in turn imports re which uses
sre_parse.py, needs to import string.  So, the whole thing fails.  It is
fixable, but it sucks to have to start doing work-arounds from the
outset just to compensate for something that adds no value to begin
with.

The package has already complicated review and updates.  I'm having to
re-write it in non-package form just to add doctest (which fails by the
way) and other small fix-ups.

My recommendation:  Create a stringtools module with the new template
stuff in it and with room for growth.  Leave string.py alone and let it
die in peace.

The business end of the template code is only a few lines long.
Wrapping it in a package is like using cannon to kill a mosquito.  Let's
not make something hard out of something trivially simple.



Raymond



P.S.  Side rant:  Also, in the last few months, the #ifdefs have
multiplied.  While I understand that some things like time stamping are
worthwhile, the code is getting harder to read and maintain because of
all this b.s.  

At some point, someone changed a definition and I had to add an #ifdef
to import addrinfo.h for VC++ 6.0.  I noticed the same had to be done
for Cygwin and there was a similar GCC entry.  Each checkin was done by
a separate person fixing a crash in their own part of the universe.  The
right answer would have likely been to refactor the original change that
mucked up everyone else's compile.   Instead we ended up with a pile of
conditionals that will be hard to maintain going forward because the
surrounding code has become so stiff.

From bob at redivi.com  Thu Aug 12 23:25:24 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Aug 12 23:25:58 2004
Subject: [Python-Dev] Re: Another approach to decorators.
In-Reply-To: <200408122053.i7CKrRg30941@guido.python.org>
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<411A898F.3060004@ocf.berkeley.edu>
	<20040811142354.661970e0.michel@dialnetwork.com>
	<20040811142802.0386bd60.michel@dialnetwork.com>
	<200408120426.i7C4QU829083@guido.python.org>
	<uwu04dy76.fsf@boost-consulting.com>
	<200408122053.i7CKrRg30941@guido.python.org>
Message-ID: <1F6D2F67-ECA6-11D8-B20E-000A95686CD8@redivi.com>


On Aug 12, 2004, at 4:53 PM, Guido van Rossum wrote:

>>> I think this is too far-fetched to consider as an alternative to the
>>> humble @decorator.
>>
>> Maybe, but if you're still serious about improving support for
>> domain-specific embedded languages in Python you ought to give the
>> ideas another look.  If a reasonable syntax for decorators falls out
>> of a more general and useful mechanism, so much the better.
>
> How does
>
>     decorate <expr>, ...:
>        <block>
>
> improve support for domain-specific embedded languages?

I guess it depends on what is passed to the decorators.  If enough 
information is passed to the decorators it might almost be like a 
macro.  Practically speaking, I don't see how it would help very much 
considering that the block would be executed before the expression.  If 
it wasn't executed before the decorators, then I don't see how it would 
be any better than

	@<expr>
	def throwawayname():
		<block>

I suppose that decorators in the decorate <expr>, ...: <block> syntax 
would take a dict of the block's local namespace and return a dict of 
anything that should be set in the local namespace of the decorate 
statement.

But now that I think about it some more, it would probably be more 
useful to domain-specific languages if the decorators were allowed to 
modify the namespace *before* and *after* the block.  For example, to 
import a whole bunch of stuff for a domain specific purpose (say, 
writing SQL code, doing AppleEvents, etc.) and then get rid of it 
afterwards?

Anyway, I think that the decorate <expr>, ...: syntax might be useful 
someday, but I don't think that it could usefully have the same 
semantics as the current decorator PEP.

-bob
From foom at fuhm.net  Thu Aug 12 23:28:05 2004
From: foom at fuhm.net (James Y Knight)
Date: Thu Aug 12 23:28:10 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but
	aren't
In-Reply-To: <200408110002.i7B02gu26415@guido.python.org>
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com>
	<2macx31423.fsf@starship.python.net>
	<200408101501.i7AF1HZ25725@guido.python.org>
	<BE05E963-EAE8-11D8-9899-000A95A50FB2@fuhm.net>
	<200408110002.i7B02gu26415@guido.python.org>
Message-ID: <7FA48FA3-ECA6-11D8-BF5E-000A95A50FB2@fuhm.net>

On Aug 10, 2004, at 8:02 PM, Guido van Rossum wrote:
>> Sooo should (for 'generator' in objects that claim to be in
>> __builtins__ but aren't),
>> 1) 'generator' be added to __builtins__
>> 2) 'generator' be added to types.py and its __module__ be set to 
>> 'types'
>> 3) 'generator' be added to <newmodule>.py and its __module__ be set to
>> '<newmodule>' (and a name for the module chosen)
>
> I guess (1).
>
> Please submit a patch to SF...

Okay, so, I don't have a patch, but I have made a script to examine the 
situation. It seems the problem is a little more widespread than I had 
realized. I grepped through the python C source to find all the type 
objects, and then verified their existence. Just looking at 
Objects/*.c, there are 31 classes without appropriate bindings. 
Additionally, some aren't even valid identifier names. It seems like 
there's 4 categories here: iterators, descrobject stuffs, list 
comparison wrappers, and other. I'm not sure it's a good idea to add 
all these names to the builtins -- perhaps only the ones in the 'other' 
group?

For {Modules,Mac,PC,RISCOS,Python}/*.c, most seem like they could 
relatively straightforwardly be added to their module, but I haven't 
examined them too closely. Some likely ought to have constructors added 
(e.g. the dl.dl('file') could be equivalent to dl.open('file')).

__builtin__.dictionary-keyiterator   not found (./Objects/dictobject.c)
__builtin__.dictionary-valueiterator   not found 
(./Objects/dictobject.c)
__builtin__.dictionary-itemiterator   not found (./Objects/dictobject.c)
__builtin__.tupleiterator   not found (./Objects/tupleobject.c)
__builtin__.rangeiterator   not found (./Objects/rangeobject.c)
__builtin__.iterator   not found (./Objects/iterobject.c)
__builtin__.callable-iterator   not found (./Objects/iterobject.c)
__builtin__.listiterator   not found (./Objects/listobject.c)
__builtin__.listreverseiterator   not found (./Objects/listobject.c)

__builtin__.method_descriptor   not found (./Objects/descrobject.c)
__builtin__.classmethod_descriptor   not found (./Objects/descrobject.c)
__builtin__.member_descriptor   not found (./Objects/descrobject.c)
__builtin__.getset_descriptor   not found (./Objects/descrobject.c)
__builtin__.wrapper_descriptor   not found (./Objects/descrobject.c)
__builtin__.method-wrapper   not found (./Objects/descrobject.c)

__builtin__.sortwrapper   not found (./Objects/listobject.c)
__builtin__.cmpwrapper   not found (./Objects/listobject.c)

__builtin__.ellipsis   not found (./Objects/sliceobject.c)   
types.EllipsisType
__builtin__.builtin_function_or_method   not found 
(./Objects/methodobject.c)   types.BuiltinFunctionType 
types.BuiltinMethodType
__builtin__.dictproxy   not found (./Objects/descrobject.c)   
types.DictProxyType
__builtin__.generator   not found (./Objects/genobject.c)   
types.GeneratorType
__builtin__.PyCObject   not found (./Objects/cobject.c)
__builtin__.classobj   not found (./Objects/classobject.c)   
types.ClassType
__builtin__.instance   not found (./Objects/classobject.c)   
types.InstanceType
__builtin__.instancemethod   not found (./Objects/classobject.c)   
types.MethodType types.UnboundMethodType
__builtin__.cell   not found (./Objects/cellobject.c)
__builtin__.NoneType   not found (./Objects/object.c)   types.NoneType
__builtin__.NotImplementedType   not found (./Objects/object.c)   
types.NotImplementedType
__builtin__.frame   not found (./Objects/frameobject.c)   
types.FrameType
__builtin__.function   not found (./Objects/funcobject.c)   
types.FunctionType types.LambdaType
__builtin__.module   not found (./Objects/moduleobject.c)   
types.ModuleType


(the Mac ones I checked on my mac w/ python 2.3.0)
_Qt.IdleManager   not found (./Mac/Modules/qt/_Qtmodule.c)
_Qt.SGOutput   not found (./Mac/Modules/qt/_Qtmodule.c)
module _OSA not installed (./Mac/Modules/osa/_OSAmodule.c)
Nav.NavReplyRecord   not found (./Mac/Modules/Nav.c)
_Scrap.Scrap   not found (./Mac/Modules/scrap/_Scrapmodule.c)
module waste not installed (./Mac/Modules/waste/wastemodule.c)
MacOS.ResourceFork   not found (./Mac/Modules/macosmodule.c)
icglue.ic_instance   not found (./Mac/Modules/icgluemodule.c)


__builtin__.PyHKEY   not found (./PC/_winreg.c)
__builtin__.drawf   not found (./RISCOS/Modules/drawfmodule.c)
__builtin__.block   not found (./RISCOS/Modules/swimodule.c)
__builtin__.traceback   not found (./Python/traceback.c)   
types.TracebackType
__builtin__.code   not found (./Python/compile.c)   types.CodeType
__builtin__.symtable entry   not found (./Python/symtable.c)
__builtin__.tktimertoken   not found (./Modules/_tkinter.c)
__builtin__.tkapp   not found (./Modules/_tkinter.c)
__builtin__.arrayiterator   not found (./Modules/arraymodule.c)
_curses_panel.curses panel   not found (./Modules/_curses_panel.c)
linuxaudiodev.linux_audio_device   not found (./Modules/linuxaudiodev.c)
module fl not installed (./Modules/flmodule.c)
__builtin__.DB   not found (./Modules/_bsddb.c)
__builtin__.DBCursor   not found (./Modules/_bsddb.c)
__builtin__.DBEnv   not found (./Modules/_bsddb.c)
__builtin__.DBTxn   not found (./Modules/_bsddb.c)
__builtin__.DBLock   not found (./Modules/_bsddb.c)
sha.SHA   not found (./Modules/shamodule.c)
module sv not installed (./Modules/svmodule.c)
itertools._grouper   not found (./Modules/itertoolsmodule.c)
itertools.tee_dataobject   not found (./Modules/itertoolsmodule.c)
rotor.rotor   not found (./Modules/rotormodule.c)
module cl not installed (./Modules/clmodule.c)
_sre.SRE_Pattern   not found (./Modules/_sre.c)
_sre.SRE_Match   not found (./Modules/_sre.c)
_sre.SRE_Scanner   not found (./Modules/_sre.c)
socket.SSL   not found (./Modules/_ssl.c)
_curses.curses window   not found (./Modules/_cursesmodule.c)
parser.st   not found (./Modules/parsermodule.c)
cStringIO.StringO   not found (./Modules/cStringIO.c)
cStringIO.StringI   not found (./Modules/cStringIO.c)
module sunaudiodev not installed (./Modules/sunaudiodev.c)
module dbm not installed (./Modules/dbmmodule.c)
dl.dl   not found (./Modules/dlmodule.c)
module fm not installed (./Modules/fmmodule.c)
regex.regex   not found (./Modules/regexmodule.c)
pyexpat.xmlparser   not found (./Modules/pyexpat.c)
__builtin__.MultibyteCodec   not found 
(./Modules/cjkcodecs/multibytecodec.c)
__builtin__.MultibyteStreamReader   not found 
(./Modules/cjkcodecs/multibytecodec.c)
__builtin__.MultibyteStreamWriter   not found 
(./Modules/cjkcodecs/multibytecodec.c)
bsddb.bsddb   not found (./Modules/bsddbmodule.c)
module cd not installed (./Modules/cdmodule.c)
cPickle.Pdata   not found (./Modules/cPickle.c)
module al not installed (./Modules/almodule.c)
__builtin__.deque_iterator   not found (./Modules/collectionsmodule.c)
__builtin__.deque_reverse_iterator   not found 
(./Modules/collectionsmodule.c)
thread.lock   not found (./Modules/threadmodule.c)
zlib.Compress   not found (./Modules/zlibmodule.c)
zlib.Decompress   not found (./Modules/zlibmodule.c)
gdbm.gdbm   not found (./Modules/gdbmmodule.c)
ossaudiodev.oss_audio_device   not found (./Modules/ossaudiodev.c)
ossaudiodev.oss_mixer_device   not found (./Modules/ossaudiodev.c)

From barry at python.org  Fri Aug 13 00:16:10 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 13 00:15:56 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/nondist/sandbox/string/string
	__init__.py, 1.2, 1.3 template.py, 1.1, 1.2
In-Reply-To: <E1BvMzs-0001ww-6H@sc8-pr-cvs1.sourceforge.net>
References: <E1BvMzs-0001ww-6H@sc8-pr-cvs1.sourceforge.net>
Message-ID: <1092348970.8679.120.camel@localhost>

On Thu, 2004-08-12 at 17:20, rhettinger@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/nondist/sandbox/string/string
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7407
> 
> Modified Files:
> 	__init__.py template.py 
> Log Message:
> Capitalized the class names.
> Fixed a broken example
> Cleaned-up the namespace.
> Added __slots__ for compactness.
> Doctest the example.

Thanks!

pining-for-my-page-breaks-ly y'rs,
-Barry

:)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040812/454a8c6f/attachment.pgp
From mike at skew.org  Fri Aug 13 00:26:23 2004
From: mike at skew.org (Mike Brown)
Date: Fri Aug 13 00:26:21 2004
Subject: [Python-Dev] Could getpass use streams other than stdin?
Message-ID: <200408122226.i7CMQNvQ017619@chilled.skew.org>

On posix, it would be useful if I could have getpass.getpass() write the 
prompt to stderr or to an arbitrary stream (file-like object), rather than 
always writing to stdin.

In my use case, I need to send the results of a command-line script to stdout 
after the user authenticates, and the user may be redirecting stdout, so I 
don't want the password prompt to pollute the output. If I could have the 
"Password: " show up in stderr, that would be ideal.

I don't think this would be terribly difficult to implement an API like

def getpass(prompt='', stream=None)
    if stream is None:
        stream = sys.stdin
    ...

but I'm unsure of the nuances of termios and arbitrary streams. Would it be 
better to just make the choice be between stdin and stderr? -- in which case 
the API would be something like

def getpass(prompt='', use_stderr=False)

Or is the whole exercise a waste of time?

Thanks
From mike at skew.org  Fri Aug 13 00:35:00 2004
From: mike at skew.org (Mike Brown)
Date: Fri Aug 13 00:34:56 2004
Subject: [Python-Dev] Could getpass use streams other than stdin?
In-Reply-To: <200408122226.i7CMQNvQ017619@chilled.skew.org> "from Mike Brown
	at Aug 12, 2004 04:26:23 pm"
Message-ID: <200408122235.i7CMZ0Eg017699@chilled.skew.org>

> def getpass(prompt='', stream=None)
>     if stream is None:
>         stream = sys.stdin
>     ...

Of course I meant sys.stdout there.
From arigo at tunes.org  Fri Aug 13 00:59:41 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri Aug 13 01:04:21 2004
Subject: [Python-Dev] Third Bug Day outcome
In-Reply-To: <20040809213820.GC11199@rogue.amk.ca>
References: <20040809213820.GC11199@rogue.amk.ca>
Message-ID: <20040812225941.GA24248@vicky.ecs.soton.ac.uk>

Hi,

On Mon, Aug 09, 2004 at 05:38:20PM -0400, A.M. Kuchling wrote:
> During the day, Armin Rigo wrote a nifty IRC bot that takes an SF
> bug/patch ID and returns the title; taking an example from the transcript:
> 
> 20:53:56 <jlgijsbers> Looking at #777659.
> 20:53:57 <sf_number> * sf_number bug 777659 - Uninitialized variable used in Tools/faqwiz/faqwiz.py

Source code is at  http://codespeak.net/svn/user/arigo/pypy-hack/ , files
pysf*.py.


Armin
From tdelaney at avaya.com  Fri Aug 13 01:20:43 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Fri Aug 13 01:22:04 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF460F@au3010avexu1.global.avaya.com>

Gareth McCaughan wrote:

> I don't think this is a good reason for rejecting "integer"
> as the name of the common supertype of int and long. You'd use
> isinstance(x,integer) to check whether x is an integer of
> built-in type, just as you currently use isinstance(x,float)
> to check whether x is a floating-point number of built-in
> type. I see no reason why a cumbersome name is much advantage.

All this brings to mind - is there actually a good reason to need a base
type? Why not just define baseinteger as:

    baseinteger = int, long

if the only reason for it is to use isinstance?

Tim Delaney
From tdelaney at avaya.com  Fri Aug 13 01:25:36 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Fri Aug 13 01:27:00 2004
Subject: [Python-Dev] Re: Decorators after 'def' should be reconsidered
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF4617@au3010avexu1.global.avaya.com>

Barry Warsaw wrote:

> On Thu, 2004-08-12 at 15:14, Skip Montanaro wrote:
> 
>> I think '*' is syntactically available for this use.
> 
> That was one of the characters I tried back when I was looking for @
> alternatives.  For some reason I can't explain, this one just rubs me
> the wrong way.
>
> i'll-let-tim-psychoanalyze-the-reasons-why-ly y'rs,

Well, not the Tim you were thinking of, but ...

I think the reason * doesn't work is that it is partially superscripted
in most fonts.

For example, compare:

    *deco
    @deco
    -deco

@ and - are vertically centred with the text. However, * is positioned
slightly above the text.

Tim Delaney
From guido at python.org  Fri Aug 13 01:30:16 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug 13 01:30:22 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: Your message of "Fri, 13 Aug 2004 09:20:43 +1000."
	<338366A6D2E2CA4C9DAEAE652E12A1DE01CF460F@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF460F@au3010avexu1.global.avaya.com>
Message-ID: <200408122330.i7CNUGG31292@guido.python.org>

> All this brings to mind - is there actually a good reason to need a base
> type? Why not just define baseinteger as:
> 
>     baseinteger = int, long
> 
> if the only reason for it is to use isinstance?

So that an extension author *could* write an int-like type deriving
from it?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tdelaney at avaya.com  Fri Aug 13 01:33:41 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Fri Aug 13 01:34:49 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01CF461C@au3010avexu1.global.avaya.com>

guido@python.org wrote:

>> All this brings to mind - is there actually a good reason to need a
>> base type? Why not just define baseinteger as:
>> 
>>     baseinteger = int, long
>> 
>> if the only reason for it is to use isinstance?
> 
> So that an extension author *could* write an int-like type deriving
> from it?

Fair 'nuff.

Tim Delaney
From greg at cosc.canterbury.ac.nz  Fri Aug 13 03:21:22 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug 13 03:21:29 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <200408121420.i7CEKdo30140@guido.python.org>
Message-ID: <200408130121.i7D1LMqZ013619@cosc353.cosc.canterbury.ac.nz>

Guido:

> > Perhaps the constructor for a byte array could accept a string
> > argument as long as it contained only ascii characters?
> > 
> >    h.putrequest(bytes('GET'), selector)
> 
> Yeah, but that's what Martin called depending on the default encoding.

I don't see anything wrong with that. It would be a fixed default,
defined by the language, not something site-dependent that could shift
under you.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From mkc at mathdogs.com  Fri Aug 13 03:43:01 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Fri Aug 13 03:43:06 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<lkdwu0bcz7r.fsf@panix2.panix.com>
	<87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp>
	<lkdacx37oxr.fsf@panix2.panix.com>
	<873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp>
	<lkdpt5x3rul.fsf@panix2.panix.com> <411B4A0A.1030201@heneryd.com>
Message-ID: <lkdzn4zn78a.fsf@panix2.panix.com>

Erik Heneryd <erik@heneryd.com> writes:
> Well, I guess that if you want structmatch into the stdlib you'll have to show
> that it's better than it's alternatives.  Including those parser packages.

As a practical matter, this may well be the case.  I look at it kind of like

    'structmatch' is to (a grammar parser) 
    as 
    'sed' is to (a full scripting language)

That is, it has its niche, but it certainly not as general or powerful as a
full parsing package.  I'm not sure, either, exactly how to show that it's
better.  That said, I use sed all the time, not because it's a better than
full scripting languages, but because it nicely fits the problem I'm
addressing better.

> You'd still have to do a real implementation.  If it can't be done without
> rewriting a whole lot of code, that would be a problem.

I agree.  Being busy and lazy, I'm trying to get a bead first on whether this
would be a wasted effort.

> Hmm... think this is the wrong approach.  Your PEP is not just about
> "structured matching", it tries to deal with a couple of issues and I think it
> would be better to address them separately, one by one:
> 
> * Parsing/scanning - this is mostly what's been discussed so far...
> 
> * Capturing repeated groups - IMO nice-to-have (tm) but not something I would
> lose sleep over.  Hard to do.
> 
> * Partial matches - would be great for debugging more complex regexes. Why not
> a general re.RAISE flag raising an exception on failure?

This is true.  For me, the second is fundamental--it's why I'm bothering with
this.  The third is a useful add-on, and as you suggest could probably be
added orthogonally to several of the existing methods.

The first I'm not sure about.  I don't think re.structmatch does
scanning--that's not really the problem it tries to solve.  As for "parsing",
I guess it depends on what you mean by that.

Certainly it would be possible to address the "repeated groups" point without
the whole structured return value thing, but I'm not seeing what would be
better.

Mike

From greg at cosc.canterbury.ac.nz  Fri Aug 13 03:57:45 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug 13 03:57:49 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <200408122330.i7CNUGG31292@guido.python.org>
Message-ID: <200408130157.i7D1vjoT013689@cosc353.cosc.canterbury.ac.nz>

Guido:

> > if the only reason for it is to use isinstance?
> 
> So that an extension author *could* write an int-like type deriving
> from it?

But didn't you just say that people shouldn't be
deriving their own int-like types from baseinteger?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From guido at python.org  Fri Aug 13 04:35:24 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug 13 04:35:34 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: Your message of "Fri, 13 Aug 2004 13:57:45 +1200."
	<200408130157.i7D1vjoT013689@cosc353.cosc.canterbury.ac.nz> 
References: <200408130157.i7D1vjoT013689@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408130235.i7D2ZOE31624@guido.python.org>

> Guido:
> 
> > > if the only reason for it is to use isinstance?
> > 
> > So that an extension author *could* write an int-like type deriving
> > from it?
> 
> But didn't you just say that people shouldn't be
> deriving their own int-like types from baseinteger?

Indeed, in general they shouldn't.  But for specialized purposes it
might be needed (that's why I emphasized *could*).

Unfortunately, for the purpose of having one's ersatz integers
accepted as the real thing by the Python core, you pretty much have to
inherit from int.

So my only reason to make baseinteger a class rather than a tuple is
consistency with basestring.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From bac at OCF.Berkeley.EDU  Fri Aug 13 04:46:10 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug 13 04:46:15 2004
Subject: [Python-Dev] Re: re.split on empty patterns
In-Reply-To: <lkd4qner2xq.fsf@panix2.panix.com>
References: <20040807145142.GB2529@rogue.amk.ca>
	<lkd4qner2xq.fsf@panix2.panix.com>
Message-ID: <411C2B72.4090809@ocf.berkeley.edu>

Mike Coleman wrote:

[SNIP]
>     # alternative 2:
>     re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', 'bbczz']
                                                                ^
>     re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz']
> 
>     # alternative 3:
>     re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', '', 'bbczz']
                                                                    ^
>     re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz']
> 

I take it the first 'b' in both of the first examples for each 
alternative were supposed to be 'a'?

And as for which version, I actually like Mike's version more than the 
one AMK and Tim like.  The reason is that the '' in the middle of the 
example in question in the OP tells you where the split would have 
occurred had split0 (I like that or 'split_empty') not been turned on. 
That way there is no real loss of data between the two, but a gain with 
the new feature being used.

-Brett
From klm at zope.com  Fri Aug 13 00:28:57 2004
From: klm at zope.com (Ken Manheimer)
Date: Fri Aug 13 06:00:31 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
References: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz>
	<1092108578.18208.10.camel@localhost>
	<1f7befae040809210679c2d4a0@mail.gmail.com>
Message-ID: <87n0109eja.fsf@smtp.zope.com>

Tim Peters <tim.peters@gmail.com> writes:

> [Barry Warsaw]
> > Actually, that's an interesting case study.  Myself and others advocated
> > for a keyword other than 'def' for introducing generators.  That wasn't
> > how things turned out and in hindsight I think Guido made the right
> > decision.  I'm confident the same thing will happen with decorators.
> 
> Ya, but it's sure taking Guido long enough to figure that out in this case!
> 
> def staticmethod:
> def author(name='Barry'):
> def returns([int]):
> def method(self):
>      return 42

Oh my.  I'm in the unfortunate position of liking this more than any of the
other options i've seen - while strongly suspecting you were not serious,
and would think a fool anyone that would consider it for a moment.  Ah
well.  The truth is out.

The only adjustment i would make is to not include the ":" on the modifier
lines:

  def staticmethod
  def author(name='yarrB')
  def returns([int])
  def plugh(self):
       return 42

> a-keyword-for-all-seasons-ly y'rs  - tim

Could be!

Ken
klm@zope.com

From mark at prothon.org  Thu Aug 12 23:20:26 2004
From: mark at prothon.org (Mark Hahn)
Date: Fri Aug 13 06:26:09 2004
Subject: [Python-Dev] Re: Prothon on CPython intrepreter? [PROTHON]
References: <000a01c47f0b$b3cf4690$0b01a8c0@mark>
Message-ID: <h6uqaavdlk6k.1xgy4tpg1nt2a.dlg@40tude.net>

On Tue, 10 Aug 2004 11:56:05 -0700, Mark Hahn wrote:

> This CPython->.Net->CProthon roadmap will get Prothon out quickly and slowly
> ease my wild ideas for the interpreter into Prothon.  It will also make
> CPython a multiple language VM for the first time.

I've gotten a cold reception to this idea of implementing Prothon on the 
CPython interpreter.  Not only did Christian Tismer (and he should know) 
and others warn me of the technnical challenge I was signing up for, but 
there wasn't exactly a groundswell of enthusiasm for the idea overall.

So forget I bothered you about this crazy idea.  I'll be sticking with my 
public announcement about the first implementation of Prothon being on .Net 
and Mono.  The native CProthon interpreter will be next in line in 2005.

I will of course always be interested in any and all ideas for alternatives 
to .Net for getting libraries for Prothon.  Besides CPython, I have 
considered Parrot, Mozilla, PyPy, and Java and ruled them out for 
alternatives to .Net for the first implementation.

From ncoghlan at iinet.net.au  Fri Aug 13 07:50:29 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Fri Aug 13 07:50:38 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <87n0109eja.fsf@smtp.zope.com>
References: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz>	<1092108578.18208.10.camel@localhost>	<1f7befae040809210679c2d4a0@mail.gmail.com>
	<87n0109eja.fsf@smtp.zope.com>
Message-ID: <411C56A5.8070404@iinet.net.au>

Ken Manheimer wrote:
> Oh my.  I'm in the unfortunate position of liking this more than any of the
> other options i've seen - while strongly suspecting you were not serious,
> and would think a fool anyone that would consider it for a moment.  Ah
> well.  The truth is out.
> 
> The only adjustment i would make is to not include the ":" on the modifier
> lines:
> 
>   def staticmethod
>   def author(name='yarrB')
>   def returns([int])
>   def plugh(self):
>        return 42

Did you see the 'predef' proposal I posted the other day? It seemed 
perfect to me, but given the staggering lack of response to the post, I 
suspect others were rather underwhelmed. The above would become:

    predef staticmethod
    predef author(name='yarrB')
    predef returns([int])
    def plugh(self):
         return 42


To me, 'predef' seems to shout "Look, I'm telling you something about 
the 'def' that you are about to encounter in a line or two."

Ah well, guess I'll go back to being a lurker on the deco issue. . .

Cheers,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From fredrik at pythonware.com  Fri Aug 13 08:29:58 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri Aug 13 08:28:18 2004
Subject: [Python-Dev] Re: Could getpass use streams other than stdin?
References: <200408122226.i7CMQNvQ017619@chilled.skew.org>
Message-ID: <cfhn1u$c61$1@sea.gmane.org>

Mike Brown wrote:

> In my use case, I need to send the results of a command-line script to stdout
> after the user authenticates, and the user may be redirecting stdout, so I
> don't want the password prompt to pollute the output. If I could have the
> "Password: " show up in stderr, that would be ideal.

you can redirect sys.stdin/sys.stdout before you call getpass:

    try:
        old_stdout = sys.stdout
        sys.stdout = sys.stderr
        password = getpass.getpass(prompt)
    finally:
        sys.stdout = old_stdout

</F> 



From anthony at interlink.com.au  Fri Aug 13 10:15:07 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug 13 10:15:39 2004
Subject: [Python-Dev] Error in Python Tutorial on Multiple Inheritence
In-Reply-To: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au>
References: <200408120120.i7C1KKUh027264@jupiter.flashnet.com.au>
Message-ID: <411C788B.8030105@interlink.com.au>

Steven Kah Hien Wong wrote:
>     class CommonBase:
>         x = 0
> 
>     class ChildOne(CommonBase):
>         None
> 
>     class ChildTwo(CommonBase):
>         None
> 
>     class ChildOneAndTwo(ChildOne, ChildTwo):
>         def run(self):
>             ChildOne.x = 1
>             print "one =", ChildOne.x
>             print "two =", ChildTwo.x
> 
>     ChildOneAndTwo().run()
> 
> And the output was:
> 
>     $ python multi.py
>     one = 1
>     two = 0
> 
> According to the documentation, I thought it should be:
> 
>     one = 1
>     two = 1

Nope. In the code, you set a new attribute 'x' on the ChildOne
class.

Change the end of your code to
ChildOneAndTwo().run()
print "CommonBase.x", CommonBase.x
print "ChildOne.x", ChildOne.x
print "ChildTwo.x", ChildTwo.x

Anthony
From mal at egenix.com  Fri Aug 13 11:11:18 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Aug 13 11:11:31 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <16667.54746.535997.120878@montanaro.dyndns.org>
References: <1092258146.411a896246d1a@mcherm.com>	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>	<16666.56033.65067.241794@montanaro.dyndns.org>	<200408120414.i7C4Er429003@guido.python.org>	<411AFBD4.5080903@v.loewis.de>	<200408120545.i7C5j4W29250@guido.python.org>	<411B0DA0.4030109@v.loewis.de>	<16667.32457.28553.468008@montanaro.dyndns.org>	<411BB50C.8000804@v.loewis.de>
	<16667.54746.535997.120878@montanaro.dyndns.org>
Message-ID: <411C85B6.2060104@egenix.com>

Skip Montanaro wrote:
>     >> That's why I think a special literal is necessary.  There'd be no
>     >> unicode foolishness involved. ;-) They'd just be raw uninterpreted
>     >> bytes.
> 
>     Martin> But you'd spell them b"GET", no? If so, which numeric value has
>     Martin> "G"?
> 
> Good point...

I don't think I understand the example...

What's binary about 'GET' ?

Why would you want to put non-ASCII into a binary literal
definition ?

If we switch the binding of 'yyy' to mean unicode('yyy')
some day, why can't we just continue to use the existing implementation
for 8-bit strings for b'xxx' (the current implementation is already
doing the right thing, meaning that it is 8-bit safe regardeless
of the source code encoding) ?

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 13 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From ncoghlan at iinet.net.au  Fri Aug 13 12:39:26 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Fri Aug 13 12:39:33 2004
Subject: [Python-Dev] 'commands' module on Win32
Message-ID: <411C9A5E.3070109@iinet.net.au>

Bug #1008275 refers to attempting to run commands.getstatusoutput() on 
win32, and the fact that it breaks.

The reason it breaks is because it expects a posix shell, and this is 
noted in the documentation for the module (or rather, the module is 
noted as only supported on Unix).

commands.getstatusoutput() and commands.getoutput() can easily be fixed 
to run properly on Windows (using "if os.name == 'posix'" to select the 
correct formatting for the shell command). Only the first one actually 
needs to be fixed, since getoutput() just returns the second element of 
the tuple returned by getstatusoutput()

The dirty holdout is commands.getstatus().

This function does *not* simply return the first element of the tuple 
returned by commands.getstatusoutput() as you might expect. Instead of 
returning the status code resulting from executing its argument as a 
shell command, it insteads calls 'ls -ld <arg>' to get the file 
information. (And this is a function in the commands module exactly why?)

This actually works for me, but only because I have MSYS & MINGW 
installed :)

Anyway, getstatusoutput() and getoutput() seem like very handy ways to 
make a command line call and get its results. I'd never seen them before 
because they're squirelled away as Unix specific modules.

Are there any objections to changing this module so that only 
getstatus() is marked as Unix specific?

I ask, because I don't want to start work on a documentation patch if 
the idea is a non-starter (the code & test changes are already done, 
butI think the hardest part of the exercise will be moving the 
documentation out of the 'Unix-specific' part of the doc tree).

Cheers,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA

From mwh at python.net  Fri Aug 13 13:05:20 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug 13 13:05:22 2004
Subject: [Python-Dev] #ifdeffery
In-Reply-To: <007301c480b1$0867bec0$5229c797@oemcomputer> (Raymond
	Hettinger's message of "Thu, 12 Aug 2004 17:12:10 -0400")
References: <007301c480b1$0867bec0$5229c797@oemcomputer>
Message-ID: <2mwu03wb67.fsf@starship.python.net>

"Raymond Hettinger" <python@rcn.com> writes:

> P.S.  Side rant:  Also, in the last few months, the #ifdefs have
> multiplied.  While I understand that some things like time stamping are
> worthwhile, the code is getting harder to read and maintain because of
> all this b.s.  

Is it really getting worse?

I think there are two qualitatively different sorts of #ifdef mess:
feature ones, like the WITH_TSC I've recently been fiddling with
recently and portability ones.

The feature #ifdefs aren't (or shouldn't be) that much of a big deal.
Ideally, they are documented in Misc/SpecialBuilds.txt and depending
on whether they are defined or not, bits and pieces of code are or are
not included.

Platform #ifdefs, on the other hand, are pain in a tin.

> At some point, someone changed a definition and I had to add an #ifdef
> to import addrinfo.h for VC++ 6.0.  I noticed the same had to be done
> for Cygwin and there was a similar GCC entry.  Each checkin was done by
> a separate person fixing a crash in their own part of the universe.  The
> right answer would have likely been to refactor the original change that
> mucked up everyone else's compile.   Instead we ended up with a pile of
> conditionals that will be hard to maintain going forward because the
> surrounding code has become so stiff.

Of course the problem here is that it's takes a lot of courage to make
changes that affect systems you don't have access to.  One of the
goals of Include/pyport.h is that this localizes the mess: think of a
sensibly named preprocessor symbol (PY_WE_NEED_ADDRINFO_H?) and put
all the bullshit needed to work out whether to define it in one place.
But you really want to get things like this right, especially if one
of the affected platforms is something a little obscure.  Break things
on linux/x86 and you'll find out pretty quickly, but break them on
Irix and it may take months and it won't be obvious which change is
at fault.

Still, I want at some point to do test builds on all the test drive
machines at some point before 2.4 final, so any changes will hopefully
get some amount of testing on these systems at least.

Cheers,
mwh

-- 
  (Unfortunately, while you get Tom Baker saying "then we 
   were attacked by monsters", he doesn't flash and make 
  "neeeeooww-sploot" noises.)
      -- Gareth Marlow, ucam.chat, from Owen Dunn's review of the year
From jlgijsbers at planet.nl  Fri Aug 13 14:54:30 2004
From: jlgijsbers at planet.nl (Johannes Gijsbers)
Date: Fri Aug 13 14:53:14 2004
Subject: [Python-Dev] Re: 'commands' module on Win32
Message-ID: <20040813125429.GA22786@mail.planet.nl>

Nick Coghlan <ncoghlan@iinet.net.au> wrote:
> Are there any objections to changing this module so that only
> getstatus() is marked as Unix specific?

Not from me.
 
> I ask, because I don't want to start work on a documentation patch if
> the idea is a non-starter (the code & test changes are already done,
> butI think the hardest part of the exercise will be moving the
> documentation out of the 'Unix-specific' part of the doc tree).

This is pretty easy actually. In fact, while I was checking whether it
really was easy, I created a patch you could use. It's attached.

Johannes
-------------- next part --------------
cvs diff: Diffing .
Index: lib.tex
===================================================================
RCS file: /cvsroot/python/python/dist/src/Doc/lib/lib.tex,v
retrieving revision 1.229
diff -B -c -r1.229 lib.tex
*** lib.tex	7 Aug 2004 16:53:59 -0000	1.229
--- lib.tex	13 Aug 2004 12:27:42 -0000
***************
*** 189,194 ****
--- 189,195 ----
  \input{libtarfile}
  \input{libreadline}
  \input{librlcompleter}
+ \input{libcommands}
  
  \input{libunix}                 % UNIX Specific Services
  \input{libposix}
***************
*** 207,213 ****
  \input{libresource}
  \input{libnis}
  \input{libsyslog}
- \input{libcommands}
  
  \input{libpdb}                  % The Python Debugger
  
--- 208,213 ----
Index: libcommands.tex
===================================================================
RCS file: /cvsroot/python/python/dist/src/Doc/lib/libcommands.tex,v
retrieving revision 1.12
diff -B -c -r1.12 libcommands.tex
*** libcommands.tex	22 Apr 1999 15:53:35 -0000	1.12
--- libcommands.tex	13 Aug 2004 12:27:42 -0000
***************
*** 2,8 ****
           Utilities for running commands}
  
  \declaremodule{standard}{commands}
!   \platform{Unix}
  \modulesynopsis{Utility functions for running external commands.}
  \sectionauthor{Sue Williams}{sbw@provis.com}
  
--- 2,8 ----
           Utilities for running commands}
  
  \declaremodule{standard}{commands}
! \platform{Unix, Windows}
  \modulesynopsis{Utility functions for running external commands.}
  \sectionauthor{Sue Williams}{sbw@provis.com}
  
***************
*** 33,39 ****
  \begin{funcdesc}{getstatus}{file}
  Return the output of \samp{ls -ld \var{file}} as a string.  This
  function uses the \function{getoutput()} function, and properly
! escapes backslashes and dollar signs in the argument.
  \end{funcdesc}
  
  Example:
--- 33,39 ----
  \begin{funcdesc}{getstatus}{file}
  Return the output of \samp{ls -ld \var{file}} as a string.  This
  function uses the \function{getoutput()} function, and properly
! escapes backslashes and dollar signs in the argument. Availability: \UNIX.
  \end{funcdesc}
  
  Example:
From aahz at pythoncraft.com  Fri Aug 13 16:10:42 2004
From: aahz at pythoncraft.com (Aahz)
Date: Fri Aug 13 16:10:46 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <411C85B6.2060104@egenix.com>
References: <200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org>
	<411B0DA0.4030109@v.loewis.de>
	<16667.32457.28553.468008@montanaro.dyndns.org>
	<411BB50C.8000804@v.loewis.de>
	<16667.54746.535997.120878@montanaro.dyndns.org>
	<411C85B6.2060104@egenix.com>
Message-ID: <20040813141042.GB21742@panix.com>

On Fri, Aug 13, 2004, M.-A. Lemburg wrote:
>
> What's binary about 'GET' ?

It's an ASCII, human-readable representation of a set of bytes sent over
a network interface to command a server.  It could just as easily have
been "\010\011\012", but it was selected for the convenience of
English-speaking humans.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From dave at boost-consulting.com  Fri Aug 13 16:11:21 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Aug 13 16:12:04 2004
Subject: [Python-Dev] Re: Another approach to decorators.
In-Reply-To: <200408122053.i7CKrRg30941@guido.python.org> (Guido van Rossum's
	message of "Thu, 12 Aug 2004 13:53:27 -0700")
References: <20040811195454.9BD311E400A@bag.python.org>
	<20040811133711.5f1986b6.michel@dialnetwork.com>
	<411A898F.3060004@ocf.berkeley.edu>
	<20040811142354.661970e0.michel@dialnetwork.com>
	<20040811142802.0386bd60.michel@dialnetwork.com>
	<200408120426.i7C4QU829083@guido.python.org>
	<uwu04dy76.fsf@boost-consulting.com>
	<200408122053.i7CKrRg30941@guido.python.org>
Message-ID: <uoelfcely.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

>> > I think this is too far-fetched to consider as an alternative to the
>> > humble @decorator.
>> 
>> Maybe, but if you're still serious about improving support for
>> domain-specific embedded languages in Python you ought to give the
>> ideas another look.  If a reasonable syntax for decorators falls out
>> of a more general and useful mechanism, so much the better.
>
> How does
>
>     decorate <expr>, ...:
>        <block>
>
> improve support for domain-specific embedded languages?

I was thinking -- as well as I could through yesterday's allergy
attack -- more of the

      some-decorator:
          <block>

form, which allows people to create pseudo-statements of various
kinds.

Probably Bob I. is right that it doesn't help as much as it might if
<block> is executed before some-decorator is invoked.  That said,
maybe it's possible to pass the *code* for the block to
some-decorator in this scenario.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From guido at python.org  Fri Aug 13 16:30:42 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug 13 16:30:47 2004
Subject: [Python-Dev] 'commands' module on Win32
In-Reply-To: Your message of "Fri, 13 Aug 2004 03:39:26 PDT."
	<411C9A5E.3070109@iinet.net.au> 
References: <411C9A5E.3070109@iinet.net.au> 
Message-ID: <200408131430.i7DEUgK32685@guido.python.org>

> Bug #1008275 refers to attempting to run commands.getstatusoutput() on 
> win32, and the fact that it breaks.

Hm...  That module's functionality is such a small improvement over
raw os.popen, it would never have been admitted into the standard
library these days.

I'd rather see this added as a higher-level API to one of the process
management modules that are floating around, one of which will
eventually land in the library, at which point we can deprecate
commands.py.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Fri Aug 13 16:52:10 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug 13 16:52:20 2004
Subject: [Python-Dev] 'commands' module on Win32
In-Reply-To: <411C9A5E.3070109@iinet.net.au>
References: <411C9A5E.3070109@iinet.net.au>
Message-ID: <1f7befae040813075216398c46@mail.gmail.com>

[Nick Coghlan]
> ...
> Are there any objections to changing this module so that only
> getstatus() is marked as Unix specific?

-0.  It works on Unix.  On Windows you have to distinguish among
command.com systems (95/98/ME), cmd.exe systems (NT/2K/XP), and
systems where the user installed their own COMSPEC shell.  The
command.com systems can't redirect stderr, and always return a status
of 0 (command.com's own exit code, which has nothing to do with the
exit code of the command being run).  "It should work" on cmd.exe
systems.  Can't say anything definitive about use-my-own-shell
systems.

IOW, telling the truth would complicate the docs, and the result
wouldn't be portable across Windows boxes anyway.  So I'd rather leave
commands.py in peace, short, simple, obvious, but limited to Unix. 
It's not a suitable basis for a *good* solution to x-platform
command-line work (Tcl's "exec" subsystem is an example of a good
solution, but it's a lot of work).
From skip at pobox.com  Fri Aug 13 17:44:17 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug 13 17:44:23 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <slrnchnlm3.vq2.nick@irishsea.home.craig-wood.com>
References: <mailman.1506.1092237421.5135.python-list@python.org>
	<PcGdnQAV7olvoIfcRVn-ig@powergate.ca>
	<cfdh4q$kpm$1@nntp0.reith.bbc.co.uk>
	<d11dcfba.0408111433.43f025f8@posting.google.com>
	<slrnchnlm3.vq2.nick@irishsea.home.craig-wood.com>
Message-ID: <16668.57809.362313.36355@montanaro.dyndns.org>

(posting to python-dev separately from c.l.py so the thread can live or die
on its own in both places)

I responded on c.l.py to a couple proposals:

    Steven> decorate:
    Steven>     grammarrule('statement : expression')
    Steven>     versioninfo("Added in 2.4")
    Steven>     deprecated
    Steven>     typeinfo(None)
    Steven> def p_statement_expr(self, p):
    Steven>     print p[1]

    Nick>  as:
    Nick>      staticmethod
    Nick>      grammarrule('statement : expression')
    Nick>      version("Added in 2.4")
    Nick>      deprecatedmethod
    Nick>      type_(None)
    Nick>  def p_statement_expr(self, p):
    Nick>      print p[1]

with

    def p_statement_expr:
        staticmethod
        grammarrule('statement : expression')
        version("Added in 2.4")
        deprecatedmethod
        type_(None)
    decorate (self, p):
        """docstring here"""
        print p[1]

It seems different enough from other solutions that I thought it worth
tossing out.  I didn't see it mentioned in the PythonDecorators moin page.

Read it something like "define a function named p_statement_expr using a
bunch of functions to decorate the basic function".

It solves a couple problems:

1. "def" introduces the function definition instead of an arbitrary number
   of @-expressions.

2. There is no extra indentation of the main body.

3. The name of the function is known early on.

4. "def"/"decorate" pair up visually much the same as "try"/"except" or
   "if"/"then", though they don't represent alternative blocks of code to be
   executed.

On the minus side it introduces a vertical separation between the function
name and parameter list and introduces a new keyword, "decorate".

>From a parsing standpoint I think it will work.  You'll see either a colon
or a left paren after the function name to distinguish between the two types
of function definition.  I'm not sure if a token needs to be used to
separate the various decorator functions or if requiring a newline and
indentation is sufficient.

Skip
From rnd at onego.ru  Fri Aug 13 18:24:21 2004
From: rnd at onego.ru (Roman Suzi)
Date: Fri Aug 13 18:23:30 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <16668.57809.362313.36355@montanaro.dyndns.org>
References: <mailman.1506.1092237421.5135.python-list@python.org>
	<PcGdnQAV7olvoIfcRVn-ig@powergate.ca>
	<cfdh4q$kpm$1@nntp0.reith.bbc.co.uk>
	<d11dcfba.0408111433.43f025f8@posting.google.com>
	<slrnchnlm3.vq2.nick@irishsea.home.craig-wood.com>
	<16668.57809.362313.36355@montanaro.dyndns.org>
Message-ID: <Pine.LNX.4.58.0408132003160.3596@rnd.onego.ru>

On Fri, 13 Aug 2004, Skip Montanaro wrote:

>with
>
>    def p_statement_expr:
>        staticmethod
>        grammarrule('statement : expression')
>        version("Added in 2.4")
>        deprecatedmethod
>        type_(None)
>    decorate (self, p):
>        """docstring here"""
>        print p[1]

Bingo!

Just replace decorate with "from" and the nice syntax is found:

    def f:
        staticmethod
        grammarrule('statement : expression')
        version("Added in 2.4")
        deprecatedmethod
        type_(None)
    from self, p:
        """docstring here"""
        print p[1]

 + doesn't surprise Python programmer, because it is like try-except, etc
 + reads a natural language (with implicit "with"  after "f")
 + doesn't require any new keywords or symbols and "prefix" operators
 + is explicit about transformation
 + no additional indentation
 - grepping for defs now require more effort, so probably second name should
   be allowed after "from" or exactly this.

As a variant, docstring could be moved to the upper part.

I'd also liked being able to write:

   def f:
        staticmethod; grammarrule('statement : expression')
        version("Added in 2.4");  deprecatedmethod;  type_(None)
   from x, y:
      pass #...

and:

  def f: staticmethod
  from x, y: return x+y



>It seems different enough from other solutions that I thought it worth
>tossing out.  I didn't see it mentioned in the PythonDecorators moin page.
>
>Read it something like "define a function named p_statement_expr using a
>bunch of functions to decorate the basic function".
>
>It solves a couple problems:
>
>1. "def" introduces the function definition instead of an arbitrary number
>   of @-expressions.
>
>2. There is no extra indentation of the main body.
>
>3. The name of the function is known early on.
>
>4. "def"/"decorate" pair up visually much the same as "try"/"except" or
>   "if"/"then", though they don't represent alternative blocks of code to be
>   executed.
>
>On the minus side it introduces a vertical separation between the function
>name and parameter list and introduces a new keyword, "decorate".
>
>>From a parsing standpoint I think it will work.  You'll see either a colon
>or a left paren after the function name to distinguish between the two types
>of function definition.  I'm not sure if a token needs to be used to
>separate the various decorator functions or if requiring a newline and
>indentation is sufficient.
>
>Skip
>_______________________________________________
>Python-Dev mailing list
>Python-Dev@python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: http://mail.python.org/mailman/options/python-dev/rnd%40onego.ru
>

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From jcarlson at uci.edu  Fri Aug 13 18:54:14 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri Aug 13 19:01:13 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <Pine.LNX.4.58.0408132003160.3596@rnd.onego.ru>
References: <16668.57809.362313.36355@montanaro.dyndns.org>
	<Pine.LNX.4.58.0408132003160.3596@rnd.onego.ru>
Message-ID: <20040813093137.63C0.JCARLSON@uci.edu>

I've kept my nose out of the decorator discussion, but I thought I would
give my opinion on this one...


> On Fri, 13 Aug 2004, Skip Montanaro wrote:
> 
> >with
> >
> >    def p_statement_expr:
> >        staticmethod
> >        grammarrule('statement : expression')
> >        version("Added in 2.4")
> >        deprecatedmethod
> >        type_(None)
> >    decorate (self, p):
> >        """docstring here"""
> >        print p[1]
> 
> Bingo!

Oh god no.


> Just replace decorate with "from" and the nice syntax is found:
> 
>     def f:
>         staticmethod
>         grammarrule('statement : expression')
>         version("Added in 2.4")
>         deprecatedmethod
>         type_(None)
>     from self, p:
>         """docstring here"""
>         print p[1]

Gah, the horror.


>  + doesn't surprise Python programmer, because it is like try-except, etc

It surprises anyone who expects the function signature immediately after
the name.

>  + reads a natural language (with implicit "with"  after "f")

But it is so radically different from current definitions that it breaks
current understanding about the language.  It also requires people to
learn two significantly different semantics for defining a function. 
The good proposals (IMO) keep the function name and signature unchanged.

So let us read this "natural language" (my interpretation of the syntax
given above):

define f as a staticmethod... from the arguments self and p with
function body...  Not bad, though I don't like the visual aesthetics.

>  + doesn't require any new keywords or symbols and "prefix" operators

Prefix operators don't seem that bad to me, nor to Guido (who offered
them in the first place), nor to the dozen+ others who have stated that
"@, !, %, *, -, +, | or & is cool".

As for keywords, there has been an offered solution; the use of a future
import.  Similar to the way yield was used in 2.3.

From is an import semantic, not function definition.  Overloading 'from'
does remove the necessity of a new keyword, but so does the use of the
symbols previously offered.

>  + is explicit about transformation
>  + no additional indentation

Ahh, but you indent the block of decorators before the function
definition must /always/ be read in order to figure out . While this has
been done before (try, except, finally)

>  - grepping for defs now require more effort, so probably second name should
>    be allowed after "from" or exactly this.
> 
> As a variant, docstring could be moved to the upper part.
> 
> I'd also liked being able to write:
> 
>    def f:
>         staticmethod; grammarrule('statement : expression')
>         version("Added in 2.4");  deprecatedmethod;  type_(None)
>    from x, y:
>       pass #...
> 
> and:
> 
>   def f: staticmethod
>   from x, y: return x+y

Yikes.


If there were voting, I'd like to put my vote in for:

@deco
def fun(arg1):
    #body

or

def fun(arg1):
    @deco
    #body

...with a preference towards the second one, and a slight prefence
toward some other symbol than @ (for compatibility reasons).  |, &, ! are
all aesthetically pleasing to me, in that order.

Reading the second one:
define fun with arguments arg decorated with deco and body...

To me it reads better than the three syntaxes offered by Skip and Roman
this morning.

 - Josiah

From rnd at onego.ru  Fri Aug 13 19:04:34 2004
From: rnd at onego.ru (Roman Suzi)
Date: Fri Aug 13 19:03:42 2004
Subject: [Python-Dev] Re: Call for defense of @decorators
In-Reply-To: <411C56A5.8070404@iinet.net.au>
References: <200408100244.i7A2iLII007622@cosc353.cosc.canterbury.ac.nz>
	<1092108578.18208.10.camel@localhost>
	<1f7befae040809210679c2d4a0@mail.gmail.com>
	<87n0109eja.fsf@smtp.zope.com> <411C56A5.8070404@iinet.net.au>
Message-ID: <Pine.LNX.4.58.0408132102120.3596@rnd.onego.ru>

On Thu, 12 Aug 2004, Nick Coghlan wrote:

>    predef staticmethod
>    predef author(name='yarrB')
>    predef returns([int])
>    def plugh(self):
>         return 42
>
>
>To me, 'predef' seems to shout "Look, I'm telling you something about
>the 'def' that you are about to encounter in a line or two."

This is too much repetition to please programmers ;)
They like to write things ONCE. This is exactly the reason why
all this decorator hell raised:


def X:
   ...
X = decor(X)

uses X three times...


>Ah well, guess I'll go back to being a lurker on the deco issue. . .
>
>Cheers,
>Nick.
>
>

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From rnd at onego.ru  Fri Aug 13 19:27:23 2004
From: rnd at onego.ru (Roman Suzi)
Date: Fri Aug 13 19:26:33 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <20040813093137.63C0.JCARLSON@uci.edu>
References: <16668.57809.362313.36355@montanaro.dyndns.org>
	<Pine.LNX.4.58.0408132003160.3596@rnd.onego.ru>
	<20040813093137.63C0.JCARLSON@uci.edu>
Message-ID: <Pine.LNX.4.58.0408132108250.3596@rnd.onego.ru>

On Fri, 13 Aug 2004, Josiah Carlson wrote:

>I've kept my nose out of the decorator discussion, but I thought I would
>give my opinion on this one...

>> Bingo!
>Oh god no.

;)

>> Just replace decorate with "from" and the nice syntax is found:
>>
>>     def f:
>>         staticmethod
>>         grammarrule('statement : expression')
>>         version("Added in 2.4")
>>         deprecatedmethod
>>         type_(None)
>>     from self, p:
>>         """docstring here"""
>>         print p[1]
>
>Gah, the horror.
>
>>  + doesn't surprise Python programmer, because it is like try-except, etc
>
>It surprises anyone who expects the function signature immediately after
>the name.

I do disagree. In my suggestion function is being defined in natural order
hich is illustratable by use of lambda:


f =                               \
  decor(                          \
    lambda arg:                   \
      return_val)

Decorators could be destructive and change function completely, so it's
pitiful signature doesn't mean much.

>>  + reads a natural language (with implicit "with"  after "f")
>
>But it is so radically different from current definitions that it breaks
>current understanding about the language.

Why? IMHO @s are much more radical.

>It also requires people to
>learn two significantly different semantics for defining a function.

No more "significantly different" than learning

import m

and

from m import something

>The good proposals (IMO) keep the function name and signature unchanged.
>
>So let us read this "natural language" (my interpretation of the syntax
>given above):
>
>define f as a staticmethod... from the arguments self and p with
>function body...  Not bad, though I don't like the visual aesthetics.

Ok. Now I see our disagrement. I am thinking in terms of
applying decorators to make some other function FROM
a given function while you are reasoning about "adding" attributes to the
defined function.

>>  + doesn't require any new keywords or symbols and "prefix" operators
>
>Prefix operators don't seem that bad to me, nor to Guido (who offered
>them in the first place), nor to the dozen+ others who have stated that
>"@, !, %, *, -, +, | or & is cool".

(ooops! I mean "prefix statements" - statements sticking to
other statements, modifying their semantics)

>As for keywords, there has been an offered solution; the use of a future
>import.  Similar to the way yield was used in 2.3.

IMO, decorators deserve no keywords...

>>From is an import semantic, not function definition.  Overloading 'from'
>does remove the necessity of a new keyword, but so does the use of the
>symbols previously offered.

I see no problem overloading preposition like "from", "as", "to", etc,
if it adds natural readability to language statements.

>>  + is explicit about transformation
>>  + no additional indentation
>
>Ahh, but you indent the block of decorators before the function
>definition must /always/ be read in order to figure out . While this has
>been done before (try, except, finally)

I mean "no additional" level, like other group decoration proposals.

>>  - grepping for defs now require more effort, so probably second name should
>>    be allowed after "from" or exactly this.

However, I am not sure Guido will change his mind and replace
'@' with something else. Probably, Python became too boring without
fresh syntax solutions like pies.

If so, I'd added a possibility to use ";" to delimit decorators:

@ decor1; decor2
def f(x):
  ...

Or:

@decor1; @decor2
def f(x):
  ...

May double pie mean "repeat last time used pie":

@staticmethod
def m(self, a):
  ...

@@
def n(self, a):
  ...


Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From bac at OCF.Berkeley.EDU  Fri Aug 13 19:26:18 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug 13 19:26:49 2004
Subject: [Python-Dev] Re: Prothon on CPython intrepreter? [PROTHON]
In-Reply-To: <h6uqaavdlk6k.1xgy4tpg1nt2a.dlg@40tude.net>
References: <000a01c47f0b$b3cf4690$0b01a8c0@mark>
	<h6uqaavdlk6k.1xgy4tpg1nt2a.dlg@40tude.net>
Message-ID: <411CF9BA.1070102@ocf.berkeley.edu>

Mark Hahn wrote:
> On Tue, 10 Aug 2004 11:56:05 -0700, Mark Hahn wrote:
> 
> 
>>This CPython->.Net->CProthon roadmap will get Prothon out quickly and slowly
>>ease my wild ideas for the interpreter into Prothon.  It will also make
>>CPython a multiple language VM for the first time.
> 
> 
> I've gotten a cold reception to this idea of implementing Prothon on the 
> CPython interpreter.  Not only did Christian Tismer (and he should know) 
> and others warn me of the technnical challenge I was signing up for, but 
> there wasn't exactly a groundswell of enthusiasm for the idea overall.
> 

Sorry to hear about this.  Hopefully you can end up with a nice enough 
AST for Prothon that targetting other VMs won't be any more difficult 
than just writing walker for the AST.

-Brett
From ws-news at gmx.at  Fri Aug 13 19:52:48 2004
From: ws-news at gmx.at (Werner Schiendl)
Date: Fri Aug 13 19:52:59 2004
Subject: [Python-Dev] Re: def ... decorate
References: <mailman.1506.1092237421.5135.python-list@python.org><PcGdnQAV7olvoIfcRVn-ig@powergate.ca><cfdh4q$kpm$1@nntp0.reith.bbc.co.uk><d11dcfba.0408111433.43f025f8@posting.google.com><slrnchnlm3.vq2.nick@irishsea.home.craig-wood.com>
	<16668.57809.362313.36355@montanaro.dyndns.org>
Message-ID: <cfiv5m$fes$1@sea.gmane.org>

[Skip]
>
>     def p_statement_expr:
>         staticmethod
>         grammarrule('statement : expression')
>         version("Added in 2.4")
>         deprecatedmethod
>         type_(None)
>     decorate (self, p):
>         """docstring here"""
>         print p[1]
>

was the following suggested already? (don't think so):

     def p_statement_expr(self, p):
         staticmethod
         grammarrule('statement : expression')
         version("Added in 2.4")
         deprecatedmethod
         type_(None)
     body:
         """docstring here"""
         print p[1]


This would eliminate the problem of the moved parameters, yet keep all the
pros you listed.

The docstring could allowed to be with the decorators so that all
"meta-data" (from the perspective of a user) would be together.

/ Werner




From gmccaughan at synaptics-uk.com  Fri Aug 13 20:06:13 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Fri Aug 13 20:06:45 2004
Subject: [Python-Dev] Re: def ... decorate
In-Reply-To: <cfiv5m$fes$1@sea.gmane.org>
References: <mailman.1506.1092237421.5135.python-list@python.org>
	<16668.57809.362313.36355@montanaro.dyndns.org>
	<cfiv5m$fes$1@sea.gmane.org>
Message-ID: <200408131906.13149.gmccaughan@synaptics-uk.com>

On Friday 2004-08-13 18:52, Werner Schiendl wrote:
> [Skip]
> >
> >     def p_statement_expr:
> >         staticmethod
> >         grammarrule('statement : expression')
> >         version("Added in 2.4")
> >         deprecatedmethod
> >         type_(None)
> >     decorate (self, p):
> >         """docstring here"""
> >         print p[1]
> >
> 
> was the following suggested already? (don't think so):
> 
>      def p_statement_expr(self, p):
>          staticmethod
>          grammarrule('statement : expression')
>          version("Added in 2.4")
>          deprecatedmethod
>          type_(None)
>      body:
>          """docstring here"""
>          print p[1]

So you see "def name(args):" followed by what looks at
first glance like a function body ... and then it turns out
N lines later that it was really a set of decorators.
Surely if this sort of two-suite approach is going to be used,
there is only one sane way to do it (modulo choice of keyword):

    def p_statement_expr(self, p):
        """docstring goes here"""
        print p[1]
    decorated:
        staticmethod
        grammarrule('statement : expression')
        version("Added in 2.4")
        deprecatedmethod
        type_(None)

which keeps the arguments with the function name,
keeps the body right after the name and arguments, and
puts the decoration after the body which corresponds
with the order in which things actually happen (though
not necessarily the best order for understanding the
code).

Another merit to "post-position" decoration (which
applies here, where decoration is after the body,
and also to the original after-arglist proposal)
is that it's fairly obvious what order the decorators
are applied in: the two criteria are order of appearance
and order of proximity to the thing the decorators
seem visually to be attached to, and those agree.
For the @foo syntax they don't.

I actually quite like this. It seems more Pythonic
than the @foo proposal. Its obvious problem is that
it involves something that looks at first glance like
an ordinary suite of statements or expressions, but
whose interpretation is substantially different. At
least the @foo proposal avoids that.

I'm +1 on the original []-after-def proposal (which is
to be expected, since IIRC I was the first person to
propose it), +0.5 on what I've suggested above, +0 on
Guido's @foo, and a big hairy -1 on the weird distortions
of "def" I'm replying to :-).

-- 
g

From python at rcn.com  Fri Aug 13 20:11:06 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug 13 20:11:22 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <20040813093137.63C0.JCARLSON@uci.edu>
Message-ID: <004d01c48160$e7602d60$0bb0958d@oemcomputer>

> I've kept my nose out of the decorator discussion, but I thought I
would
> give my opinion on this one...
> 
> 
> > On Fri, 13 Aug 2004, Skip Montanaro wrote:
> >
> > >with
> > >
> > >    def p_statement_expr:
> > >        staticmethod
> > >        grammarrule('statement : expression')
> > >        version("Added in 2.4")
> > >        deprecatedmethod
> > >        type_(None)
> > >    decorate (self, p):
> > >        """docstring here"""
> > >        print p[1]
> >
> > Bingo!
> 
> Oh god no.
> 
> 
> > Just replace decorate with "from" and the nice syntax is found:
> >
> >     def f:
> >         staticmethod
> >         grammarrule('statement : expression')
> >         version("Added in 2.4")
> >         deprecatedmethod
> >         type_(None)
> >     from self, p:
> >         """docstring here"""
> >         print p[1]
> 
> Gah, the horror.

I think we should take maximum advantage of prior art.  Other languages
have proven the efficacy of backwards spellings.  Likewise, the reST
module has proven the utility of ASCII markup.

    dothemcitats::
    elurrammarg  ``statement : expression''::
    dohtemdetacerped::
    ``````````````````
    def f(arg):
         . . .



Advantages:

* Won't delude people into thinking they can understand a python program
just by reading it.

* Highly adaptable to electronic parsing while discouraging the adverse
trend towards human readability.

* Accomplishes the life saving core goal of moving decorators up a few
lines while only incurring the minor cost of complicating the snot out
of the language.

* Unlikely to be confused with any existing python constructs.

* Takes less effort to implement, document, and test than to read all of
the decorator syntax emails to date.


Alternative:

A further application of prior art would be to hypergeneralize the
solution to this problem and design a solution using XML.  This takes
maximum advantage of existing tools while providing a framework for all
potential future proposals to add a second or third way to accomplish
things we can already do now. 


citsacras-ly yours,


Raymond

From python at discworld.dyndns.org  Fri Aug 13 20:27:51 2004
From: python at discworld.dyndns.org (Charles Cazabon)
Date: Fri Aug 13 20:20:29 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <004d01c48160$e7602d60$0bb0958d@oemcomputer>
References: <20040813093137.63C0.JCARLSON@uci.edu>
	<004d01c48160$e7602d60$0bb0958d@oemcomputer>
Message-ID: <20040813182751.GB20947@discworld.dyndns.org>

Raymond Hettinger <python@rcn.com> wrote:
> > I've kept my nose out of the decorator discussion, but I thought I would
> > give my opinion on this one...
[...]
> Advantages:
> 
> * Won't delude people into thinking they can understand a python program
> just by reading it.

:)

I've lost track of all the decorator proposals.  Anyone proposed something
like this?

  def f { 'accepts' : (type1, type2), 'returns' : (int, str)} funcname (args):

?  The decorator dictionary could span multiple lines, of course.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python@discworld.dyndns.org>
GPL'ed software available at:     http://www.qcc.ca/~charlesc/software/
-----------------------------------------------------------------------
From pje at telecommunity.com  Fri Aug 13 20:26:14 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug 13 20:26:34 2004
Subject: [Python-Dev] @-after-def-before-colon (was Re: def ...
  decorate)
In-Reply-To: <cfiv5m$fes$1@sea.gmane.org>
References: <mailman.1506.1092237421.5135.python-list@python.org>
	<PcGdnQAV7olvoIfcRVn-ig@powergate.ca>
	<cfdh4q$kpm$1@nntp0.reith.bbc.co.uk>
	<d11dcfba.0408111433.43f025f8@posting.google.com>
	<slrnchnlm3.vq2.nick@irishsea.home.craig-wood.com>
	<16668.57809.362313.36355@montanaro.dyndns.org>
Message-ID: <5.1.1.6.0.20040813140145.026ddec0@mail.telecommunity.com>

At 07:52 PM 8/13/04 +0200, Werner Schiendl wrote:

>was the following suggested already? (don't think so):
>
>      def p_statement_expr(self, p):
>          staticmethod
>          grammarrule('statement : expression')
>          version("Added in 2.4")
>          deprecatedmethod
>          type_(None)
>      body:
>          """docstring here"""
>          print p[1]

hmmmm....  that's really not much different from:


    def p_statement_expr(self, p) [
        staticmethod,
        grammarrule('statement : expression'),
        version("Added in 2.4"),
        returns(None)
    ]:
        """Docstring"""
        body

is it?

But then, how about *this*:

    def p_statement_expr(self, p)
        @staticmethod
        @grammarrule('statement : expression')
        @version("Added in 2.4")
        @returns(None)
    as:
        """Docstring"""
        body

The decorators are much more visible with the @ signs.  Also, we could 
possibly allow the docstring to appear in the def statement, rather than in 
the body, e.g.:

    def p_statement_expr(self, p)
        @staticmethod
        @grammarrule('statement : expression')
        @version("Added in 2.4")
        @returns(None)
        """Docstring"""
    as:
        body



Some advantages:

* Tools that just only look at 'def funcname(signature)' still work

* Tools that only look for '@something' at the start of the line will work 
(e.g. I believe this means IPython will still work, dunno about Leo)

* Decorators are much more visible than in the list-after-def forms, but 
are still visibly part of the function definition

* You know the name/signature of the thing being defined before wading 
through decorators


Disadvantages:

* line continuation isn't standard; there ought to be brackets or 
parentheses, otherwise tokenizer will say there's an INDENT, and it's not 
standard to have an INDENT without a colon.

* How would we define the syntax?  Should it be something like this:

   classdef: 'class' NAME ['(' testlist ')'] [decorations] ':' suite
   funcdef: 'def' NAME parameters [decorations] ':' suite

   decorations: NEWLINE INDENT decorator* [docstring NEWLINE] DEDENT 'as'
   decorator: '@' test NEWLINE

In other words, should indent and one-decorator-per-line be enforced?

* Guido has argued that list-after-def is bad in the case of a long 
argument list, because of poor visibility.  Does this approach fix that?

From amk at amk.ca  Fri Aug 13 20:51:31 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Aug 13 20:52:08 2004
Subject: [Python-Dev] Checkin Privileges (was: Re: Third Bug Day outcome)
In-Reply-To: <loom.20040811T124517-239@post.gmane.org>
References: <20040809213820.GC11199@rogue.amk.ca>
	<2mllgn15cl.fsf@starship.python.net>
	<loom.20040811T124517-239@post.gmane.org>
Message-ID: <20040813185131.GB28569@rogue.amk.ca>

On Wed, Aug 11, 2004 at 12:17:27PM +0000, Johannes Gijsbers wrote:
> Consider this a request for checkin privileges.

Seconded.

--amk
From jcarlson at uci.edu  Fri Aug 13 21:02:22 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri Aug 13 21:09:14 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <20040813182751.GB20947@discworld.dyndns.org>
References: <004d01c48160$e7602d60$0bb0958d@oemcomputer>
	<20040813182751.GB20947@discworld.dyndns.org>
Message-ID: <20040813120041.63C6.JCARLSON@uci.edu>


> I've lost track of all the decorator proposals.  Anyone proposed something
> like this?
> 
>   def f { 'accepts' : (type1, type2), 'returns' : (int, str)} funcname (args):
> 
> ?  The decorator dictionary could span multiple lines, of course.

Basically yes.  See Ka-Ping's listing of options here:
http://mail.python.org/pipermail/python-dev/2004-March/043685.html


 - Josiah

From jcarlson at uci.edu  Fri Aug 13 21:06:35 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri Aug 13 21:13:13 2004
Subject: [Python-Dev] Re: def ... decorate
In-Reply-To: <200408131906.13149.gmccaughan@synaptics-uk.com>
References: <cfiv5m$fes$1@sea.gmane.org>
	<200408131906.13149.gmccaughan@synaptics-uk.com>
Message-ID: <20040813120226.63C9.JCARLSON@uci.edu>


On Fri, 13 Aug 2004 19:06:13 +0100
Gareth McCaughan <gmccaughan@synaptics-uk.com> wrote:

>     def p_statement_expr(self, p):
>         """docstring goes here"""
>         print p[1]
>     decorated:
>         staticmethod
>         grammarrule('statement : expression')
>         version("Added in 2.4")
>         deprecatedmethod
>         type_(None)
> 
> which keeps the arguments with the function name,
> keeps the body right after the name and arguments, and
> puts the decoration after the body which corresponds
> with the order in which things actually happen (though
> not necessarily the best order for understanding the
> code).

You seem to have missed the point of function decorations entirely.  We
already have what you offer in current Python syntax.  The point was to
move decorations to near/next to the function signature.

Read the PEP:
http://www.python.org/peps/pep-0318.html


> I actually quite like this. It seems more Pythonic
> than the @foo proposal. Its obvious problem is that
> it involves something that looks at first glance like
> an ordinary suite of statements or expressions, but
> whose interpretation is substantially different. At
> least the @foo proposal avoids that.

What you like is what has existed with Python since the beginning.

 - Josiah

From jcarlson at uci.edu  Fri Aug 13 21:13:55 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri Aug 13 21:20:14 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <Pine.LNX.4.58.0408132108250.3596@rnd.onego.ru>
References: <20040813093137.63C0.JCARLSON@uci.edu>
	<Pine.LNX.4.58.0408132108250.3596@rnd.onego.ru>
Message-ID: <20040813104200.63C3.JCARLSON@uci.edu>


On Fri, 13 Aug 2004 21:27:23 +0400 (MSD)
Roman Suzi <rnd@onego.ru> wrote:

> >>  + doesn't surprise Python programmer, because it is like try-except, etc
> >
> >It surprises anyone who expects the function signature immediately after
> >the name.
> 
> I do disagree. In my suggestion function is being defined in natural order
> hich is illustratable by use of lambda:
> 
> 
> f =                               \
>   decor(                          \
>     lambda arg:                   \
>       return_val)
> 
> Decorators could be destructive and change function completely, so it's
> pitiful signature doesn't mean much.

That is funny, because in the 100,000+ lines of Python I've written over
the years, function signatures are, in fact, important.  Knowing a
signature allows you to call a function, and I believe that any
decorator that destroys a signature willingly should be considered
broken.

One could make the claim that function signatures are at least as
important as the function body; if you don't know how to call a function,
then its body is nearly worthless.  Interestingly enough, decorators are
important to PyObjC developers because they use decorators to specify
signatures.

As for the "natural order" of definitions, in Python, the natural order
of definitions uses 'def', 'lambda' is considered by many (Guido
included) to be a mistake.  I believe your argument using lambda is poor.


> >>  + reads a natural language (with implicit "with"  after "f")
> >
> >But it is so radically different from current definitions that it breaks
> >current understanding about the language.
> 
> Why? IMHO @s are much more radical.

Seemingly in Guido's opinion, @s (or other symbols) were minimal.  You
can have a humble (or not so humble) opinion, but unless you can
convince Guido and/or the community, it is not going to happen (I've
been bitten by this myself).


> >It also requires people to
> >learn two significantly different semantics for defining a function.
> 
> No more "significantly different" than learning
> 
> import m
> 
> and
> 
> from m import something

While the use of 'from' implies something has changed in imports, and it
would imply something has changed in the decorator syntax, the use of
from differs.  Also, with your proposed decorator syntax, one inserts
'from' into the middle of the defintion of the function with decorators,
basically making 'from' into noise.


> >The good proposals (IMO) keep the function name and signature unchanged.
> >
> >So let us read this "natural language" (my interpretation of the syntax
> >given above):
> >
> >define f as a staticmethod... from the arguments self and p with
> >function body...  Not bad, though I don't like the visual aesthetics.
> 
> Ok. Now I see our disagrement. I am thinking in terms of
> applying decorators to make some other function FROM
> a given function while you are reasoning about "adding" attributes to the
> defined function.

No, our disagreement isn't about how we understand decorators.  I never
argue it is about adding attributes, as I have actually read the 1800+
postings on decorators in the last 8 months, where attributes are
basically stated as being not arguable.


Our disagreement is about what is a reasonable decorator syntax.  I
don't believe that anything that changes the look of a function
signature to be a reasonable decorator syntax.


> >>  + doesn't require any new keywords or symbols and "prefix" operators
> >
> >Prefix operators don't seem that bad to me, nor to Guido (who offered
> >them in the first place), nor to the dozen+ others who have stated that
> >"@, !, %, *, -, +, | or & is cool".
> 
> (ooops! I mean "prefix statements" - statements sticking to
> other statements, modifying their semantics)

What, like the following?

decorate(memoize)
def fun(arg):
    #body

I think the fact that the above is actually implemented suggests that it
makes sense for many people.  Some have even stated "Just stick it in
the standard library and call it good."  If it didn't rely on
implementation-specific CPython hooks, I believe that Guido would have
preferred it to '@'.


> >As for keywords, there has been an offered solution; the use of a future
> >import.  Similar to the way yield was used in 2.3.
> 
> IMO, decorators deserve no keywords...

I wouldn't say anything of the sort.  I don't believe keywords are
/necessary/ for decorators, but if a new keyword makes it click for
Guido (very unlikely), then so be it.


> >>From is an import semantic, not function definition.  Overloading 'from'
> >does remove the necessity of a new keyword, but so does the use of the
> >symbols previously offered.
> 
> I see no problem overloading preposition like "from", "as", "to", etc,
> if it adds natural readability to language statements.

Ahh, you seem to have missed that neither "as" nor "to" are keywords:

>>> def as(arg):
...     print "as", arg
...
>>> as(1)
as 1
>>> def to(arg):
...     print "to", arg
...
>>> to(1)
to 1


Personally, I have no issue with keyword reuse, as long as they make
sense.  Considering the logical separation of the use of from during
imports, and your use of from, I think it could confuse new users of the
language.


> >>  + is explicit about transformation
> >>  + no additional indentation
> >
> >Ahh, but you indent the block of decorators before the function
> >definition must /always/ be read in order to figure out . While this has
> >been done before (try, except, finally)
> 
> I mean "no additional" level, like other group decoration proposals.

Oops, I didn't finish my argument.

I know what you meant.  In order to understand what a function does, one
must read the decorators.  While associated indented blocks have been
done before (try, except, finally, else), one doesn't need to read the
try block to understand that if an exception were to occur, the except
block would be executed.  In order to understand the semantics of a
function definition, you need to read the decorators.  Having an
indented block previous to the function definition, which includes the
decorators for the function below, only makes sense if the name is given
before the decorators (the brain-stack argument).  However, since the
function signature is as important as any decorator, then the signature
should be prominant.

I believe your arguments are better focused toward a different syntax,
namely:

def fun(args):
    @deco
    #body

I think it beats your offered syntax hands-down due to its conciseness
and lack of need to include a separate indented block for decorations. 
It also doesn't require re-learning the meaning of "from".


> However, I am not sure Guido will change his mind and replace
> '@' with something else. Probably, Python became too boring without
> fresh syntax solutions like pies.

Guido has stated, quite clearly, that if we can come up with a better
symbol than "@", then he will go with it.


> If so, I'd added a possibility to use ";" to delimit decorators:

With the @ syntax, I believe it has been stated that multiple decorators
on one line is not to be supported.

 
> May double pie mean "repeat last time used pie":

Good for typing reduction, bad for understanding.  While a user can know
that the function is being decorated, the decorators are not immediately
available, nor are named.  Since we can name things, perhaps we should.
In this case, I believe copy+paste is preferable, or even named multiple
application of decorators.

def multi_decorate(*decorators):
    def decorate(function):
        a = decorators[:]
        while a:
            function = a.pop()(function)
        return function
    return decorate


my_deco = multi_decorate(staticmethod, returns(int))

@my_deco
def foo(args):
    #body

@my_deco
def goo(args):
    #body


 - Josiah

From skip at pobox.com  Fri Aug 13 21:51:37 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug 13 21:51:51 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <004d01c48160$e7602d60$0bb0958d@oemcomputer>
References: <20040813093137.63C0.JCARLSON@uci.edu>
	<004d01c48160$e7602d60$0bb0958d@oemcomputer>
Message-ID: <16669.7113.531528.462699@montanaro.dyndns.org>


I understand that not everyone will like every idea and that in this
particular arena it seems that no one idea is acceptable to everyone, but
can we return to my proposal and discuss its pros and cons?  I didn't pick
the example out of thin air.  I cast an existing example from c.l.py in this
form (thus the function name and somewhat unusual collection of decorator
names).  The only thing I added besides the syntax change was to identify
where I thought the docstring should go.

    def p_statement_expr:
        staticmethod
        grammarrule('statement : expression')
        version("Added in 2.4")
        deprecatedmethod
        type_(None)
    decorate (self, p):
        """docstring here"""
        print p[1]

Roman Suzi seemed to like it, given his "Bingo!" remark, but felt
immediately moved to alter it with no explanation.  Josiah replied to that
with "Oh god no".  It went downhill from there, to the point where Raymond
posted a sarcastic note about using reST and XML.  I don't know if Roman's
immediate modification implies his "Bingo!" was just lukewarm.  I have no
idea what Josiah was saying "Oh god no" to.

It seems to satisfy most obvious requirements:

    * It's clear from the start what function we are modifying, and you know
      early on that it's not your grandfather's Oldsmobile, err... function.

    * All decorators are listed up near the top (parseable I think using
      grammar bits requiring one decorator per line, similar to the current
      @-syntax).

    * It doesn't put the cart before the horse by introducing the decorators
      before the def.

    * If someone looks at it and thinks, "WTF!", it's readily apparent they
      should seek help with the "def" or "decorate" keywords.

    * It doesn't add an extra level of indentation to the function body.

    * It leaves '@' alone, implying that at some point in the future Guido
      (if he so chooses) proclaim that "'@' will never be used in Python at
      the start of a line, so it is free for external tools to use as a
      markup character".

    * It looks more like current Python block structure (at least to me)
      than the @-syntax proposal.

I understand that it has obvious weaknesses as well:

    * It separates the function name from its parameter list.  That does
      allow us to aboid parsing ambiguities though.

    * It introduces a new 'decorate' keyword, though I think it could sneak
      by as a pseudo-keyword like 'as'.

    * It can't readily be extended to class definitions, though the demand
      for that seems minor and mostly argued for symmetry reasons.

One or more of those weaknesses may well be a show stopper.  There may be
other pros and cons I'm missing.  That's fine, but can people please be
specific in their criticism?  I purposely posted twice, once here and once
on c.l.py, to avoid random smartass responses on c.l.py from leaking over to
python-dev.  Little did I realize I need not have worried about that.

Skip
From tim.peters at gmail.com  Fri Aug 13 21:58:28 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug 13 21:58:32 2004
Subject: [Python-Dev] 2.4 asyncore headache
Message-ID: <1f7befae040813125824acdb87@mail.gmail.com>

asyncore gives me a headache.  Windows sockets give me a headache. 
Zope's ZEO gives me a headache.  The ZEO test suite gives me
nightmares.

With that encouraging background, this ZEO test in ZODB 3.3 fails
every time when run on Windows under Python 2.4, but never under
Python 2.3.4:

checkReconnection
(ZEO.tests.testConnection.MappingStorageConnectionTests) ... Traceback
(most recent call last):
  File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\tests\zeoserver.py",
line 217, in ?
    main()
  File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\tests\zeoserver.py",
line 194, in main
    auth_realm=zo.auth_realm)
  File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\StorageServer.py",
line 738, in __init__
    factory=self.new_connection)
  File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\zrpc\server.py", line
34, in __init__
    self._open_socket()
  File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\zrpc\server.py", line
43, in _open_socket
    self.bind(self.addr)
  File "C:\Code\python\lib\asyncore.py", line 304, in bind
    return self.socket.bind(addr)
  File "<string>", line 1, in bind
socket.error: (10048, 'Address already in use')

unittest doesn't notice that failure, but the test hangs there for a
minute or so, and unittest does notice the subsequent:

Traceback (most recent call last):
  File "C:\Code\python\lib\unittest.py", line 260, in run
    testMethod()
  File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\tests\ConnectionTests.py",
line 458, in checkReconnection
    self.startServer(create=0)
  File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\tests\ConnectionTests.py",
line 202, in startServer
    zeoport, adminaddr, pid, path = forker.start_zeo_server(
  File "C:\Code\ZODB3.3\build\lib.win32-2.4\ZEO\tests\forker.py", line
149, in start_zeo_server
    s.connect(adminaddr)
  File "<string>", line 1, in connect
error: (10061, 'Connection refused')

failure.

Now somebody, for some reason, added this to 2.4's asyncore.py:

            if sys.platform == 'win32':
                reuse_constant = socket.SO_EXCLUSIVEADDRUSE
            else:
                reuse_constant = socket.SO_REUSEADDR

If I take away the new special case for Windows there, the
checkReconnection test works fine again.

Now I don't understand any of this.  Does anyone?  From the *looks* of
it, the error from bind() comes from trying to reuse an address, which
used to work fine, but no longer works at all on Windows -- at least
not in the way ZEO needs it to work.  ZEO calls set_reuse_addr()
because it needs to reuse ports quickly, and offhand it makes no sense
to me to change the meaing of set_reuse_addr() on Windows to something
that refuses to allow port reuse.
From walter at livinglogic.de  Fri Aug 13 22:04:04 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Fri Aug 13 22:04:09 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <411B6D79.2040503@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>
	<4107F0E8.3010005@egenix.com> <411920E4.9020404@livinglogic.de>
	<411B6D79.2040503@egenix.com>
Message-ID: <411D1EB4.10103@livinglogic.de>

M.-A. Lemburg wrote:

> Hi Walter,
> 
> I don't have time to comment on this this week, I'll respond
> next week.

OK.

> Overall, I don't like the idea of adding extra
> APIs breaking the existing codec API.

Adding a final argument that defaults to False doesn't
break the API for the callers, only for the implementors.

And if we drop the final argument, the API is completely
backwards compatible both for users and implementors.
The only thing that gets added is the feed() method,
that implementors don't have to overwrite.

> I believe that we can
> extend stream codecs to also work in a feed mode without
> breaking the API.

Abandoning the final argument and adding a feed() method
would IMHO be the simplest way to do this.

But then there's no way to make sure that every byte from
the input stream is really consumed.

Bye,
    Walter D?rwald


From amk at amk.ca  Fri Aug 13 22:03:47 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Aug 13 22:04:24 2004
Subject: [Python-Dev] 2.4 asyncore headache
In-Reply-To: <1f7befae040813125824acdb87@mail.gmail.com>
References: <1f7befae040813125824acdb87@mail.gmail.com>
Message-ID: <20040813200347.GB29067@rogue.amk.ca>

On Fri, Aug 13, 2004 at 03:58:28PM -0400, Tim Peters wrote:
> Now somebody, for some reason, added this to 2.4's asyncore.py:
>             if sys.platform == 'win32':
>                 reuse_constant = socket.SO_EXCLUSIVEADDRUSE

I'll back out this change.

--amk
From rnd at onego.ru  Fri Aug 13 22:21:03 2004
From: rnd at onego.ru (Roman Suzi)
Date: Fri Aug 13 22:20:13 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <20040813104200.63C3.JCARLSON@uci.edu>
References: <20040813093137.63C0.JCARLSON@uci.edu>
	<Pine.LNX.4.58.0408132108250.3596@rnd.onego.ru>
	<20040813104200.63C3.JCARLSON@uci.edu>
Message-ID: <Pine.LNX.4.58.0408132346540.3596@rnd.onego.ru>

On Fri, 13 Aug 2004, Josiah Carlson wrote:

>> f =                               \
>>   decor(                          \
>>     lambda arg:                   \
>>       return_val)
>>
>> Decorators could be destructive and change function completely, so it's
>> pitiful signature doesn't mean much.
>
>That is funny, because in the 100,000+ lines of Python I've written over
>the years, function signatures are, in fact, important.

Of course, they are! However, with a possibility to apply decorators
function signature could change!

>Knowing a
>signature allows you to call a function, and I believe that any
>decorator that destroys a signature willingly should be considered
>broken.

Hmmmm... It's up to discipline of a programmer...

>As for the "natural order" of definitions, in Python, the natural order
>of definitions uses 'def', 'lambda' is considered by many (Guido
>included) to be a mistake.  I believe your argument using lambda is poor.

def f (args): body
f = lambda args: body

Hmmm... slightly diferent order, indeed. But quite understandable,
because lambda creates anonymous function in an expression, while
def does namebinding.

I always thought the natural order of definition is value to define first,
definition to follow it.

lambda a mistake? lambda a mistake... -OX

>> >>  + reads a natural language (with implicit "with"  after "f")
>> >But it is so radically different from current definitions that it breaks
>> >current understanding about the language.
>>
>> Why? IMHO @s are much more radical.
>
>Seemingly in Guido's opinion, @s (or other symbols) were minimal.  You
>can have a humble (or not so humble) opinion, but unless you can
>convince Guido and/or the community, it is not going to happen (I've
>been bitten by this myself).

I am pretty sure @-syntax will do it into Python 2.4 and I am also
sure that nobody will remeber these discussions 1-2 years from now.

I wonder WHAT Python looked like if it's community discussed
EVERY syntax feature it had before 1.5.2. My feeling is Python
still missed IF-THEN-ELSE...

GvR already decided on the syntax. His brain has already
calculated all those millions of possibilities (moves),
community response included. We have almost zero chance to
find better syntax for decorators.

>> >It also requires people to
>> >learn two significantly different semantics for defining a function.
>>
>> >define f as a staticmethod... from the arguments self and p with
>> >function body...  Not bad, though I don't like the visual aesthetics.
>>
>> Ok. Now I see our disagrement. I am thinking in terms of
>> applying decorators to make some other function FROM
>> a given function while you are reasoning about "adding" attributes to the
>> defined function.
>
>No, our disagreement isn't about how we understand decorators.  I never
>argue it is about adding attributes, as I have actually read the 1800+

I hope you do not receive treatment at mental hospital right now? ;-)
(Disclaimer: no pun intended. Just expressing my surprise)

>postings on decorators in the last 8 months, where attributes are
>basically stated as being not arguable.

>Our disagreement is about what is a reasonable decorator syntax.  I
>don't believe that anything that changes the look of a function
>signature to be a reasonable decorator syntax.

In fact, I've already proposed 2 syntaxes: C4 and J3.

But aren't decorators part of a signature? Are there any languages with
decorators similar to being implemented in Python?
(Full-fledged, not things like "static volatile ..." in C/Java/C++ like
languages.)

Aren't decorators functional programming everyday feature, when
(in some programming languages) it is normal to define function as
a superposition of simpler ones?

[ neither "as" nor "to" are keywords ] ok

>> I mean "no additional" level, like other group decoration proposals.
>
>Oops, I didn't finish my argument.
>
>I know what you meant.  In order to understand what a function does, one
>must read the decorators.  While associated indented blocks have been
>done before (try, except, finally, else), one doesn't need to read the
>try block to understand that if an exception were to occur, the except
>block would be executed.  In order to understand the semantics of a
>function definition, you need to read the decorators.  Having an
>indented block previous to the function definition, which includes the
>decorators for the function below, only makes sense if the name is given
>before the decorators (the brain-stack argument).

> However, since the
>function signature is as important as any decorator, then the signature
>should be prominant.


>I believe your arguments are better focused toward a different syntax,
>namely:
>
>def fun(args):
>    @deco
>    #body
>
>I think it beats your offered syntax hands-down due to its conciseness
>and lack of need to include a separate indented block for decorations.
>It also doesn't require re-learning the meaning of "from".

>> However, I am not sure Guido will change his mind and replace
>> '@' with something else. Probably, Python became too boring without
>> fresh syntax solutions like pies.
>
>Guido has stated, quite clearly, that if we can come up with a better
>symbol than "@", then he will go with it.

The ord "better" is subjective. In this case it means "better"
in Guido's view. Probably, we can even write a syntax-proposing
program which could give thousands of variants (and probably even
validate them with existing Python grammar to exclude clashes)
from which Guido could choose. Wiki already has enough material to
do this sort of machine.



Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From tim.peters at gmail.com  Fri Aug 13 22:25:07 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug 13 22:25:12 2004
Subject: [Python-Dev] 2.4 asyncore headache
In-Reply-To: <20040813200347.GB29067@rogue.amk.ca>
References: <1f7befae040813125824acdb87@mail.gmail.com>
	<20040813200347.GB29067@rogue.amk.ca>
Message-ID: <1f7befae04081313252cb3304a@mail.gmail.com>

[Tim]
>> Now somebody, for some reason, added this to 2.4's asyncore.py:
>>             if sys.platform == 'win32':
>>                 reuse_constant = socket.SO_EXCLUSIVEADDRUSE

[Andrew]
> I'll back out this change.

But only if that's the right thing to do <0.9 wink>.  I gave up
reading the Windows SO_EXCLUSIVEADDRUSE docs after it got to the "This
situation can be quite complicated; even though the socket has been
closed, the underlying transport may not terminate its connection ..."
part.  But by that point, it sure didn't seem like SO_EXCLUSIVEADDRUSE
was a plausible Windowsy way to spell the Unixy SO_REUSEADDR ...
From jack at performancedrivers.com  Fri Aug 13 23:31:00 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Fri Aug 13 23:31:04 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <16669.7113.531528.462699@montanaro.dyndns.org>
References: <20040813093137.63C0.JCARLSON@uci.edu>
	<004d01c48160$e7602d60$0bb0958d@oemcomputer>
	<16669.7113.531528.462699@montanaro.dyndns.org>
Message-ID: <20040813213100.GQ23725@performancedrivers.com>

On Fri, Aug 13, 2004 at 02:51:37PM -0500, Skip Montanaro wrote:
> 
> I understand that not everyone will like every idea and that in this
> particular arena it seems that no one idea is acceptable to everyone, but
> can we return to my proposal and discuss its pros and cons?  I didn't pick
> the example out of thin air.  I cast an existing example from c.l.py in this
> form (thus the function name and somewhat unusual collection of decorator
> names).  The only thing I added besides the syntax change was to identify
> where I thought the docstring should go.
> 
>     def p_statement_expr:
>         staticmethod
>         grammarrule('statement : expression')
>         version("Added in 2.4")
>         deprecatedmethod
>         type_(None)
>     decorate (self, p):
>         """docstring here"""
>         print p[1]
>
> I understand that it has obvious weaknesses as well:
> 
>     * It separates the function name from its parameter list.  That does
>       allow us to aboid parsing ambiguities though.
This is huge, it would mean two ways to do it.  When I want to decorate the
function with '@'  I add an extra line above the existing definition.
With this suggestion I have to pick the line apart and shuffle stuff around.
Guido has said being able to add/remove a decorator should be as simple as
cut/pasting a line and I agree.

>     * It can't readily be extended to class definitions, though the demand
>       for that seems minor and mostly argued for symmetry reasons.
Class decorators are pretty important to me.  I would replace most of my meta
class usages with decorators.  I care enough I spent two days writing a patch
and fishing through the bowels of compile.c (*plug plug*).

* It uses a suite in a way different from all other suites.

A group of points
* it breaks grepability badly
* it breaks existing tools by radically changing the grammar
* breaking machine tools also means breaking meat tools.  I now have to fit
  two ways of recognizing functions in my head all the time.  This is the
  showstopper of all showstoppers right there.

-Jack
From rnd at onego.ru  Sat Aug 14 00:05:08 2004
From: rnd at onego.ru (Roman Suzi)
Date: Sat Aug 14 00:04:29 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <16669.7113.531528.462699@montanaro.dyndns.org>
References: <20040813093137.63C0.JCARLSON@uci.edu>       
	<004d01c48160$e7602d60$0bb0958d@oemcomputer>
	<16669.7113.531528.462699@montanaro.dyndns.org>
Message-ID: <Pine.LNX.4.58.0408140143490.31795@rnd.onego.ru>

On Fri, 13 Aug 2004, Skip Montanaro wrote:

>I understand that not everyone will like every idea and that in this
>particular arena it seems that no one idea is acceptable to everyone, but
>can we return to my proposal and discuss its pros and cons?  I didn't pick
>the example out of thin air.  I cast an existing example from c.l.py in this
>form (thus the function name and somewhat unusual collection of decorator
>names).  The only thing I added besides the syntax change was to identify
>where I thought the docstring should go.
>
>    def p_statement_expr:
>        staticmethod
>        grammarrule('statement : expression')
>        version("Added in 2.4")
>        deprecatedmethod
>        type_(None)
>    decorate (self, p):
>        """docstring here"""
>        print p[1]
>
>Roman Suzi seemed to like it, given his "Bingo!" remark, but felt
>immediately moved to alter it with no explanation.  Josiah replied to that
>with "Oh god no".  It went downhill from there, to the point where Raymond
>posted a sarcastic note about using reST and XML.  I don't know if Roman's
>immediate modification implies his "Bingo!" was just lukewarm.

Well, your example lead me to my syntax and I liked it very much
at the moment.

> I have no
>idea what Josiah was saying "Oh god no" to.

Now it seems less wonderful to me, because I forgot about
decorating classes.

class lalalal:
   decor
from (A,B):
   @definition

doesn't have properties I'd liked to see. Same applies to Skip's
syntax.


>It seems to satisfy most obvious requirements:
>
>    * It's clear from the start what function we are modifying, and you know
>      early on that it's not your grandfather's Oldsmobile, err... function.
>
>    * All decorators are listed up near the top (parseable I think using
>      grammar bits requiring one decorator per line, similar to the current
>      @-syntax).
>
>    * It doesn't put the cart before the horse by introducing the decorators
>      before the def.

I agree with this.


>    * If someone looks at it and thinks, "WTF!", it's readily apparent they
>      should seek help with the "def" or "decorate" keywords.

Ummm.... Me thinks it's much more important to facilitate
those who write lots of code than those who looks at it
sheepishly (sp?)

>    * It doesn't add an extra level of indentation to the function body.
>
>    * It leaves '@' alone, implying that at some point in the future Guido
>      (if he so chooses) proclaim that "'@' will never be used in Python at
>      the start of a line, so it is free for external tools to use as a
>      markup character".

>    * It looks more like current Python block structure (at least to me)
>      than the @-syntax proposal.
>
>I understand that it has obvious weaknesses as well:
>
>    * It separates the function name from its parameter list.  That does
>      allow us to aboid parsing ambiguities though.
>
>    * It introduces a new 'decorate' keyword, though I think it could sneak
>      by as a pseudo-keyword like 'as'.

... or no keyword at alll, as some suggested.

>    * It can't readily be extended to class definitions, though the demand
>      for that seems minor and mostly argued for symmetry reasons.
>
>One or more of those weaknesses may well be a show stopper.  There may be
>other pros and cons I'm missing.  That's fine, but can people please be
>specific in their criticism?  I purposely posted twice, once here and once
>on c.l.py, to avoid random smartass responses on c.l.py from leaking over to
>python-dev.  Little did I realize I need not have worried about that.

IMO, "decorate" is too wordy. "dec" is associated with "decimal", so
your proposal has no chance to fly :-( And mine too as Josiah
showed above.

This @discussion reminds me of inline-if one. With the difference
that at that time GvR did not favor ANY syntax and now he favors
the @syntax. Lets tell to themselves that there are no better
choices than @. Amen.

>Skip

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From jcarlson at uci.edu  Sat Aug 14 00:12:14 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sat Aug 14 00:18:20 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <16669.7113.531528.462699@montanaro.dyndns.org>
References: <004d01c48160$e7602d60$0bb0958d@oemcomputer>
	<16669.7113.531528.462699@montanaro.dyndns.org>
Message-ID: <20040813132721.63CC.JCARLSON@uci.edu>

> I understand that not everyone will like every idea and that in this
> particular arena it seems that no one idea is acceptable to everyone, but
> can we return to my proposal and discuss its pros and cons?  I didn't pick
> the example out of thin air.  I cast an existing example from c.l.py in this
> form (thus the function name and somewhat unusual collection of decorator
> names).  The only thing I added besides the syntax change was to identify
> where I thought the docstring should go.
> 
>     def p_statement_expr:
>         staticmethod
>         grammarrule('statement : expression')
>         version("Added in 2.4")
>         deprecatedmethod
>         type_(None)
>     decorate (self, p):
>         """docstring here"""
>         print p[1]
> 
> Roman Suzi seemed to like it, given his "Bingo!" remark, but felt
> immediately moved to alter it with no explanation.  Josiah replied to that
> with "Oh god no".  It went downhill from there, to the point where Raymond
> posted a sarcastic note about using reST and XML.  I don't know if Roman's
> immediate modification implies his "Bingo!" was just lukewarm.  I have no
> idea what Josiah was saying "Oh god no" to.

Mostly the separating the function name from the arguments.


>     * It doesn't put the cart before the horse by introducing the decorators
>       before the def.

Great.

>     * It doesn't add an extra level of indentation to the function body.

Great.

>     * It leaves '@' alone, implying that at some point in the future Guido
>       (if he so chooses) proclaim that "'@' will never be used in Python at
>       the start of a line, so it is free for external tools to use as a
>       markup character".

!|&-+* also leave the '@' alone.  I think that if it werent for '|'
being mistakable for I,l or 1, it would already have been chosen.


>     * It looks more like current Python block structure (at least to me)
>       than the @-syntax proposal.

Check my P.S. at the end.


> I understand that it has obvious weaknesses as well:
> 
>     * It separates the function name from its parameter list.  That does
>       allow us to avoid parsing ambiguities though.

Separating the argument list from the function name is enough to kill it
for me.


>     * It can't readily be extended to class definitions, though the demand
>       for that seems minor and mostly argued for symmetry reasons.

Why can't it be extended?

class decorated_class:
    decorators
decorate:  #or (subcls1, subcls2) before the colon with subclasses
    "docstring"
    body


Personally, I don't desire decorators for classes for symmetry reasons,
I desire it to replace metaclasses, at least in those things that I
could conceivably want metaclasses to do.


> One or more of those weaknesses may well be a show stopper.  There may be
> other pros and cons I'm missing.  That's fine, but can people please be
> specific in their criticism?  I purposely posted twice, once here and once
> on c.l.py, to avoid random smartass responses on c.l.py from leaking over to
> python-dev.  Little did I realize I need not have worried about that.

My major issue is that I think we should be making a small change to the
language.  I believe that any change that could be considered a large
change should be avoided.

Introducing an additional block structure to specify decorators seems to
be a large change.

Separating the name of a function from the arguments seems to be a large
change.

Other proposals attempt to indent function definitions inside decorators,
which is also a large change (which also destroy readability).


Let us try to keep the change small.  Sound reasonable?

 - Josiah

P.S.
One could make the claim that assigning directly to __slots__ and
__metaclass__ are a bit of magic that could have been avoided with
proper decorators (or even that metaclasses could have been avoided with
decorators).

@__metaclass__(metaclass)
@__slots__("slot1", "slot2")
class classname:
    #etc.

Heck, it almost looks like what we already have in Python when we do the
following...

class classname(object):
    @__metaclass__(metaclass)
    #or @metaclass_like_decorator
    @__slots__("slot1", "slot2")
    @__doc__("docstring")
    #etc.


Goodness, there's a syntax to rule them all.  Not only does it kind of
look like what we already have, but it is further extendable with other
__name__ decorators.

Choose any @ replacement if desireable.

From bob at redivi.com  Sat Aug 14 00:27:37 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sat Aug 14 00:27:40 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <20040813132721.63CC.JCARLSON@uci.edu>
References: <004d01c48160$e7602d60$0bb0958d@oemcomputer>
	<16669.7113.531528.462699@montanaro.dyndns.org>
	<20040813132721.63CC.JCARLSON@uci.edu>
Message-ID: <FAEDDF94-ED77-11D8-AF4E-000A95686CD8@redivi.com>

On Aug 13, 2004, at 6:12 PM, Josiah Carlson wrote:

> Personally, I don't desire decorators for classes for symmetry reasons,
> I desire it to replace metaclasses, at least in those things that I
> could conceivably want metaclasses to do.

In some cases, what you actually want ARE metaclasses.  I wouldn't say 
get rid of them, because I use them in cases where what I actually want 
ARE metaclasses, but I do concede that many of the (ab)uses of 
metaclasses could be replaced by decorators (assuming that the 
decorator syntax applied to classes as well).

-bob
From jcarlson at uci.edu  Sat Aug 14 00:53:06 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sat Aug 14 00:59:21 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <Pine.LNX.4.58.0408132346540.3596@rnd.onego.ru>
References: <20040813104200.63C3.JCARLSON@uci.edu>
	<Pine.LNX.4.58.0408132346540.3596@rnd.onego.ru>
Message-ID: <20040813135102.63CF.JCARLSON@uci.edu>

> >> Decorators could be destructive and change function completely, so it's
> >> pitiful signature doesn't mean much.
> >
> >That is funny, because in the 100,000+ lines of Python I've written over
> >the years, function signatures are, in fact, important.
> 
> Of course, they are! However, with a possibility to apply decorators
> function signature could change!

You must be joking.  Just because a signature can change doesn't mean
that the signature is not important.

I'll say it again, just in case you missed it the first time, the first
major use of decorators are going to be in PyObjC, WHICH USES DECORATORS
TO SPECIFY SIGNATURES.

If you continue to claim that signatures are unimportant, I can't
imagine how this discussion could continue in any sort of a productive
fashion.


> >Knowing a
> >signature allows you to call a function, and I believe that any
> >decorator that destroys a signature willingly should be considered
> >broken.
> 
> Hmmmm... It's up to discipline of a programmer...

How does the discipline of a programmer have anything to do with the use
of signature-changing decorators?  When I (and I imagine most everyone
else) define a function or class, its signature tells me almost
everything I need to know about them.  What you are suggesting is that
this method of using signatures to describe functionality should not be
relied upon, because signatures can change.  That's like saying that I
shouldn't rely on the amount of money in my checking account, because
that amount changes.


> >As for the "natural order" of definitions, in Python, the natural order
> >of definitions uses 'def', 'lambda' is considered by many (Guido
> >included) to be a mistake.  I believe your argument using lambda is poor.
> 
> def f (args): body
> f = lambda args: body
> 
> Hmmm... slightly diferent order, indeed. But quite understandable,
> because lambda creates anonymous function in an expression, while
> def does namebinding.

You don't need to explain what it does.

Generally, describing decoration in terms of a functional approach
is pointless, we already have a functional approach to decorations,
which is what we are trying to remove.

def f(args):
    body
f = decorate(f)

Also, if we were to give a language description of the lambda form you
describe, it would be:
Define an anonymous function with arguments args and body bdy,
decorating it with decorate, and naming it name.

Ick.

I much prefer: define a function called name, with arguments args,
decorated by decorator, with body bdy.  Which surprisingly enough looks
like...

def fun(args):
    @decorator
    #body

Again, replace @ if desired.

> I always thought the natural order of definition is value to define first,
> definition to follow it.

I don't know where this came from, but it is what we already have.
def fun(args):
    body


> lambda a mistake? lambda a mistake... -OX

Yes, lambda a mistake:
http://www.linuxjournal.com/article.php?sid=2959


> >Seemingly in Guido's opinion, @s (or other symbols) were minimal.  You
> >can have a humble (or not so humble) opinion, but unless you can
> >convince Guido and/or the community, it is not going to happen (I've
> >been bitten by this myself).
> 
> I am pretty sure @-syntax will do it into Python 2.4 and I am also
> sure that nobody will remeber these discussions 1-2 years from now.

If you are so sure, why continue arguing towards your "J3" syntax (which
is a variation on the syntax Skip proposed)?

> I wonder WHAT Python looked like if it's community discussed
> EVERY syntax feature it had before 1.5.2. My feeling is Python
> still missed IF-THEN-ELSE...

You are moving off-topic.

> GvR already decided on the syntax. His brain has already
> calculated all those millions of possibilities (moves),
> community response included. We have almost zero chance to
> find better syntax for decorators.

Considering that you were, earlier today, offering a different syntax,
this seems quite out of character for you.


> >No, our disagreement isn't about how we understand decorators.  I never
> >argue it is about adding attributes, as I have actually read the 1800+
> 
> I hope you do not receive treatment at mental hospital right now? ;-)
> (Disclaimer: no pun intended. Just expressing my surprise)

No, I'm working on a PhD in computer science theory.  A few thousand
python-dev messages are just personal research into my favorite
programming language.

> >postings on decorators in the last 8 months, where attributes are
> >basically stated as being not arguable.
> 
> >Our disagreement is about what is a reasonable decorator syntax.  I
> >don't believe that anything that changes the look of a function
> >signature to be a reasonable decorator syntax.
> 
> In fact, I've already proposed 2 syntaxes: C4 and J3.

C4 is a derivative of what has been known since before March.
J3 has serious problems, as stated by Jack and myself.

> But aren't decorators part of a signature? Are there any languages with
> decorators similar to being implemented in Python?
> (Full-fledged, not things like "static volatile ..." in C/Java/C++ like
> languages.)

Decorators are only part of the signature if you want them to be.  I
believe the vast majority of decorators will not change function
signatures.

I am not aware of decorators in other languages, but I have kept my nose
out of most langauges developed since I discovered Python in 2000.


> Aren't decorators functional programming everyday feature, when
> (in some programming languages) it is normal to define function as
> a superposition of simpler ones?

That is like saying that all programming is functional.  Is prolog a
functional programming language?  No, it is a logic programming language,
yet it is common in Prolog to see the following:

my_and(a, b) = my_bool(a) and my_bool(b)


> >Guido has stated, quite clearly, that if we can come up with a better
> >symbol than "@", then he will go with it.
> 
> The ord "better" is subjective. In this case it means "better"
> in Guido's view. Probably, we can even write a syntax-proposing
> program which could give thousands of variants (and probably even
> validate them with existing Python grammar to exclude clashes)
> from which Guido could choose. Wiki already has enough material to
> do this sort of machine.

So then, it seems to me like the community should take a vote on what
symbol (if any) is better than @, and commit to sticking behind that one. 
If Guido likes it, great, we have a symbol; then we can argue about any
slight variation of Guido's preferred syntax (if he is willing to listen),
and call it good.

 - Josiah

From zathras at thwackety.com  Sat Aug 14 01:31:24 2004
From: zathras at thwackety.com (Michael Sparks)
Date: Sat Aug 14 01:11:16 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def),
	with implementation
Message-ID: <Pine.LNX.4.44.0408140006060.26777-100000@pingu.thwackety.com>

Hi,


I suspect people are getting sick of the decorator discussion by now, so
I'll try to be brief, and useful.

Syntax option J2 from the wiki seems to have provoked an interesting
discussion[1] at least by a small subset of c.l.p, so I wondered how
difficult it would be to implement - this turned out to be relatively
simple [2].
   [1] http://mail.python.org/pipermail/python-list/2004-August/233413.html
   [2] http://mail.python.org/pipermail/python-list/2004-August/233591.html
       (implementation)

Example of the syntax... (I'm sure it's familiar...)

#Syntax J2 from http://www.python.org/moin/PythonDecorators
#Some random comment
#More random comment
decorate:
     staticmethod
     grammarrule('statement : expression')
     versioninfo("Added in 2.4")
     deprecated
     typeinfo(None)
def p_statement_expr(self, p):
     print p[1]

For comparison, this is intended as equivalent to:

#Current syntax in 2.4a2
#Some random comment
#More random comment
@staticmethod
@grammarrule('statement : expression')
@versioninfo("Added in 2.4")
@deprecated
@typeinfo(None)
def p_statement_expr(self, p):
     print p[1]

With the exact semantics and restrictions remaining the same - this is
just syntactic sugar.


This was my first time looking at the python source to see how easy it
would be to implement - so it's not perfect, and probably far from ideal.
My reasons I looking and giving it a go though were:

   * To me it looks much clearer - you can instantly see where the method
     (and decorator block) starts.
   * Lack of repetition of the decoration hint
   * It hints to a non-python programmer very clearly that this is
     something different. If they're aware of the decorator patterm,
     they will realise (perhaps) what it means, and if they don't they
     have something to google for.

I have read most of the arguments against this syntax, and if the decision
ends up to stay with @ or take a different syntax, I'm sure it'll work out
fine :) (This isn't my favourite syntax, my favourite has a keyword inside
the function suite, which I'm well aware is dead in the water!)

The post I made to c.l.p with a first attempt at an implementation (has
minor issues regarding scopes, but does work) can be found here:

   * http://mail.python.org/pipermail/python-list/2004-August/233591.html

If there is interest in this syntax, then I suspect/hope making it
suitable for a "proper" patch would be relatively simple.

In case people are wondering what the grammar change I made for this is,
I'll include here:

-decorator: '@' dotted_name [ '(' [arglist] ')' ]
+decorator: dotted_name [ '(' [arglist] ')' ]
 decorators: decorator ([NEWLINE] decorator)* NEWLINE
-funcdef: [decorators] 'def' NAME parameters ':' suite
+funcdef: ['decorate' ':' NEWLINE INDENT decorators DEDENT ] 'def' NAME parameters ':' suite
 parameters: '(' [varargslist] ')'

Downsides here:
   * Looks like a suite, implying it allows arbitrary statements, but
     isn't
   * Introduces a new keyword
   * Some people have expressed reservations about the syntax, some of
     which also apply to the @pie syntax, and some which don't.
   * The patch above has scope issues, which means currently it requires
     some cruft to force functions into scope.

Biggest upside from a personal perspective though is this: (aside from
others)
   * Forces things like the common cases of classmethod and staticmethod
     to jump out at a maintainer.

     As the wiki mentions, this two line syntax is relatively heavyweight
     for these cases, and I agree. I disagree about this being  bad -
     static methods and class methods are rare.

     Having a BIG indication they exist when you have (say) 20 methods in
     a class strikes me as a very positive thing, and is surely 90% of
     the reason for moving from the existing mechanism of:

        foo = staticmethod(foo)

Example tested:

class Foo:
   staticmethod # This is cruft to force staticmethod into scope

   decorate:
      staticmethod
   def hello(who):
      print "woo?", who

Anyway, hope this is considered useful, rather than extending the
headache! (However, I am for one very happy  that decorators are now
available in something clearer than we had in pre(2.4) and it certainly
made implementing the syntax/semantics above relatively trivial :)

Regards,


Michael.
--
"The 'nice' thing about decorator syntaxes is there are so many to choose
 from."

From bob at redivi.com  Sat Aug 14 01:24:24 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sat Aug 14 01:24:26 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def),
	with implementation
In-Reply-To: <Pine.LNX.4.44.0408140006060.26777-100000@pingu.thwackety.com>
References: <Pine.LNX.4.44.0408140006060.26777-100000@pingu.thwackety.com>
Message-ID: <E9AED893-ED7F-11D8-82BD-000A95686CD8@redivi.com>


On Aug 13, 2004, at 7:31 PM, Michael Sparks wrote:

> Syntax option J2 from the wiki seems to have provoked an interesting
> discussion[1] at least by a small subset of c.l.p, so I wondered how
> difficult it would be to implement - this turned out to be relatively
> simple [2].
>    [1] 
> http://mail.python.org/pipermail/python-list/2004-August/233413.html
>    [2] 
> http://mail.python.org/pipermail/python-list/2004-August/233591.html
>        (implementation)
>
> #Syntax J2 from http://www.python.org/moin/PythonDecorators
> decorate:
>      staticmethod
>      grammarrule('statement : expression')
>      versioninfo("Added in 2.4")
>      deprecated
>      typeinfo(None)
> def p_statement_expr(self, p):
>      print p[1]

My only problem with this syntax is that I expect the common cases 
(mine, anyway) to be zero or one decorators per function, so the extra 
block and indent seems a bit excessive compared to the current 
@prefix-symbol-decorator proposal(s).  It is, of course, far better 
than typing the function name three times though! :)

-0

-bob
From ncoghlan at iinet.net.au  Sat Aug 14 01:30:47 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Aug 14 01:30:55 2004
Subject: [Python-Dev] 2.4 asyncore headache
In-Reply-To: <1f7befae04081313252cb3304a@mail.gmail.com>
References: <1f7befae040813125824acdb87@mail.gmail.com>	<20040813200347.GB29067@rogue.amk.ca>
	<1f7befae04081313252cb3304a@mail.gmail.com>
Message-ID: <411D4F27.3030605@iinet.net.au>

Tim Peters wrote:
> [Tim]
> 
>>>Now somebody, for some reason, added this to 2.4's asyncore.py:
>>>            if sys.platform == 'win32':
>>>                reuse_constant = socket.SO_EXCLUSIVEADDRUSE
> 
> 
> [Andrew]
> 
>>I'll back out this change.
> 
> 
> But only if that's the right thing to do <0.9 wink>.  I gave up
> reading the Windows SO_EXCLUSIVEADDRUSE docs after it got to the "This
> situation can be quite complicated; even though the socket has been
> closed, the underlying transport may not terminate its connection ..."
> part.  But by that point, it sure didn't seem like SO_EXCLUSIVEADDRUSE
> was a plausible Windowsy way to spell the Unixy SO_REUSEADDR ...

Is test_socketserver currently passing for you?

A couple of days ago I ran the test suite with -unetwork, and that test 
failed, but I haven't got around to looking into it yet. Could this be 
related?

[rambling discussion of what I found out about this so far follows]

Anyway, after some hunting, the difference between Windows and Unix 
SO_REUSEADDR, is that on Unix, this only allows local address reuse when 
the existing socket is in a TIME_WAIT state. On Windows, it allows local 
address reuse _anytime_.

See http://hea-www.harvard.edu/~fine/Tech/addrinuse.html. Look for the 
phrase 'security stigma', then read the bit starting with 'On some 
operating systems. . .'

The issue is that if we *don't* set SO_REUSEADDR on Windows, then you 
get into problems with getting EADDRINUSE when you shutdown and restart 
a TCP/IP server on the same local address (generally if the socket has 
gone into TIME_WAIT while shutting down).

Looking at MSDN, SO_EXCLUSIVEADDRUSE is a way to say 'make my socket 
behave like a UNIX socket without SO_REUSEADDR specified'. That is, with 
no options specified, a Windows socket is vulnerable to having another 
process come along, specify SO_REUSEADDR for their socket options, and 
succeed in reusing the same local port.

(http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/using_so_exclusiveaddruse.asp)

What I can't find is a way to get Unix-like SO_REUSEADDR behaviour on 
Windows.

Cheers,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From ncoghlan at iinet.net.au  Sat Aug 14 01:31:08 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Aug 14 01:31:15 2004
Subject: [Python-Dev] 'commands' module on Win32
In-Reply-To: <200408131430.i7DEUgK32685@guido.python.org>
References: <411C9A5E.3070109@iinet.net.au>
	<200408131430.i7DEUgK32685@guido.python.org>
Message-ID: <411D4F3C.9010608@iinet.net.au>

Guido van Rossum wrote:

>>Bug #1008275 refers to attempting to run commands.getstatusoutput() on 
>>win32, and the fact that it breaks.
> 
> 
> Hm...  That module's functionality is such a small improvement over
> raw os.popen, it would never have been admitted into the standard
> library these days.
> 
> I'd rather see this added as a higher-level API to one of the process
> management modules that are floating around, one of which will
> eventually land in the library, at which point we can deprecate
> commands.py.
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> 

OK - I'll add something along these lines to the bug report.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From paul.dubois at gmail.com  Sat Aug 14 01:35:32 2004
From: paul.dubois at gmail.com (Paul Du Bois)
Date: Sat Aug 14 01:35:35 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def),
	with implementation
In-Reply-To: <E9AED893-ED7F-11D8-82BD-000A95686CD8@redivi.com>
References: <Pine.LNX.4.44.0408140006060.26777-100000@pingu.thwackety.com>
	<E9AED893-ED7F-11D8-82BD-000A95686CD8@redivi.com>
Message-ID: <85f6a31f04081316356c8a49ce@mail.gmail.com>

[bob, apologies for the re-send... I'm a gmail newbie]

On Fri, 13 Aug 2004 19:24:24 -0400, Bob Ippolito <bob@redivi.com> wrote:
> My only problem with this syntax is that I expect the common cases
> (mine, anyway) to be zero or one decorators per function, so the extra
> block and indent seems a bit excessive compared to the current
> @prefix-symbol-decorator proposal(s).  It is, of course, far better

For the case of one decorator, one could allow:

decorate: staticmethod
def p_statement_expr(self, p):
   pass

Which is admittedly more typing than @staticmethod -- but is brevity a feature?

Even if "decorate" is an inaccurate description of what the decorator
is doing (it may registering, creating grayscale ramps, and so on) it
at least matches the name of the language feature: decorators. Good
for documentation!

This is my favorite proposal so far.
From tim.peters at gmail.com  Sat Aug 14 03:22:32 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug 14 03:22:38 2004
Subject: [Python-Dev] 2.4 asyncore headache
In-Reply-To: <411D4F27.3030605@iinet.net.au>
References: <1f7befae040813125824acdb87@mail.gmail.com>	<20040813200347.GB29067@rogue.amk.ca>
	<1f7befae04081313252cb3304a@mail.gmail.com>
	<411D4F27.3030605@iinet.net.au>
Message-ID: <1f7befae04081318224503fb44@mail.gmail.com>

[Nick Coghlan]
> Is test_socketserver currently passing for you?

Yes, on WinXP, and regardless of whether "currently" means before or
after Andrew reverted the change (& all the ZODB 3.3 tests pass under
current CVS Python on Windows after the reversion).

> A couple of days ago I ran the test suite with -unetwork, and that test
> failed, but I haven't got around to looking into it yet. Could this be
> related?

Which OS?  asyncore isn't involved in that test, so the patch could
not have affected it.  SocketServer.py has its own, separate code
using SO_REUSEADDR in some cases.

test_socketserver.pickport() never reuses a port number anyway. 
However, it may pick a port number that happens to be already in use
outside the test, so that could fail.  The port numbers it picks
depend on the process id, so a "picked a port # already in use"
failure mode may disappear on the next run.

> [rambling discussion of what I found out about this so far follows]
> 
> Anyway, after some hunting, the difference between Windows and Unix
> SO_REUSEADDR, is that on Unix, this only allows local address reuse when
> the existing socket is in a TIME_WAIT state. On Windows, it allows local
> address reuse _anytime_.
>
> See http://hea-www.harvard.edu/~fine/Tech/addrinuse.html. Look for the
> phrase 'security stigma', then read the bit starting with 'On some
> operating systems. . .'

It also says:

    This is only a problem on multi-user machines that don't have restricted
    logins, it is NOT a vulnerability from outside the machine. And it is easily
    avoided by binding your server to the machine's address.

If you're running a server on an open Windows machine, worrying about
details of socket reuse may not be the best use of your security time
<wink>.

> The issue is that if we *don't* set SO_REUSEADDR on Windows, then you
> get into problems with getting EADDRINUSE when you shutdown and restart
> a TCP/IP server on the same local address (generally if the socket has
> gone into TIME_WAIT while shutting down).

That's why ZEO calls asyncore's set_reuse_addr(), so that it can
shutdown/restart rapidly.  This has worked fine for years on Windows.

> Looking at MSDN, SO_EXCLUSIVEADDRUSE is a way to say 'make my socket
> behave like a UNIX socket without SO_REUSEADDR specified'. That is, with
> no options specified, a Windows socket is vulnerable to having another
> process come along, specify SO_REUSEADDR for their socket options, and
> succeed in reusing the same local port.
>
> (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/using_so_exclusiveaddruse.asp)

asyncore shouldn't use this.

> What I can't find is a way to get Unix-like SO_REUSEADDR behaviour on
> Windows.

Well, the more I learn about this one, the less I care about it.  ZEO
can live with Unix-like SO_REUSEADDR behavior too -- I think.  I have
to wonder, because reports of "connection refused" kinds of ZEO test
failures have always been much more common on Linux boxes.
From foom at fuhm.net  Sat Aug 14 03:52:04 2004
From: foom at fuhm.net (James Y Knight)
Date: Sat Aug 14 03:52:11 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <20040813135102.63CF.JCARLSON@uci.edu>
References: <20040813104200.63C3.JCARLSON@uci.edu>
	<Pine.LNX.4.58.0408132346540.3596@rnd.onego.ru>
	<20040813135102.63CF.JCARLSON@uci.edu>
Message-ID: <8ACD1FFD-ED94-11D8-8449-000A95A50FB2@fuhm.net>


On Aug 13, 2004, at 6:53 PM, Josiah Carlson wrote:

>> lambda a mistake? lambda a mistake... -OX
>
> Yes, lambda a mistake:
> http://www.linuxjournal.com/article.php?sid=2959

Of course, in that article, Guido says lambda is a mistake because the 
scoping rules of Python were broken at that point. As they are no 
longer broken, that particular argument doesn't hold much sway...

James


From raymond.hettinger at verizon.net  Sat Aug 14 04:09:28 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sat Aug 14 04:09:54 2004
Subject: [Python-Dev] New Developer
Message-ID: <008701c481a3$bb7250a0$0bb0958d@oemcomputer>

[Johannes Gijsbers]
> > Consider this a request for checkin privileges.
 
[amk]
> Seconded.
 
 
Johannes, your permissions are enabled.
 
Welcome.
 
 
Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040813/ee8a7999/attachment.html
From python at rcn.com  Sat Aug 14 03:30:17 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sat Aug 14 04:16:35 2004
Subject: [Python-Dev] New Developer
In-Reply-To: <20040813185131.GB28569@rogue.amk.ca>
Message-ID: <008001c4819e$4180ea40$0bb0958d@oemcomputer>

[Johannes Gijsbers]
> > Consider this a request for checkin privileges.

[amk]
> Seconded.


Johannes, your permissions are enabled.

Welcome.


Raymond 

From jiangyu at adtec.com.cn  Fri Aug 13 12:16:05 2004
From: jiangyu at adtec.com.cn (Jiang Yu)
Date: Sat Aug 14 05:43:31 2004
Subject: [Python-Dev] Question about win32com in MS-Excel
Message-ID: <000001c481b1$4acf2360$e7100a0a@jiangyu>

Hello, My name is Jiangyu, a Chinese man. You can call me Joe.
I learn Python for only one year.  I think it's a simple tools in my job some time.
But in my country, Python is a new tools for many people. So If i have some 
problems on it, I found it's difficult to look for the answers?So I add the this mail list.

Now, I have a question about use win32com moudle for MS-Excel.
Question: I want to insert a file into a Excel sheet.  But a exception occured in my code.
                 Using VBA, I insert a file OK. And the processing in Python is equivalent in VBA.

My environment: Python2.2.3 + win32com + Win2000 Server + Excel(Office XP)

My Python code:
>>> from win32com.client import Dispatch
>>> excel = Dispatch("Excel.Application")
>>> excel.Visible = 1
>>> excel.WorkBooks.Add()
>>> cursheet = excel.ActiveWorkBook.ActiveSheet
>>> oleobjs = cursheet.OLEObjects()
>>> oleobjs.Add(Filename = r"c:\temp.txt")

My VBA code:
ActiveWorkbook.ActiveSheet.OLEObjects.Add Filename:="c:\temp.txt", Link:=False, DisplayAsIcon:=False

So, If you know how to do it. please answer me. Thanks!

joe
2004/8/13
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040813/12d32d08/attachment.html
From tim.peters at gmail.com  Sat Aug 14 06:37:49 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug 14 06:37:55 2004
Subject: [Python-Dev] Another 2.4 asyncore headache
In-Reply-To: <20040813200347.GB29067@rogue.amk.ca>
References: <1f7befae040813125824acdb87@mail.gmail.com>
	<20040813200347.GB29067@rogue.amk.ca>
Message-ID: <1f7befae040813213728e50a44@mail.gmail.com>

Another change to asyncore broke Zope:

    http://collector.zope.org/Zope3-dev/261

I may have mentioned before that asyncore gives me a headache.

The checkin comment for rev 1.57 just says:

    In poll(), check connections for exceptional conditions

and it's not mentioned in NEWS or in any doc changes.  I don't want to
guess what the intent was.  It definitely has a bad effect on Zope.

I can't guess what asyncore's intent is in the presence of threads
either.  It appears to be robust against other threads mutating the
socket map *across* select() calls.  But Zope appears to mutate the
socket map between the time map.items() captures a snapshot and the
time select() is called.  There's a thread race here when an object in
the socket map is closing, and removing itself from the socket map. 
In 2.3.4 it apparently didn't matter because the exception set passed
to select() was always empty.  In 2.4 it's passing a non-readable
non-writable object in select's exception set, and Zope manages to
close the underlying socket before select() is called.  The result is
a select() error that (a) couldn't happen in previous Pythons, and (b)
Zope sure isn't expecting.

More generally, in 2.3.4 it didn't hurt to leave all kinds of crap in
the socket map, provided that any such crap didn't call itself ready
to read or ready to write.  select() would never see it then.  In 2.4,
select() always sees it.  That's a big change, particularly because
everyone I've seen wrestling with asyncore works via iterative
poke-and-hope, fiddling their code more-or-less randomly, until
mysterious errors stop appearing.

So if it's of real value to you (I don't know the motivating use case
for the change) to pass stuff in the exception set, would it be good
enough to do

    e = r + w

instead of (in effect)

    e = map.keys()

?  Then stuff that doesn't call itself readable or writable would
again, as in 2.3.4, not get passed to select().

Or Zope has to change.
From exarkun at divmod.com  Sat Aug 14 07:32:04 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Sat Aug 14 07:32:09 2004
Subject: [Python-Dev] def ... decorate
In-Reply-To: <20040813213100.GQ23725@performancedrivers.com>
References: <20040813093137.63C0.JCARLSON@uci.edu>	<004d01c48160$e7602d60$0bb0958d@oemcomputer>	<16669.7113.531528.462699@montanaro.dyndns.org>
	<20040813213100.GQ23725@performancedrivers.com>
Message-ID: <411DA3D4.4030007@divmod.com>

Jack Diederich wrote:
> On Fri, Aug 13, 2004 at 02:51:37PM -0500, Skip Montanaro wrote:
> 
>> [snip]
>>
>>    def p_statement_expr:
>>        staticmethod
>>        grammarrule('statement : expression')
>>        version("Added in 2.4")
>>        deprecatedmethod
>>        type_(None)
>>    decorate (self, p):
>>        """docstring here"""
>>        print p[1]
>>
> [snip]
> 
> A group of points
> * it breaks grepability badly

   "@" isn't all that greppable.

     exarkun@boson:~/Twisted$ grep -E '^[ \t]+@\w+' ./ -r | wc -l
     5025

   Personally, greppability doesn't seem particular important either way.

> * it breaks existing tools by radically changing the grammar

   Existing tools are going to break.  That's what happens when you 
change the grammar.

   Jp
From ncoghlan at iinet.net.au  Sat Aug 14 09:40:18 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Aug 14 09:40:26 2004
Subject: [Python-Dev] 2.4 asyncore headache
In-Reply-To: <1f7befae04081318224503fb44@mail.gmail.com>
References: <1f7befae040813125824acdb87@mail.gmail.com>	<20040813200347.GB29067@rogue.amk.ca>	<1f7befae04081313252cb3304a@mail.gmail.com>
	<411D4F27.3030605@iinet.net.au>
	<1f7befae04081318224503fb44@mail.gmail.com>
Message-ID: <411DC1E2.9080200@iinet.net.au>

Tim Peters wrote:

> [Nick Coghlan]
> 
>>A couple of days ago I ran the test suite with -unetwork, and that test
>>failed, but I haven't got around to looking into it yet. Could this be
>>related?
> 
> 
> Which OS?  asyncore isn't involved in that test, so the patch could
> not have affected it.  SocketServer.py has its own, separate code
> using SO_REUSEADDR in some cases.

Ah, OK, it was just a thought. I'm on Windows XP SP1, and it's 
consistently choking on the second attempt to test the UDP Server:

==Test=Output==
[snipped output from TCP tests]
ADDR = ('localhost', 12963)
CLASS = SocketServer.UDPServer
server created
thread: creating server
server running
thread: serving three times
test client 0
test client 1
test test_socketserver failed -- test failed
1 test failed:
     test_socketserver
^C
==End=Test=Output==

The Ctrl-Break comes from the fact that the test hangs at that point 
without exiting. 'netstat -a' shows that there isn't anything using 
anything even close to those port numbers.

I plan to do some more poking to try and narrow down a bit further what 
is going on before going to SF with it. (There are 5 different network 
connections on this laptop, including a virtual one to the Linux 
install, so I wouldn't be surprised if there is something odd going on 
with the network interface)

> It also says:
> 
>     This is only a problem on multi-user machines that don't have restricted
>     logins, it is NOT a vulnerability from outside the machine. And it is easily
>     avoided by binding your server to the machine's address.
> 
> If you're running a server on an open Windows machine, worrying about
> details of socket reuse may not be the best use of your security time
> <wink>.

Indeed. I think it suggests that continuing to use SO_REUSEADDR normally 
on Windows is the best option for ayncore, though.

Windows users just need to be aware that it doesn't mean quite the same 
thing as it does on a Unix-like system (actually, some of the other 
Google hits made me wonder if it really means the same thing in all the 
different Unix variants in the first place. . .)

Regards,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From martin at v.loewis.de  Sat Aug 14 09:48:45 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug 14 09:48:52 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <411C85B6.2060104@egenix.com>
References: <1092258146.411a896246d1a@mcherm.com>	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>	<16666.56033.65067.241794@montanaro.dyndns.org>	<200408120414.i7C4Er429003@guido.python.org>	<411AFBD4.5080903@v.loewis.de>	<200408120545.i7C5j4W29250@guido.python.org>	<411B0DA0.4030109@v.loewis.de>	<16667.32457.28553.468008@montanaro.dyndns.org>	<411BB50C.8000804@v.loewis.de>
	<16667.54746.535997.120878@montanaro.dyndns.org>
	<411C85B6.2060104@egenix.com>
Message-ID: <411DC3DD.3080900@v.loewis.de>

M.-A. Lemburg wrote:
> If we switch the binding of 'yyy' to mean unicode('yyy')
> some day, why can't we just continue to use the existing implementation
> for 8-bit strings for b'xxx' (the current implementation is already
> doing the right thing, meaning that it is 8-bit safe regardeless
> of the source code encoding) ?

Not exactly - the current implementation is not safe with respect to
re-encoding source in a different encoding.

Regards,
Martin
From dima at trit.org  Sat Aug 14 12:31:50 2004
From: dima at trit.org (Dima Dorfman)
Date: Sat Aug 14 12:34:33 2004
Subject: [Python-Dev] Multi-line import implementation (was: 2.4a2,
	and @decorators)
In-Reply-To: <200408111446.i7BEkHI27825@guido.python.org>
References: <1092232661.411a25d55a567@mcherm.com>
	<200408111446.i7BEkHI27825@guido.python.org>
Message-ID: <20040814103150.GR82909@trit.org>

Guido van Rossum <guido@python.org> wrote:
> Michael Chermside wrote:
> > Others have already mentioned that "from sys import xxx, xxx, xxx"
> > is the case that badly needs line wrapping. I would, however, like
> > to suggest that (if it's easy to do) you allow a trailing comma when
> > parentheses are used. [...]
> 
> +1

Certainly easy to do (in fact, changing the grammar was almost
sufficient). I've uploaded a new patch to SF, with the comment saying,
in part,

  - Support trailing comma in from list (suggested by Michael
    Chermside on python-dev). This form is now acceptable:

      from os import (path,)

    but this is not:

      from os import path,

    I originally wanted to support the latter for consistency, but I
    don't think it's necessary (who would want to do that but not use
    parens?) and it's potentially confusing in a case like this:

      from sys import stdin,
      stdout

    which looks like it was intended to be one statement, but the
    second line is actually separate. The way the grammar is written,
    the case with a trailing comma but no parentheses parses okay, but
    it is intentionally rejected by the compiler.

If we want to support that case for some reason, it's simple enough to
remove the rejection test from the compiler.

Dima.
From skip at pobox.com  Sat Aug 14 15:02:59 2004
From: skip at pobox.com (Skip Montanaro)
Date: Sat Aug 14 15:03:07 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <411DC3DD.3080900@v.loewis.de>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org>
	<411B0DA0.4030109@v.loewis.de>
	<16667.32457.28553.468008@montanaro.dyndns.org>
	<411BB50C.8000804@v.loewis.de>
	<16667.54746.535997.120878@montanaro.dyndns.org>
	<411C85B6.2060104@egenix.com> <411DC3DD.3080900@v.loewis.de>
Message-ID: <16670.3459.285216.939504@montanaro.dyndns.org>

    >> ... 8-bit strings for b'xxx' (the current implementation is already
    >> doing the right thing, meaning that it is 8-bit safe regardeless of
    >> the source code encoding) ?

    Martin> Not exactly - the current implementation is not safe with
    Martin> respect to re-encoding source in a different encoding.

Can't such re-encoding tools (whatever they are) be intelligent about
b"..."?  If they are Python-aware that seems fairly trivial.  If not, the
job is more difficult.

Skip

From martin at v.loewis.de  Sat Aug 14 17:21:27 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Aug 14 17:21:33 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <16670.3459.285216.939504@montanaro.dyndns.org>
References: <1092258146.411a896246d1a@mcherm.com>
	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>
	<16666.56033.65067.241794@montanaro.dyndns.org>
	<200408120414.i7C4Er429003@guido.python.org>
	<411AFBD4.5080903@v.loewis.de>
	<200408120545.i7C5j4W29250@guido.python.org>
	<411B0DA0.4030109@v.loewis.de>
	<16667.32457.28553.468008@montanaro.dyndns.org>
	<411BB50C.8000804@v.loewis.de>
	<16667.54746.535997.120878@montanaro.dyndns.org>
	<411C85B6.2060104@egenix.com> <411DC3DD.3080900@v.loewis.de>
	<16670.3459.285216.939504@montanaro.dyndns.org>
Message-ID: <411E2DF7.3010705@v.loewis.de>

Skip Montanaro wrote:
> Can't such re-encoding tools (whatever they are) be intelligent about
> b"..."?  If they are Python-aware that seems fairly trivial.  If not, the
> job is more difficult.

What precisely should they do, though? A byte sequence in one encoding
might be invalid in another.

Regards,
Martin
From tim.peters at gmail.com  Sat Aug 14 18:15:35 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Aug 14 18:15:41 2004
Subject: [Python-Dev] 2.4 asyncore headache
In-Reply-To: <411DC1E2.9080200@iinet.net.au>
References: <1f7befae040813125824acdb87@mail.gmail.com>	<20040813200347.GB29067@rogue.amk.ca>	<1f7befae04081313252cb3304a@mail.gmail.com>
	<411D4F27.3030605@iinet.net.au>
	<1f7befae04081318224503fb44@mail.gmail.com>
	<411DC1E2.9080200@iinet.net.au>
Message-ID: <1f7befae04081409157340b56e@mail.gmail.com>

[Tim]
>> Which OS?  asyncore isn't involved in that test, so the patch could
>> not have affected it.  SocketServer.py has its own, separate code
>> using SO_REUSEADDR in some cases.

[Nick Coghlan]
> Ah, OK, it was just a thought. I'm on Windows XP SP1,

Same here.

> and it's consistently choking on the second attempt to test the UDP Server:

Not here, but there are many ways to run the tests, and how you run
them may also be important.  I don't know of a way that "should" fail
in this test, but I'll show you one way that works on my SP1 box:

C:\Code\python\PCbuild>python ../lib/test/test_socketserver.py
ADDR = ('localhost', 16401)
CLASS = SocketServer.TCPServer
server created
server running
thread: creating server
thread: serving three times
test client 0
test client 1
test client 2
thread: done
waiting for server
done
ADDR = ('localhost', 16402)
CLASS = SocketServer.ThreadingTCPServer
server created
server running
thread: creating server
thread: serving three times
test client 0
test client 1
test client 2
thread: done
waiting for server
done
ADDR = ('localhost', 16403)
CLASS = SocketServer.UDPServer
server created
server running
thread: creating server
thread: serving three times
test client 0
test client 1
test client 2
thread: done
waiting for server
done
ADDR = ('localhost', 16404)
CLASS = SocketServer.ThreadingUDPServer
server created
server running
thread: creating server
thread: serving three times
test client 0
test client 1
test client 2
thread: done
waiting for server
done

C:\Code\python\PCbuild>

> ...
> I plan to do some more poking to try and narrow down a bit further what
> is going on before going to SF with it. (There are 5 different network
> connections on this laptop, including a virtual one to the Linux
> install, so I wouldn't be surprised if there is something odd going on
> with the network interface)

Neither would I <wink>.

...
> Indeed. I think it suggests that continuing to use SO_REUSEADDR normally
> on Windows is the best option for ayncore, though.

"normally" is loaded:  *normally* (by default), asyncore doesn't use
SO_REUSEADDR at all, on any box.  You have to call
asyncore.dispatcher.set_reuse_addr() explicitly if you *want* to get
into the reuse game (and, e.g., ZEO does).  The only asyncore client
in the Python core that does this is smtpd.SMTPServer.

> Windows users just need to be aware that it doesn't mean quite the same
> thing as it does on a Unix-like system

In general, they aren't aware of such subtle distinctions -- and most
would simply be annoyed if you insisted that they should be <0.7
wink>.

> (actually, some of the other Google hits made me wonder if it really means
> the same thing in all the different Unix variants in the first place. . .)

Alas, portability across Unix variants is often spelled #ifdef.
From jhylton at gmail.com  Sat Aug 14 19:54:06 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Sat Aug 14 19:54:12 2004
Subject: [Python-Dev] Another 2.4 asyncore headache
In-Reply-To: <1f7befae040813213728e50a44@mail.gmail.com>
References: <1f7befae040813125824acdb87@mail.gmail.com>
	<20040813200347.GB29067@rogue.amk.ca>
	<1f7befae040813213728e50a44@mail.gmail.com>
Message-ID: <e8bf7a530408141054ddec16@mail.gmail.com>

On Sat, 14 Aug 2004 00:37:49 -0400, Tim Peters <tim.peters@gmail.com> wrote:
> More generally, in 2.3.4 it didn't hurt to leave all kinds of crap in
> the socket map, provided that any such crap didn't call itself ready
> to read or ready to write.  select() would never see it then.  In 2.4,
> select() always sees it.  That's a big change, particularly because
> everyone I've seen wrestling with asyncore works via iterative
> poke-and-hope, fiddling their code more-or-less randomly, until
> mysterious errors stop appearing.
> 
> So if it's of real value to you (I don't know the motivating use case
> for the change) to pass stuff in the exception set, would it be good
> enough to do
> 
>     e = r + w
> 
> instead of (in effect)
> 
>     e = map.keys()
> 
> ?  Then stuff that doesn't call itself readable or writable would
> again, as in 2.3.4, not get passed to select().

I don't understand the use case or the correct intended use of
asyncore <wink>, but
    e = r + w
looks appealing.  It allows object to be in the socket map without
requiring that they be ready to do some kind of I/O at any instant
that select is called.

Jeremy
From bac at OCF.Berkeley.EDU  Sun Aug 15 01:01:30 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Aug 15 01:01:41 2004
Subject: [Python-Dev] Question about win32com in MS-Excel
In-Reply-To: <000001c481b1$4acf2360$e7100a0a@jiangyu>
References: <000001c481b1$4acf2360$e7100a0a@jiangyu>
Message-ID: <411E99CA.5040403@ocf.berkeley.edu>

Jiang Yu wrote:
[SNIP]
> Now, I have a question about use win32com moudle for MS-Excel.
[SNIP]

This is the wrong place to ask about win32com (and the win32all
package), Jiang.  This list is to discuss the development of Python the
language and not any extension modules.

Try comp.lang.python, the newsgroup, or the win32all mailing list.

-Brett
From hans at zephyrfalcon.org  Sun Aug 15 04:26:09 2004
From: hans at zephyrfalcon.org (Hans Nowak)
Date: Sun Aug 15 04:30:23 2004
Subject: [Python-Dev] Decorators and docstrings
Message-ID: <cfmhkp$4j7$1@sea.gmane.org>


Hi,

When playing with the decorators in 2.4a2, I saw the following behavior:

# wraps an object with the function, and prints this object
# when the function is called.
 >>> def wrapwith(obj):
...     def decorator(f):
...         def _wrapper(*args, **kwargs):
...             print "##", obj
...             return f(*args, **kwargs)
...         return _wrapper
...     return decorator
...
 >>> @wrapwith("hello")
... def foo(x, y):
...     """ foo! """
...     return 2*x + y
...
 >>> foo(4, 5)
## hello
13
 >>> foo.__doc__
 >>>

I.e. if you wrap the function foo with the decorator, it loses the docstring. 
foo.__doc__ is empty.

Is this behavior intended?  Should the decorator in question (in this case, 
'wrapwith') take care of copying of the docstring itself?  Or should the 
decorator mechanism do this automatically (if possible)?

Thanks,

--
Hans Nowak (hans@zephyrfalcon.org)
http://zephyrfalcon.org/

From bob at redivi.com  Sun Aug 15 04:43:30 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Aug 15 04:43:30 2004
Subject: [Python-Dev] Decorators and docstrings
In-Reply-To: <cfmhkp$4j7$1@sea.gmane.org>
References: <cfmhkp$4j7$1@sea.gmane.org>
Message-ID: <E47F91B7-EE64-11D8-8645-000A95686CD8@redivi.com>

On Aug 14, 2004, at 10:26 PM, Hans Nowak wrote:

> When playing with the decorators in 2.4a2, I saw the following 
> behavior:
>
> # wraps an object with the function, and prints this object
> # when the function is called.
> >>> def wrapwith(obj):
> ...     def decorator(f):
> ...         def _wrapper(*args, **kwargs):
> ...             print "##", obj
> ...             return f(*args, **kwargs)
> ...         return _wrapper
> ...     return decorator
> ...
> >>> @wrapwith("hello")
> ... def foo(x, y):
> ...     """ foo! """
> ...     return 2*x + y
> ...
> >>> foo(4, 5)
> ## hello
> 13
> >>> foo.__doc__
> >>>
>
> I.e. if you wrap the function foo with the decorator, it loses the 
> docstring. foo.__doc__ is empty.
>
> Is this behavior intended?  Should the decorator in question (in this 
> case, 'wrapwith') take care of copying of the docstring itself?  Or 
> should the decorator mechanism do this automatically (if possible)?

Decorators aren't that special.  If you want to return a function 
object that has the same doc string as the original then you're going 
to have to return the original object or explicitly make sure your new 
function object has the same doc string.  _wrapper.__doc__ = 
decorator.__doc__ before the return should probably do it.

I don't think that automatically setting a __doc__ string on the result 
object is a good idea at all because it could cause unnecessary 
problems for some result objects (instances of a class, for example).

-bob
From bac at OCF.Berkeley.EDU  Sun Aug 15 08:09:09 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Aug 15 08:09:17 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__
	but	aren't
In-Reply-To: <200408110002.i7B02gu26415@guido.python.org>
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>	<2mekmf14jc.fsf@starship.python.net>
	<4118D57C.9090306@divmod.com>	<2macx31423.fsf@starship.python.net>	<200408101501.i7AF1HZ25725@guido.python.org>
	<BE05E963-EAE8-11D8-9899-000A95A50FB2@fuhm.net>
	<200408110002.i7B02gu26415@guido.python.org>
Message-ID: <411EFE05.7090507@ocf.berkeley.edu>

Guido van Rossum wrote:

>>Sooo should (for 'generator' in objects that claim to be in 
>>__builtins__ but aren't),
>>1) 'generator' be added to __builtins__
>>2) 'generator' be added to types.py and its __module__ be set to 'types'
>>3) 'generator' be added to <newmodule>.py and its __module__ be set to 
>>'<newmodule>' (and a name for the module chosen)
> 
> 
> I guess (1).
> 

The problem I see with explicitly adding this stuff to __builtins__ is 
that tab-completion in the interpreter is suddenly going to have all of 
this extra stuff that people are not going want to really see that often.

The other point I would like to make is that almost everything in 
__builtins__ is either an exception, a singleton (thinking of True and 
False), or a function (even thinking of list and dict since you can use 
their factory methods).  The only exceptions I can think of are __name__ 
and that is just part of the design.  Throwing in generator and any of 
the other types that are defined by the built-in types will go against 
all of this unless we create factory methods for them which is not 
really desired since they are returned only in certain situations.

I personally prefer option 2.

-Brett
From guido at python.org  Sun Aug 15 08:33:13 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Aug 15 08:33:22 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but
	aren't
In-Reply-To: Your message of "Sat, 14 Aug 2004 23:09:09 PDT."
	<411EFE05.7090507@ocf.berkeley.edu> 
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com>
	<2macx31423.fsf@starship.python.net>
	<200408101501.i7AF1HZ25725@guido.python.org>
	<BE05E963-EAE8-11D8-9899-000A95A50FB2@fuhm.net>
	<200408110002.i7B02gu26415@guido.python.org> 
	<411EFE05.7090507@ocf.berkeley.edu> 
Message-ID: <200408150633.i7F6XE203096@guido.python.org>

> Guido van Rossum wrote:
> 
> >>Sooo should (for 'generator' in objects that claim to be in 
> >>__builtins__ but aren't),
> >>1) 'generator' be added to __builtins__
> >>2) 'generator' be added to types.py and its __module__ be set to 'types'
> >>3) 'generator' be added to <newmodule>.py and its __module__ be set to 
> >>'<newmodule>' (and a name for the module chosen)
> > 
> > I guess (1).

[Note: it should be added to __builtin__, not __builtins__ -- the two
are related but __builtin__ is the module and __builtins__ is a secret
attribute that references the module or its __dict__ for purposes of
sandboxing.

[Brett C]
> The problem I see with explicitly adding this stuff to __builtins__
> is that tab-completion in the interpreter is suddenly going to have
> all of this extra stuff that people are not going want to really see
> that often.

There's already lots of stuff there that's rarely used.

> The other point I would like to make is that almost everything in
> __builtins__ is either an exception, a singleton (thinking of True
> and False), or a function (even thinking of list and dict since you
> can use their factory methods).

But list and dict are *not* functions, they are types, and they can be
used for subclassing and type checking too.  These are the precedent
for placing generator etc. in __builtin__.

> The only exceptions I can think of are __name__ and that is just
> part of the design.  Throwing in generator and any of the other
> types that are defined by the built-in types will go against all of
> this unless we create factory methods for them which is not really
> desired since they are returned only in certain situations.

They should not be made factory functions if they aren't already.

> I personally prefer option 2.

Bah.  The types module shouldn't be the "home" for any types -- it's
just a collection of convenient aliases, and mostly they have the
wrong names.  It should be deprecated.

Also note that at least 6 of these "anonymous" built-in types
currently have another alias in the 'new' module, which means that
they *are* factories already.  It would be nice if that module could
finally be deprecated.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From bac at OCF.Berkeley.EDU  Sun Aug 15 09:22:18 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Aug 15 09:22:35 2004
Subject: [Python-Dev] Decorator order implemented backwards?
In-Reply-To: <1092172040.1681.9.camel@localhost>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost>
Message-ID: <411F0F2A.2070709@ocf.berkeley.edu>

Mark Russell wrote:

[SNIP - stuff about the order]
> I'll do a patch to fix the order and the corresponding tests.
> 

I hope I am not stepping on Mark's toes but I went ahead and fixed it to 
be bottom-up like the PEP.  Applied as rev. 2.318 for Python/compile.c 
and rev. 1.4 for Lib/test/test_decorators.py (rewrote the order test to 
not rely on causing an error to signify an error to but actually lead to 
a failure).

I did not make any changes to to force decorators on to a single line, 
though.  Mark can still have that one.  =)

-Brett
From rnd at onego.ru  Sun Aug 15 10:50:53 2004
From: rnd at onego.ru (Roman Suzi)
Date: Sun Aug 15 10:49:55 2004
Subject: [Python-Dev] Decorators and docstrings
In-Reply-To: <E47F91B7-EE64-11D8-8645-000A95686CD8@redivi.com>
References: <cfmhkp$4j7$1@sea.gmane.org>
	<E47F91B7-EE64-11D8-8645-000A95686CD8@redivi.com>
Message-ID: <Pine.LNX.4.58.0408151246550.1851@rnd.onego.ru>

On Sat, 14 Aug 2004, Bob Ippolito wrote:

>On Aug 14, 2004, at 10:26 PM, Hans Nowak wrote:
>
>> When playing with the decorators in 2.4a2, I saw the following
>> behavior:
>>
>> # wraps an object with the function, and prints this object
>> # when the function is called.
>> >>> def wrapwith(obj):
>> ...     def decorator(f):
>> ...         def _wrapper(*args, **kwargs):
>> ...             print "##", obj
>> ...             return f(*args, **kwargs)
>> ...         return _wrapper
>> ...     return decorator
>> ...
>> >>> @wrapwith("hello")
>> ... def foo(x, y):
>> ...     """ foo! """
>> ...     return 2*x + y
>> ...
>> >>> foo(4, 5)
>> ## hello
>> 13
>> >>> foo.__doc__
>> >>>
>>
>> I.e. if you wrap the function foo with the decorator, it loses the
>> docstring. foo.__doc__ is empty.
>>
>> Is this behavior intended?  Should the decorator in question (in this
>> case, 'wrapwith') take care of copying of the docstring itself?  Or
>> should the decorator mechanism do this automatically (if possible)?
>
>Decorators aren't that special.  If you want to return a function
>object that has the same doc string as the original then you're going
>to have to return the original object or explicitly make sure your new
>function object has the same doc string.  _wrapper.__doc__ =
>decorator.__doc__ before the return should probably do it.
>
>I don't think that automatically setting a __doc__ string on the result
>object is a good idea at all because it could cause unnecessary
>problems for some result objects (instances of a class, for example).

Probably a decorator could be written to facilitate writing
docstring- and other such info saving decorators...

def wrapwith(obj):
   @docsafe
   def decorator(f):
       def _wrapper(*args, **kwargs):
           print "##", obj
           return f(*args, **kwargs)
       return _wrapper
   return decorator



>-bob
>_______________________________________________
>Python-Dev mailing list
>Python-Dev@python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: http://mail.python.org/mailman/options/python-dev/rnd%40onego.ru
>

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From rnd at onego.ru  Sun Aug 15 11:00:32 2004
From: rnd at onego.ru (Roman Suzi)
Date: Sun Aug 15 10:59:34 2004
Subject: [Python-Dev] Decorators are function/method subclassing technique?
Message-ID: <Pine.LNX.4.58.0408151252160.1851@rnd.onego.ru>


After considering the docstring problem of decorators, it occured to me
that decorators could be seen as an inheritance mechanism of functions.

class docsafe(function):
   ...

@ docsafe
def f(x, y, z):
   """ To be saved """
   ...


print f
<__main__.docsafe object at 0x401e074c>


Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From marktrussell at btopenworld.com  Sun Aug 15 17:27:27 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Sun Aug 15 17:27:31 2004
Subject: [Python-Dev] Decorator order implemented backwards?
In-Reply-To: <411F0F2A.2070709@ocf.berkeley.edu>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu>
Message-ID: <1092583647.1670.18.camel@localhost>

On Sun, 2004-08-15 at 08:22, Brett C. wrote:
> Mark Russell wrote:
> 
> [SNIP - stuff about the order]
> > I'll do a patch to fix the order and the corresponding tests.
> > 
> 
> I hope I am not stepping on Mark's toes but I went ahead and fixed it to 
> be bottom-up like the PEP. 

Now that's what I call timing :-)  I finished the patch to fix this
yesterday and decided to leave it until today to upload (I've been
swamped this week, hence the delay getting time to do this).  I've
uploaded it now (as patch #1009560) because I think it is a more
correct fix (as well as also fixing the Lib/compiler package).

It turns out that the order of evaluation of issue is more subtle than
it first appears.  Here's the comment from the start of the new
test_eval_order() in test_decorators.py:

        # Evaluating a decorated function involves four steps for each
        # decorator-maker (the function that returns a decorator):
        #
        #    1: Evaluate the decorator-maker name
        #    2: Evaluate the decorator-maker arguments (if any)
        #    3: Call the decorator-maker to make a decorator
        #    4: Call the decorator
        #
        # When there are multiple decorators, these steps should be
        # performed in the above order for each decorator, but we should
        # iterate through the decorators in the reverse of the order
        # they appear in the source.

Your patch results in the evaluation order:

    evalname1 evalargs1 makedec1
    evalname2 evalargs2 makedec2
    evalname3 evalargs3 makedec3
    calldec3 calldec2 calldec1

Mine (#1009560) gives:

    evalname3 evalargs3 makedec3 calldec3
    evalname2 evalargs2 makedec2 calldec2
    evalname1 evalargs1 makedec1 calldec1

I would defend this as the correct order because it is the same as the
order you get with the current method (func = staticmethod(func)).

> I did not make any changes to to force decorators on to a single 
> line, though.  Mark can still have that one.  =)

Thanks :-)  My patch implements that change as well (plus doc and test
updates to match).

Mark


From eppstein at ics.uci.edu  Sun Aug 15 19:11:08 2004
From: eppstein at ics.uci.edu (David Eppstein)
Date: Sun Aug 15 19:11:20 2004
Subject: [Python-Dev] Re: Decorator order implemented backwards?
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost>
	<411F0F2A.2070709@ocf.berkeley.edu>
	<1092583647.1670.18.camel@localhost>
Message-ID: <eppstein-11FD65.10110815082004@sea.gmane.org>

In article <1092583647.1670.18.camel@localhost>,
 Mark Russell <marktrussell@btopenworld.com> wrote:

[Brett C's]
> patch results in the evaluation order:
> 
>     evalname1 evalargs1 makedec1
>     evalname2 evalargs2 makedec2
>     evalname3 evalargs3 makedec3
>     calldec3 calldec2 calldec1
> 
> Mine (#1009560) gives:
> 
>     evalname3 evalargs3 makedec3 calldec3
>     evalname2 evalargs2 makedec2 calldec2
>     evalname1 evalargs1 makedec1 calldec1

It would probably be bad style to have any order dependencies in the 
evalname evalargs makedec parts, but Brett's order makes more sense to 
me. That is, I like the actual application of decorators to be in nested 
order, but it makes more sense to me for the lines constructing the 
decorators to be evaluated in the order they appear in the code.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

From marktrussell at btopenworld.com  Sun Aug 15 21:46:03 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Sun Aug 15 21:46:06 2004
Subject: [Python-Dev] Re: Decorator order implemented backwards?
In-Reply-To: <eppstein-11FD65.10110815082004@sea.gmane.org>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au> <41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost> <411F0F2A.2070709@ocf.berkeley.edu>
	<1092583647.1670.18.camel@localhost>
	<eppstein-11FD65.10110815082004@sea.gmane.org>
Message-ID: <1092599162.29945.51.camel@localhost>

On Sun, 2004-08-15 at 18:11, David Eppstein wrote:
> It would probably be bad style to have any order dependencies in the 
> evalname evalargs makedec parts, but Brett's order makes more sense to 
> me. That is, I like the actual application of decorators to be in nested 
> order, but it makes more sense to me for the lines constructing the 
> decorators to be evaluated in the order they appear in the code.

On second thoughts I think you are right.  I was concerned about the
difficulty of explaining the two different evaluation orders, but that
is outweighed by the non-intuitiveness of having decorator expressions
evaluated in the opposite of source code order.  I've updated the patch.

Here's my current attempt at describing the semantics in the reference
manual:

    A function definition may be wrapped by one or more decorator
    expressions. Decorator expressions are evaluated when the function
    is defined, in the scope that contains the function definition.
    The result must be a callable, which is invoked with the function
    object as the only argument.  The returned value is bound to the
    function name instead of the function object.  Multiple decorators
    are applied in nested fashion.  For example, the following code:

	@f1(arg)
	@f2
	def func(): pass

    is equivalent to:

	def func(): pass
	func = f1(arg)(f2(func))

Mark

From bac at OCF.Berkeley.EDU  Sun Aug 15 22:05:28 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Aug 15 22:05:44 2004
Subject: [Python-Dev] Decorator order implemented backwards?
In-Reply-To: <1092583647.1670.18.camel@localhost>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au>	<1092172040.1681.9.camel@localhost>
	<411F0F2A.2070709@ocf.berkeley.edu>
	<1092583647.1670.18.camel@localhost>
Message-ID: <411FC208.3040800@ocf.berkeley.edu>

Mark Russell wrote:

> On Sun, 2004-08-15 at 08:22, Brett C. wrote:
> 
>>Mark Russell wrote:
>>
>>[SNIP - stuff about the order]
>>
>>>I'll do a patch to fix the order and the corresponding tests.
>>>
>>
>>I hope I am not stepping on Mark's toes but I went ahead and fixed it to 
>>be bottom-up like the PEP. 
> 
> 
> Now that's what I call timing :-)  I finished the patch to fix this
> yesterday and decided to leave it until today to upload (I've been
> swamped this week, hence the delay getting time to do this).  I've
> uploaded it now (as patch #1009560) because I think it is a more
> correct fix (as well as also fixing the Lib/compiler package).
> 

=)  I kept doing cvs update, just waiting for someone to fix this right 
in the middle of me doing all of this.

> It turns out that the order of evaluation of issue is more subtle than
> it first appears.  Here's the comment from the start of the new
> test_eval_order() in test_decorators.py:
> 
>         # Evaluating a decorated function involves four steps for each
>         # decorator-maker (the function that returns a decorator):
>         #
>         #    1: Evaluate the decorator-maker name
>         #    2: Evaluate the decorator-maker arguments (if any)
>         #    3: Call the decorator-maker to make a decorator
>         #    4: Call the decorator
>         #
>         # When there are multiple decorators, these steps should be
>         # performed in the above order for each decorator, but we should
>         # iterate through the decorators in the reverse of the order
>         # they appear in the source.
> 
> Your patch results in the evaluation order:
> 
>     evalname1 evalargs1 makedec1
>     evalname2 evalargs2 makedec2
>     evalname3 evalargs3 makedec3
>     calldec3 calldec2 calldec1
> 
> Mine (#1009560) gives:
> 
>     evalname3 evalargs3 makedec3 calldec3
>     evalname2 evalargs2 makedec2 calldec2
>     evalname1 evalargs1 makedec1 calldec1
> 
> I would defend this as the correct order because it is the same as the
> order you get with the current method (func = staticmethod(func)).
> 

I am leaving this up to Guido (seems to have become the default PEP 318 
champion) to make the decision on the order.  Personally I like my order 
more since I would expect evaluation of arguments to be in the order 
listed, just the application order to be reversed.

Either way I won't be able to deal with this patch if it is accepted; 
with the python-dev Summary coverage period ending today I still have 
728 emails to summarize on top of any sent in today so I will be swamped 
with that for the rest of the week.  I am sure someone else can step up 
and applying it if Guido okays the application order.

-Brett
From bac at OCF.Berkeley.EDU  Sun Aug 15 22:09:48 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Aug 15 22:09:54 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__
	but aren't
In-Reply-To: <200408150633.i7F6XE203096@guido.python.org>
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com>
	<2macx31423.fsf@starship.python.net>
	<200408101501.i7AF1HZ25725@guido.python.org>
	<BE05E963-EAE8-11D8-9899-000A95A50FB2@fuhm.net>
	<200408110002.i7B02gu26415@guido.python.org>
	<411EFE05.7090507@ocf.berkeley.edu>
	<200408150633.i7F6XE203096@guido.python.org>
Message-ID: <411FC30C.2000200@ocf.berkeley.edu>

Guido van Rossum wrote:

[SNIP]
> Bah.  The types module shouldn't be the "home" for any types -- it's
> just a collection of convenient aliases, and mostly they have the
> wrong names.  It should be deprecated.
> 

Well that settles that one.  Didn't know you wanted to deprecate 
'types'.  Then __builtin__ does seem to be the only logical place.

-Brett
From mzarate at uoguelph.ca  Fri Aug 13 14:20:22 2004
From: mzarate at uoguelph.ca (Martin Zarate)
Date: Sun Aug 15 22:40:52 2004
Subject: [Python-Dev] block-based decorators and other block issues
Message-ID: <1092399622.411cb206345b5@webmail.uoguelph.ca>

Our discussion has taken an interesting turn, leading to this concept 
of "super-decorators" that are more versatile than normal decorators, because 
they can accept multiple objects, and because they can play with namespaces.

First of all, it is important not to get too far ahead of ourselves - 
decorators should not be full "custom blocks" like having a "threadsafe" 
decorator block and a "fork function" block wherein the decoratotor is 
changing the fundamental behaviour of a block of code.  Concepts that we see 
in Ruby (I highly, highly recommend that people interested in this subject 
read over the Ruby tutorial, it is very insightful on the concept of versatile 
block constructs) could be brought to Python, but it would have to be done 
properly through a good general-purpose mechanism.

Decorators are simply a filtering mechanism - they take an object defined with 
the decorator, and then either a) return a substitute to the parent scope or, 
possibly b) throw it out.  They are not blocking concepts.

The reason is simple - speed.  A decorator shouldn't have to perform 
fasttolocals and vice versa.  It doesn't need to be its own variable 
namespace - it can sit within the external namespace.  To do otherwise is 
suboptimal.  The decorator should not have its own scope.  Instead, it should 
provide a simple filter.

Now, it would be nice if Python had another versatile block structure usable 
for custom stuff that does involve having its own full namespace (hell, just 
a "block" keyword would do) but I'll get to that later.

First, I need a keyword.   I will use "turn", just to keep things different, 
but anything will do - has has been remarked elsewhere, a naked tuple with 
a ":" suffix would be enough - it would differentiate the conversion block 
from standard comma-delimited lists, while not looking like any other block.

At any rate, people advocating the "decorator block" concept that I suggested  
keep assuming that the descriptor functions are functions that take a whole 
namespace in and put a whole other namespace out, which is then imported into 
the parent namespace - horribly inefficient.  Use that later for custom blocks.

The call signature of a descriptor could simply be

descriptor(object, name, holder, doc, args)

where object is the object being described, name is the string with its name 
(immutable, as always) and holder is the current contents of the holder.  The 
return value of the descriptor function is assigned into the holder at the end 
of the operation - if no change is desired and the nested objects are to be 
thrown out, then simply return the holder parameter.  The name parameter 
exists for the sake of naming objects.  With the holder parameter, inspection 
of the existing object is allowed, so that the "property_get" descriptor 
(which actually alters an existing property or creates a new one if none 
exists) can work.  The reason for the longer function signature is so that 
more complicated syntax is available to the decorator function - such as a 
docstring that can be prepended to the nested docstrings.  This seems silly 
for functions, but would be appropriate when decorating non-function objects 
(such as data members).

Creating a new scope block for decorators would be too slow, but still would 
be very useful for more bizarre custom linguistic constructs.  A general 
purpose "scope" command with access to external and internal namespaces would 
be extremely useful.

Imagine the general purpose "scope" statement:

scope handler1(args), handler2(args), handler3(args):

where the contained code within the block would be run, then its local 
namespace harvested, as well as the external namespace, both passed in to

handlerN(inner, outer, args)

this would pretty much let you do whatever you want within a random scope: 
block, allowing for some bizzarro macroesque stuff that would make descriptors 
look simple by comparison.  Personally I think such a structure would be silly 
and dangerous, but its what many coders sound like they want.

A better, more general-purpose structure would be an "object" block like

object name handler(args):

wherein the contained code is run on the spot and the local namespace is 
converted into a dictionary, which is then called with the signature

handler(namespace, name, args)

where namespace is the namespace is the local namespace, name is the string of 
the name, and args is the argument tuple.  You could use this to code your own 
analogs to the class structure of Python.  Say you want a forced single-
inheritence-tree class system?  You could have one with such custom blocks, or 
any other alternate-class-structure.  Hell, with such a system the class 
itself wouldn't need to be a true keyword as it is now, but instead could 
behave as simply just one more "object" expression handler - as I understand 
it, all it does is take that namespace and the argument tuple and do some fun 
stuff with it and then plonk that info (plus the name of the class) into 
the "type" constructor.  That doesn't need to be interpreter level.

So instead of 

class foo(bar baz):

we get

object foo class(bar baz):

which could be substituted with any wierd thing you want instead of "class" - 
like a quick module object creator with none of the overhead of a class, for 
example.  Or you could do the much coveted "instance" block that many people 
have clamored for, in which you subclass an instance instead of subclassing a 
class.  Or lets say you're making a game engine and you need to make a 
separate class system for engine objects (the situation I was in, actually)and 
you want to be able to work with them as if they were classes, but also want 
it to inherit some invisible engine settings not apparent to the python 
interpreter from other engine objects, as well as instead of making some kinds 
of functions visible to the interpreter they're loaded into the event handler 
automatically (this is personal experience here - I wanted these features very 
badly).

Or you could just make a quick dict maker

def dictblock(namespace, name, args):
    return namespace;

Like sort of the reverse of a "with" block.

Take note that this system would be by far the best way to handle the property 
object, rather than mucking about with descriptors.

Works as such...

object foo propertyblock:
    def fget(self):
         return "bar"
    def fset(self):
         pass
    def fdel(self):
         del self.foo
    doc = "baz"

(alternately, docstrings could be implemented like functions, filling the 
__doc__ entry of the dictionary)

and implemented like...

propertyblock(space, name, args):
    return property(space["fget"], space["fset"], space["fdel"], space["doc"])

You could do a nice shortcut involving 
passing the namespace dict into the keywords dict while calling "property", 
but that's the more legible way.

The point is that this would be distinct from the descriptor block concept, as 
descriptors don't need to play with the namespaces beyond one object and one 
name at a time.

But none of this explains why I like using a block for descriptors - I'm just 
trying to explain to those who thought of the possibilites of a block where 
you could play with the local and surrounding namespace how yes, that stuff 
would be useful, but would have a lot more overhead than simply a descriptor 
block.

There are places a multi-line descriptor block is ugly, but there are also 
places where it is goregeous.  Public and Private are good examples that C++ 
devs would love, rather than saying public and private for each function its

turn private:
    foo = 1
    bar = 2
    baz = 3

Hell, if you really don't like the keyword, we could get even wierder and play 
with a new syntax: use the @ with a bracket and nest the block within Guido's 
syntax...

@[static,
private,
functor]:
    def foo:
         pass

Wow, I covered way, way too much ground with that post.  Seriously people - 
read the Ruby tutorial "Programming Ruby" http://www.rubycentral.com/book/ to 
see the gorgeous way that Ruby handles the whole concept of blocks.  
Unfortunately the rest of the language is thoroughly illegible, but the block 
concept is cool.

From gareth.mccaughan at pobox.com  Sat Aug 14 03:18:36 2004
From: gareth.mccaughan at pobox.com (Gareth McCaughan)
Date: Sun Aug 15 22:40:56 2004
Subject: [Python-Dev] Re: def ... decorate
Message-ID: <200408140218.36912.gareth.mccaughan@pobox.com>

Josiah Carlson wrote, in reply to me:

> You seem to have missed the point of function decorations entirely.  We
> already have what you offer in current Python syntax.  The point was to
> move decorations to near/next to the function signature.

No, that was only part of the point. That it's part of the point
is why I still prefer the list-before-colon syntax, which
seems to me to go furthest towards keeping all information
about the interface of a function together. And yes, I know
that Guido finds that syntax unacceptable.

> Read the PEP:
> http://www.python.org/peps/pep-0318.html

I have read the PEP, thanks. Heck, I'm even named in it.
(I think I was the first person to propose a generalized
decoration facility for Python, rather than just special
syntax for a few special cases such as static methods.
And, for what it's worth, the list-before-colon syntax was
my proposal. Not that it matters much who thought of
what, but I do think it's a bit off to tell me that I've
"completely missed the point" of something I *invented*.)

>> I actually quite like this. It seems more Pythonic
>> than the @foo proposal. Its obvious problem is that
>> it involves something that looks at first glance like
>> an ordinary suite of statements or expressions, but
>> whose interpretation is substantially different. At
>> least the @foo proposal avoids that.
>
> What you like is what has existed with Python since the beginning.

No, it is not. What has existed with Python since the beginning
involves redundancy:

    def f(...):
        ...
    f = decorate_somehow(f)

requires three instances of the name of f. If you have N decorators
it requires 2N+1 instances of the name of f. Not a big deal when
the name is "f", but not so good if it's "build_interface_subgraph"
or something. Not even great when the name is "f": what if you
decide to rename it "g"? *And* it's ugly. These are reasons to want
something else, even without the issue of putting all the "interface"
information together.

-- 
g
From bob at redivi.com  Sun Aug 15 22:59:35 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Aug 15 22:59:34 2004
Subject: [Python-Dev] block-based decorators and other block issues
In-Reply-To: <1092399622.411cb206345b5@webmail.uoguelph.ca>
References: <1092399622.411cb206345b5@webmail.uoguelph.ca>
Message-ID: <034FE97C-EEFE-11D8-BF62-000A95686CD8@redivi.com>


On Aug 13, 2004, at 8:20 AM, Martin Zarate wrote:

> Our discussion has taken an interesting turn, leading to this concept
> of "super-decorators" that are more versatile than normal decorators, 
> because
> they can accept multiple objects, and because they can play with 
> namespaces.
---
> The reason is simple - speed.  A decorator shouldn't have to perform
> fasttolocals and vice versa.  It doesn't need to be its own variable
> namespace - it can sit within the external namespace.  To do otherwise 
> is
> suboptimal.  The decorator should not have its own scope.  Instead, it 
> should
> provide a simple filter.

Ok, I agree with you throughout this post except for this "speed" 
business.  Decorators are largely considered a compile-time feature and 
will almost exclusively be executed as module-level code.  Each 
'decorator statement' will likely only be executed once in its entire 
lifetime upon module import.  I don't think speed is too much of a 
concern at this point and worrying about it definitely seems like one 
heck of a premature optimization.

-bob
From raymond.hettinger at verizon.net  Sun Aug 15 23:41:26 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun Aug 15 23:42:40 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
Message-ID: <001501c48310$bf15aa40$e841fea9@oemcomputer>

The documentation for the atexit module (introduced in Py2.0) states:

"""Note: This module is unlikely to work correctly when used with other
code that sets sys.exitfunc. In particular, other core Python modules
are free to use atexit without the programmer's knowledge. Authors who
use sys.exitfunc should convert their code to use atexit instead. The
simplest way to convert code that sets sys.exitfunc is to import atexit
and register the function that had been bound to sys.exitfunc."""

Can we strengthen this by deprecating sys.exitfunc?

The atexit module does attempt to co-exist by introducing code to
register a function in sys.exitfunc if it were defined before "import
atexit" was called.  However, this is unreliable because it depends on
import order and the library is free to introduce an earlier "import
atexit" which would break code relying on the co-existance mechanism.


Raymond

From skip at pobox.com  Mon Aug 16 00:49:21 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 16 00:49:23 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: <001501c48310$bf15aa40$e841fea9@oemcomputer>
References: <001501c48310$bf15aa40$e841fea9@oemcomputer>
Message-ID: <16671.59505.127669.502898@montanaro.dyndns.org>


    Raymond> Can we strengthen this by deprecating sys.exitfunc?

What would you have the atexit module use?

Skip
From barry at python.org  Mon Aug 16 01:44:13 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 16 01:43:51 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def), with
	implementation
In-Reply-To: <E9AED893-ED7F-11D8-82BD-000A95686CD8@redivi.com>
References: <Pine.LNX.4.44.0408140006060.26777-100000@pingu.thwackety.com>
	<E9AED893-ED7F-11D8-82BD-000A95686CD8@redivi.com>
Message-ID: <1092613453.8173.108.camel@localhost>

On Fri, 2004-08-13 at 19:24, Bob Ippolito wrote:

> My only problem with this syntax is that I expect the common cases 
> (mine, anyway) to be zero or one decorators per function, so the extra 
> block and indent seems a bit excessive compared to the current 
> @prefix-symbol-decorator proposal(s).  It is, of course, far better 
> than typing the function name three times though! :)

Same here.  It trades one line at the end of the function for two (or
more) before the beginning of the function.

-0

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040815/23f479b1/attachment.pgp
From python at rcn.com  Mon Aug 16 01:57:18 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug 16 01:57:38 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: <16671.59505.127669.502898@montanaro.dyndns.org>
Message-ID: <001d01c48323$99a2e4e0$e841fea9@oemcomputer>

>     Raymond> Can we strengthen this by deprecating sys.exitfunc?
> 
> What would you have the atexit module use?

Of course, it would have to stick around in some form accessible by
atexit but not by everyone else.  I'm suggesting deprecating it from the
published API and possibly changing its name to an underscore or some
such.

Right now, the docs for sys.exitfunc mention that 'atexit' is a way to
handle multiple exit functions, but it stops short of saying not to use
sys.exitfunc at all.


Raymond

From ncoghlan at iinet.net.au  Mon Aug 16 01:58:55 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Mon Aug 16 01:58:59 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def),
	with	implementation
In-Reply-To: <1092613453.8173.108.camel@localhost>
References: <Pine.LNX.4.44.0408140006060.26777-100000@pingu.thwackety.com>	<E9AED893-ED7F-11D8-82BD-000A95686CD8@redivi.com>
	<1092613453.8173.108.camel@localhost>
Message-ID: <411FF8BF.70702@iinet.net.au>

Barry Warsaw wrote:

> On Fri, 2004-08-13 at 19:24, Bob Ippolito wrote:
> 
> 
>>My only problem with this syntax is that I expect the common cases 
>>(mine, anyway) to be zero or one decorators per function, so the extra 
>>block and indent seems a bit excessive compared to the current 
>>@prefix-symbol-decorator proposal(s).  It is, of course, far better 
>>than typing the function name three times though! :)
> 
> 
> Same here.  It trades one line at the end of the function for two (or
> more) before the beginning of the function.
> 
> -0

What about the variant which makes the block optional when there is only 
one decorator?

i.e.
   decorate: staticmethod
   def foobar():
     pass

The other thing I like about this version is that a folding editor like 
PythonWin can easily compress a decorated definition down to two lines, 
while leaving it clear that the function *is* decorated. That's not a 
huge point, though.

Cheers,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From python at rcn.com  Mon Aug 16 02:09:13 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug 16 02:09:33 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
Message-ID: <002001c48325$436d9460$e841fea9@oemcomputer>

Aahz had suggested that the threading section of the tutorial's Standard
Library Tour Part II be re-written with the idea of making people
smarter about what Python threading can and cannot do, and about
approaches most likely to assure success.

Please comment on the proposed revision listed below.



Raymond

------------------------------------------------------------------------
---

---------------
Multi-threading
---------------

Threading is a technique for decoupling tasks which are not sequentially
dependent and creating the illusion of concurrency.  Threads can be used
to improve the responsiveness of applications that accept user input
while other tasks run in the background.

The following code shows how the high level threading module can run
tasks in background while the main program continues to run:

    import threading, zipfile

    class AsyncZip(threading.Thread):
        def __init__(self, infile, outfile):
            threading.Thread.__init__(self)        
            self.infile = infile
            self.outfile = outfile
        def run(self):
            f = zipfile.ZipFile(self.outfile, 'w', zipfile.ZIP_DEFLATED)
            f.write(self.infile)
            f.close()
            print 'Finished background zip of: %s' % self.infile

    background = AsyncZip('mydata.txt', 'myarchive.zip')
    background.start()
    print 'The main program continues to run in foreground.'
    
    background.join()    # Wait for the background task to finish
    print 'Main program waited until background was done.'

The principal challenge of multi-thread applications is coordinating
threads that share data or other resources.  To that end, the threading
module provides a number of synchronization primitives including locks,
events, condition variables, and semaphores.

While those tools are powerful, minor design errors can result in
problems that are difficult to reproduce.  Hence, the preferred approach
to task coordination is to concentrate all access to a resource in a
single thread and then use the Queue module to feed that thread with
requests from other threads.  Applications using Queue objects for
inter-thread communication and coordination tend to be easier to design,
more readable, and more reliable.

All that being said, a few cautions are in order.  Thread programming is
difficult to get right.  And, its overhead decreases total application
performance.  Also, multiple processors cannot boost performance because
Python's Global Interpreter Lock (GIL) precludes more than one thread
from running in the interpreter at the same time (this was done to
simplify re-entrancy issues).  Another issue is that threading doesn't
work with the event driven model used by most GUIs.



From barry at python.org  Mon Aug 16 02:19:22 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 16 02:19:02 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def),
	with	implementation
In-Reply-To: <411FF8BF.70702@iinet.net.au>
References: <Pine.LNX.4.44.0408140006060.26777-100000@pingu.thwackety.com>
	<E9AED893-ED7F-11D8-82BD-000A95686CD8@redivi.com>
	<1092613453.8173.108.camel@localhost>  <411FF8BF.70702@iinet.net.au>
Message-ID: <1092615562.8192.168.camel@localhost>

On Sun, 2004-08-15 at 19:58, Nick Coghlan wrote:

> What about the variant which makes the block optional when there is only 
> one decorator?

Better, but I'm still not crazy about it, mostly because of the new
keyword.  I think you'd have to add an __future__ to get it into Python
2.4.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040815/bcc452ac/attachment.pgp
From greg at cosc.canterbury.ac.nz  Mon Aug 16 03:09:31 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug 16 03:10:00 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but
	aren't
In-Reply-To: <200408150633.i7F6XE203096@guido.python.org>
Message-ID: <200408160109.i7G19VHN019296@cosc353.cosc.canterbury.ac.nz>

Guido:

> But list and dict are *not* functions, they are types, and they can be
> used for subclassing and type checking too.  These are the precedent
> for placing generator etc. in __builtin__.

The real question seems to be whether __builtin__ should contain *all*
built-in types, including internal ones, or only those intended for
public use. Do you have an opinion about that, Guido?

I suppose it's reasonable to put them all in __builtin__, since as you
say, they can be useful for type checking even if you can't or rarely
want to instantiate them yourself.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From pje at telecommunity.com  Mon Aug 16 03:26:13 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug 16 03:26:47 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def),
	with	implementation
In-Reply-To: <1092615562.8192.168.camel@localhost>
References: <411FF8BF.70702@iinet.net.au>
	<Pine.LNX.4.44.0408140006060.26777-100000@pingu.thwackety.com>
	<E9AED893-ED7F-11D8-82BD-000A95686CD8@redivi.com>
	<1092613453.8173.108.camel@localhost> <411FF8BF.70702@iinet.net.au>
Message-ID: <5.1.1.6.0.20040815212353.02d49300@mail.telecommunity.com>

At 08:19 PM 8/15/04 -0400, Barry Warsaw wrote:
>On Sun, 2004-08-15 at 19:58, Nick Coghlan wrote:
>
> > What about the variant which makes the block optional when there is only
> > one decorator?
>
>Better, but I'm still not crazy about it, mostly because of the new
>keyword.  I think you'd have to add an __future__ to get it into Python
>2.4.

A slight variation of the J4 syntax could allow a single-decorator variant 
to look like:

     def foo(cls,bar) @classmethod as:
         # code here

versus this for multiple decorators:

     def foo(cls,bar)
         @classmethod
         @synchronized
     as:
         # code here

Neither variant needs a new keyword or __future__.

From tim.peters at gmail.com  Mon Aug 16 03:44:19 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug 16 03:44:25 2004
Subject: [Python-Dev] Another 2.4 asyncore headache
In-Reply-To: <e8bf7a530408141054ddec16@mail.gmail.com>
References: <1f7befae040813125824acdb87@mail.gmail.com>
	<20040813200347.GB29067@rogue.amk.ca>
	<1f7befae040813213728e50a44@mail.gmail.com>
	<e8bf7a530408141054ddec16@mail.gmail.com>
Message-ID: <1f7befae04081518441fd2c797@mail.gmail.com>

[Jeremy Hylton]
> I don't understand the use case or the correct intended use of
> asyncore <wink>, but
>    e = r + w
> looks appealing.  It allows object to be in the socket map without
> requiring that they be ready to do some kind of I/O at any instant
> that select is called.

It's enough that the Zope tests pass again, but I don't know Andrew's
intended use case.  The patch certainly changed visible semantics in
one of the major apps indulging major asyncore abuse.

Playing along with asyncore's emergent design, all along it passed
something X in the read set iff X.readable() said "yup", and likewise
for the write set wrt X.writable().  If there's a use case for passing
things in the exception set too, the backward-compatible way would
have been to do that iff a new X.exceptionable() said "yup", with a
default implementation that always said "no".  That would have slowed
asyncore some for everyone, but if the use case isn't strong enough to
justify a slowdown, perhaps the use case isn't strong enough period.
From tim.peters at gmail.com  Mon Aug 16 04:05:00 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug 16 04:05:11 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <002001c48325$436d9460$e841fea9@oemcomputer>
References: <002001c48325$436d9460$e841fea9@oemcomputer>
Message-ID: <1f7befae0408151905594f785c@mail.gmail.com>

I agree it would help to say more about threading pragmatics.  It's
hard to say anything true, though <wink>.  I'll just pick on this:

[Raymond Hettinger]
> ...
> And, its overhead decreases total application performance.  Also,
> multiple processors cannot boost performance because Python's
> Global Interpreter Lock (GIL) precludes more than one thread
> from running in the interpreter at the same time (this was done to
> simplify re-entrancy issues).

CPython's threading was really designed to allow I/O to proceed in
parallel with one thread of Python computation.  If used for that
purpose, it's not true that total app performance decreases, nor is it
true that performance can't improve on a multi-CPU box.  Performance
can increase significantly in that case, and that's the prime use case
for *practical* threading in Python (that's why the GIL is released
around virtually all C-level I/O calls -- I/O concurrency is the use
case Guido cared about).

Given experience with Zope and ZEO, I'd also add about 10 pages
explaining that trying to mix asyncore with threads is legally
sufficient evidence of insanity <0.9 wink>.
From tim.peters at gmail.com  Mon Aug 16 04:50:25 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug 16 04:50:27 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <20040812204431.GA31884@vicky.ecs.soton.ac.uk>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
	<20040812185521.GA2277@vicky.ecs.soton.ac.uk>
	<1f7befae04081212414007274f@mail.gmail.com>
	<20040812204431.GA31884@vicky.ecs.soton.ac.uk>
Message-ID: <1f7befae0408151950361f0cb4@mail.gmail.com>

[Armin Rigo]
> I don't remember if it was mentioned here, but maybe we'd better check
> directly whether the C stack is too large rather than (or possibly in addition
> to) using a limit on the number of Python iterations.

How large is too large?  The Windows way checks the stack for
impending overflow directly.  A long "stack check on Unix" thread here
in 2000 fizzled out in the expected way:  no two Unices appear to have
anything in common <0.9 wink>.

> This is not really ANSI C, but I can't think of a setting in which the
> following trick would fail to be a good indication of the amount of stack
> space consumed: save a pointer to a local variable "early", e.g. in
> Py_Initialize(); then in any other function call, the distance between this
> pointer and a pointer to some local equals the amount of stack space consumed
> by Python minus a few bytes.

It fails for threads, whose stacks may be anywhere in relation to the
main thread's.  So any gimmick like this has to have a thread-local
"starting point".  If "a stack" is always a contiguous slice of
memory, then it can work.

> If this sounds too much of a hack,

No, but I'm not sure how much additional hackery it requires to make
it work in all cases.

> the (usual) recursion limit could be kept to limit nested Python frames; but all
> C-level recursions (including eval_frame) could additionally use the above trick.
>  Its advantage is that it is an extremely fast check.

The ways to do it that don't work are indeed extremely fast <wink>.

> If the total stack size is difficult to know in advance,

That's apparently the case across platforms -- and someimes the
main-thread stack is much larger than secondary-thread stacks.  The
Windows check doesn't even try to know the current stack's size, but
relies on MS-specific C exception extensions.

> we can still use PyOS_CheckStack(), but more efficiently and less randomly
> than now, by maintaining a "high tide" pointer that records how much stack we
> are sure we have, and calling PyOS_CheckStack() only to attempt to push
> the "high tide" mark further.

Threads again complicate this.  AFAICT, the only platform that defines
USE_STACKCHECK now (and actually implements PyOS_CheckStack()) is
32-bit Windows using an MS compiler.  I did boost the stack
reservation for that platform in CVS, which can't solve it, but at
least hides it better <wink>.

> While I'm in a hacking mood, there might be a way to prevent PyErr_Clear() to
> clear away "asynchronuous" exceptions like RuntimeError, MemoryError, and
> KeyboardInterrupt: for these exceptions, let's just store them away instead of
> clearing them, and re-raise them at the next occasion.  The "next occasion"
> would have to be defined more precisely, but there is probably a way to ensure
> that it will at least be before the next non-trivial opcode operation starts.
> It would overwrite an exception set later, like those spurious KeyError we get
> for dict lookups.  It might be a better-than-nothing quick fix to all these
> PyErr_Clear() all around the code base.

Threads probably complicate that too.  It's dreadful that serious
problems can get transformed into bogus KeyErrors (in the test driver
I posted, I'm not sure I spelled this out, but the key always *was* in
the dict when a MemoryError got turned into a KeyError; we called a
comparison function to begin with because the hash codes matched, and
since these were mostly small-integer keys, they were identical to
their hash codes -- so these keys were in the dicts, and the KeyErrors
were lying about that).  That's just nuts.  I don't have spare cycles
to give to it, though, so you'll have to save the world on your own
again.
From stephen at xemacs.org  Mon Aug 16 05:38:30 2004
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon Aug 16 05:38:38 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <002001c48325$436d9460$e841fea9@oemcomputer> (Raymond
	Hettinger's message of "Sun, 15 Aug 2004 20:09:13 -0400")
References: <002001c48325$436d9460$e841fea9@oemcomputer>
Message-ID: <87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Raymond" == Raymond Hettinger <python@rcn.com> writes:

    Raymond> Threading is a technique for decoupling tasks which are
    Raymond> not sequentially dependent and creating the illusion of
    Raymond> concurrency.

All simultaneity is an illusion, but the concurrency of Python threads
is not. <wink>

    Raymond> Also, multiple processors cannot boost performance
    Raymond> because Python's Global Interpreter Lock (GIL) precludes
    Raymond> more than one thread from running in the interpreter at
    Raymond> the same time (this was done to simplify re-entrancy
    Raymond> issues).

This confused me because "running" is ambiguous.  How about something
like:

    The Python interpreter itself is not fully reentrant, so threading
    is accomplished by interleaving the execution of Python code from
    different threads.  Thus, use of multiple processors cannot boost
    performance of threaded Python code, because a single instance of
    the interpreter can only execute code on one processor at a time.

(Season to taste, fix technical inaccuracies.)  I see no reason to
mention the GIL, an implementation detail, at all.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.
From bob at redivi.com  Mon Aug 16 05:44:23 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug 16 05:45:00 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <002001c48325$436d9460$e841fea9@oemcomputer>
	<87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com>

On Aug 15, 2004, at 11:38 PM, Stephen J. Turnbull wrote:

>>>>>> "Raymond" == Raymond Hettinger <python@rcn.com> writes:
>
>     Raymond> Threading is a technique for decoupling tasks which are
>     Raymond> not sequentially dependent and creating the illusion of
>     Raymond> concurrency.
>
> All simultaneity is an illusion, but the concurrency of Python threads
> is not. <wink>
>
>     Raymond> Also, multiple processors cannot boost performance
>     Raymond> because Python's Global Interpreter Lock (GIL) precludes
>     Raymond> more than one thread from running in the interpreter at
>     Raymond> the same time (this was done to simplify re-entrancy
>     Raymond> issues).
>
> This confused me because "running" is ambiguous.  How about something
> like:
>
>     The Python interpreter itself is not fully reentrant, so threading
>     is accomplished by interleaving the execution of Python code from
>     different threads.  Thus, use of multiple processors cannot boost
>     performance of threaded Python code, because a single instance of
>     the interpreter can only execute code on one processor at a time.
>
> (Season to taste, fix technical inaccuracies.)  I see no reason to
> mention the GIL, an implementation detail, at all.

I think it's worth mentioning.  It's absolutely necessary to know all 
about the GIL when writing an extension that can take advantage of 
threading or embed Python correctly in a threaded application.

-bob
From jack at performancedrivers.com  Mon Aug 16 05:45:43 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Mon Aug 16 05:45:49 2004
Subject: [Python-Dev] Re: Decorator order implemented backwards?
In-Reply-To: <1092599162.29945.51.camel@localhost>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost>
	<411F0F2A.2070709@ocf.berkeley.edu>
	<1092583647.1670.18.camel@localhost>
	<eppstein-11FD65.10110815082004@sea.gmane.org>
	<1092599162.29945.51.camel@localhost>
Message-ID: <20040816034543.GR23725@performancedrivers.com>

On Sun, Aug 15, 2004 at 08:46:03PM +0100, Mark Russell wrote:
> Here's my current attempt at describing the semantics in the reference
> manual:
> 
>     A function definition may be wrapped by one or more decorator
>     expressions. Decorator expressions are evaluated when the function
>     is defined, in the scope that contains the function definition.
>     The result must be a callable, which is invoked with the function
>     object as the only argument.  The returned value is bound to the
>     function name instead of the function object.  Multiple decorators
>     are applied in nested fashion.  For example, the following code:
> 
> 	@f1(arg)
> 	@f2
> 	def func(): pass
> 
>     is equivalent to:
> 
> 	def func(): pass
> 	func = f1(arg)(f2(func))

Is func actually defined when f1 and f2 are called?  Looking at the
code the VAR_STORE doesn't happen until after the decorators are added.
[but I'm no expert on bytecode].  This won't matter for sane decorators
but we're talking about the strange cases already.  Reading the @ code top
to bottom you would not expect func to be defined, but if you want it to be
strictly like the 'func = decorate(func)' translation is should be.

My patch to add support for class decorators defines func before the
decorators as a side effect of moving Mark's decorator code out of 
com_funcdef (and yes, that was a plug).

-Jack


From aahz at pythoncraft.com  Mon Aug 16 05:48:17 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Aug 16 05:48:20 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <002001c48325$436d9460$e841fea9@oemcomputer>
References: <002001c48325$436d9460$e841fea9@oemcomputer>
Message-ID: <20040816034817.GA25038@panix.com>

On Sun, Aug 15, 2004, Raymond Hettinger wrote:
>
> All that being said, a few cautions are in order.  Thread programming is
> difficult to get right.  And, its overhead decreases total application
> performance.  Also, multiple processors cannot boost performance because
> Python's Global Interpreter Lock (GIL) precludes more than one thread
> from running in the interpreter at the same time (this was done to
> simplify re-entrancy issues).  Another issue is that threading doesn't
> work with the event driven model used by most GUIs.

Python threading certainly does work with GUIs, and while the GIL adds
some difficulty, the overall simplicity of Python's thread model makes
it easier to get threaded GUI programs correct.  See the Tkinter example
from my OSCON slides.

I'll also emphasize what Tim said: the GIL allows only one Python
thread, but any external library can release the GIL to perform threaded
operations in parallel.  In particular, I/O libraries almost always
release the GIL (I believe this includes mxODBC, for example).
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From bob at redivi.com  Mon Aug 16 05:55:05 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug 16 05:55:40 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <20040816034817.GA25038@panix.com>
References: <002001c48325$436d9460$e841fea9@oemcomputer>
	<20040816034817.GA25038@panix.com>
Message-ID: <0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com>


On Aug 15, 2004, at 11:48 PM, Aahz wrote:

> On Sun, Aug 15, 2004, Raymond Hettinger wrote:
>>
>> All that being said, a few cautions are in order.  Thread programming 
>> is
>> difficult to get right.  And, its overhead decreases total application
>> performance.  Also, multiple processors cannot boost performance 
>> because
>> Python's Global Interpreter Lock (GIL) precludes more than one thread
>> from running in the interpreter at the same time (this was done to
>> simplify re-entrancy issues).  Another issue is that threading doesn't
>> work with the event driven model used by most GUIs.
>
> Python threading certainly does work with GUIs, and while the GIL adds
> some difficulty, the overall simplicity of Python's thread model makes
> it easier to get threaded GUI programs correct.  See the Tkinter 
> example
> from my OSCON slides.

Many platform GUIs require that all or most all GUI activities be 
consolidated to one and only one thread.  I don't think any of them 
'care' if another thread is also running, but it can't communicate 
directly with the GUI.

-bob
From stephen at xemacs.org  Mon Aug 16 05:58:27 2004
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon Aug 16 05:58:34 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com> (Bob
	Ippolito's message of "Sun, 15 Aug 2004 23:44:23 -0400")
References: <002001c48325$436d9460$e841fea9@oemcomputer>
	<87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp>
	<900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com>
Message-ID: <874qn33fa4.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Bob" == Bob Ippolito <bob@redivi.com> writes:

    Bob> I think it's worth mentioning.  It's absolutely necessary to
    Bob> know all about the GIL when writing an extension that can
    Bob> take advantage of threading or embed Python correctly in a
    Bob> threaded application.

Different audience, no?

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.
From aahz at pythoncraft.com  Mon Aug 16 05:59:36 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Aug 16 05:59:38 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com>
References: <002001c48325$436d9460$e841fea9@oemcomputer>
	<20040816034817.GA25038@panix.com>
	<0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com>
Message-ID: <20040816035936.GB7737@panix.com>

On Sun, Aug 15, 2004, Bob Ippolito wrote:
>
> Many platform GUIs require that all or most all GUI activities be 
> consolidated to one and only one thread.  I don't think any of them 
> 'care' if another thread is also running, but it can't communicate 
> directly with the GUI.

That's true, but it also applies to other external resources, such as
files.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From bob at redivi.com  Mon Aug 16 06:33:48 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug 16 06:34:26 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <20040816035936.GB7737@panix.com>
References: <002001c48325$436d9460$e841fea9@oemcomputer>
	<20040816034817.GA25038@panix.com>
	<0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com>
	<20040816035936.GB7737@panix.com>
Message-ID: <77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com>


On Aug 15, 2004, at 11:59 PM, Aahz wrote:

> On Sun, Aug 15, 2004, Bob Ippolito wrote:
>>
>> Many platform GUIs require that all or most all GUI activities be
>> consolidated to one and only one thread.  I don't think any of them
>> 'care' if another thread is also running, but it can't communicate
>> directly with the GUI.
>
> That's true, but it also applies to other external resources, such as
> files.

Maybe I don't understand what you're saying.. but I don't know any 
platform that forces you to work with files from one and only one 
specific thread.  But I definitely do know at least one platform where 
you may only perform most GUI operations from the main thread of 
execution.

-bob
From foom at fuhm.net  Mon Aug 16 06:38:06 2004
From: foom at fuhm.net (James Y Knight)
Date: Mon Aug 16 06:38:19 2004
Subject: [Python-Dev] Re: Decorator order implemented backwards?
In-Reply-To: <20040816034543.GR23725@performancedrivers.com>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost>
	<411F0F2A.2070709@ocf.berkeley.edu>
	<1092583647.1670.18.camel@localhost>
	<eppstein-11FD65.10110815082004@sea.gmane.org>
	<1092599162.29945.51.camel@localhost>
	<20040816034543.GR23725@performancedrivers.com>
Message-ID: <117756C5-EF3E-11D8-8449-000A95A50FB2@fuhm.net>

On Aug 15, 2004, at 11:45 PM, Jack Diederich wrote:
> My patch to add support for class decorators defines func before the
> decorators as a side effect of moving Mark's decorator code out of
> com_funcdef (and yes, that was a plug).

That's definitely worse. It causes a possibly incorrect temporary value 
to be bound, even if only for a short amount of time. The current 
behavior of only doing the store after executing the decorators is 
cleaner.

I'd rewrite the translation as:
"""
The following code:
	@f1(arg)
	@f2
	def func(): return True

  is nearly* equivalent to:
	func = f1(arg)(f2(lambda : True))

* However, you don't have the syntactical restrictions of a lambda, and 
the function name gets set to "func" instead of "<lambda>".
"""

James

From rnd at onego.ru  Mon Aug 16 06:41:43 2004
From: rnd at onego.ru (Roman Suzi)
Date: Mon Aug 16 06:40:45 2004
Subject: [Python-Dev] More concerns about decorators
Message-ID: <Pine.LNX.4.58.0408160833450.1972@rnd.onego.ru>


After playing with decorators alittle I found that @-syntax is nearly optimal
especially that it comes before def, but there are other things to keep in
mind.

1. Decorators must transfer __doc__-strings and probably even
add information about functions

2. As now functions could have arbitrary attributes, some
mechanism must be in place to ensure this information is not lost.
If it is purely up to the decorator to decide, we will end up with
undocumented functions...

Is it possible to change function inplace? Is it even an option?

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From bob at redivi.com  Mon Aug 16 06:52:43 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug 16 06:53:18 2004
Subject: [Python-Dev] More concerns about decorators
In-Reply-To: <Pine.LNX.4.58.0408160833450.1972@rnd.onego.ru>
References: <Pine.LNX.4.58.0408160833450.1972@rnd.onego.ru>
Message-ID: <1BE57B0C-EF40-11D8-8D77-000A95686CD8@redivi.com>

On Aug 16, 2004, at 12:41 AM, Roman Suzi wrote:

>
> After playing with decorators alittle I found that @-syntax is nearly 
> optimal
> especially that it comes before def, but there are other things to 
> keep in
> mind.
>
> 1. Decorators must transfer __doc__-strings and probably even
> add information about functions
>
> 2. As now functions could have arbitrary attributes, some
> mechanism must be in place to ensure this information is not lost.
> If it is purely up to the decorator to decide, we will end up with
> undocumented functions...

def carryover(new, orig):
     if orig.__doc__:
         new.__doc__ = orig.__doc__
     if orig.__dict__:
         d = dict(orig.__dict__)
         d.update(new.__dict__)
         new.__dict__ = d
     return new

A built-in function like this would make it easy to preserve such 
information...

> Is it possible to change function inplace? Is it even an option?

Not sure what you mean.  Functions have a __doc__ slot and a __dict__ 
slot that are read/write.  __name__ is read-only.  I have no idea if 
that answers your question or not (there are other slots, that is not a 
definitive list, but should cover all of the relevant ones).

-bob
From foom at fuhm.net  Mon Aug 16 07:11:48 2004
From: foom at fuhm.net (James Y Knight)
Date: Mon Aug 16 07:11:54 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but
	aren't
In-Reply-To: <7FA48FA3-ECA6-11D8-BF5E-000A95A50FB2@fuhm.net>
References: <71A9AEA2-EA51-11D8-8D12-000A95A50FB2@fuhm.net>
	<2mekmf14jc.fsf@starship.python.net> <4118D57C.9090306@divmod.com>
	<2macx31423.fsf@starship.python.net>
	<200408101501.i7AF1HZ25725@guido.python.org>
	<BE05E963-EAE8-11D8-9899-000A95A50FB2@fuhm.net>
	<200408110002.i7B02gu26415@guido.python.org>
	<7FA48FA3-ECA6-11D8-BF5E-000A95A50FB2@fuhm.net>
Message-ID: <C6D2A78A-EF42-11D8-8449-000A95A50FB2@fuhm.net>

On Aug 12, 2004, at 5:28 PM, James Y Knight wrote:
> Okay, so, I don't have a patch

Submitted, http://www.python.org/sf/1009811

I'll note here, too, that I didn't add all the missing core types, only 
the ones that weren't iterators, descriptor internal stuff, or list 
comparison wrappers. For any of those that are at all possible to 
access from python code, I think they ought to show up somewhere, but 
I'm not convinced __builtin__ is the right place. Also the ones with 
"-" in their name need to be renamed.

> I have made a script to examine the situation.

Improved it slightly, and found another set of problems. Some modules 
have a function bound in place of the type:

E.g.

 >>> select.poll()
<select.poll object at 0x401cdea8>
 >>> isinstance(select.poll(), select.poll)
TypeError: isinstance() arg 2 must be a class, type, or tuple of 
classes and types
 >>> select.poll
<built-in function poll>
 >>> "D'oh!"

Here's the list of these:
select.poll    not a type.
mpz.mpz    not a type.
itertools.tee    not a type.
_csv.reader    not a type.
_csv.writer    not a type.
md5.md5    not a type.
cPickle.Pickler    not a type.
cPickle.Unpickler    not a type.
mmap.mmap    not a type.
xreadlines.xreadlines    not a type.

James

From python at rcn.com  Mon Aug 16 07:29:52 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug 16 07:30:20 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <1f7befae0408151905594f785c@mail.gmail.com>
Message-ID: <000b01c48352$0ed1bd80$e841fea9@oemcomputer>

> I agree it would help to say more about threading pragmatics.  It's
> hard to say anything true, though <wink>. 

After more thought, I'm leaving the existing tour mostly as-is and just
making minor fixups such as expanding the example.  The tour is supposed
to provide a brief introduction and not try to cover all that is known.


Instead, the Library Reference on threading should have extra sections
on threading pragmatics (especially with respect to GUIs) and some
non-toy examples to get someone started.  Also, Extending and Embedding
ought to cover the GIL, its significance, how to release/acquire, and
the implications for multi-processors, etc.

Several of the topics exceed my experience and some really need the
collective experiences of various implementers, teachers, etc.  So, it
might be worthwhile to have a wiki to collect the useful thoughts and
use that as a basis for improving the rest of the docs.

The threading docs are rather like a car manual that presumes that you
already know how to drive.



Raymond

From jack at performancedrivers.com  Mon Aug 16 07:35:40 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Mon Aug 16 07:35:45 2004
Subject: [Python-Dev] Re: Decorator order implemented backwards?
In-Reply-To: <117756C5-EF3E-11D8-8449-000A95A50FB2@fuhm.net>
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost>
	<411F0F2A.2070709@ocf.berkeley.edu>
	<1092583647.1670.18.camel@localhost>
	<eppstein-11FD65.10110815082004@sea.gmane.org>
	<1092599162.29945.51.camel@localhost>
	<20040816034543.GR23725@performancedrivers.com>
	<117756C5-EF3E-11D8-8449-000A95A50FB2@fuhm.net>
Message-ID: <20040816053540.GS23725@performancedrivers.com>

On Mon, Aug 16, 2004 at 12:38:06AM -0400, James Y Knight wrote:
> On Aug 15, 2004, at 11:45 PM, Jack Diederich wrote:
> >My patch to add support for class decorators defines func before the
> >decorators as a side effect of moving Mark's decorator code out of
> >com_funcdef (and yes, that was a plug).
> 
> That's definitely worse. It causes a possibly incorrect temporary value 
> to be bound, even if only for a short amount of time. The current 
> behavior of only doing the store after executing the decorators is 
> cleaner.

I agree it would be unexpected if a function defined two lines down 
happens before the @mutate above it.

> I'd rewrite the translation as:
> """
> The following code:
> 	@f1(arg)
> 	@f2
> 	def func(): return True
> 
>  is nearly* equivalent to:
> 	func = f1(arg)(f2(lambda : True))
> 
> * However, you don't have the syntactical restrictions of a lambda, and 
> the function name gets set to "func" instead of "<lambda>".
> """

Yes, the docs just need some kind of asterisk about the evaluation order.
I was just pointing out a minor documentation bug.  Mainly I just wanted to
say "class decorators please" and to say I've submitted and would be happy
to submit more patches to make it happen (for '@' or anything else).
I currently write:

class FooOverTimeReport(object):
  __metaclass__ = RunDaily # works OK unless you need more than one
  ...

when I mean

@run_daily
class FooOverTimeReport(object):
  ...


-Jack
From jack at performancedrivers.com  Mon Aug 16 07:42:58 2004
From: jack at performancedrivers.com (Jack Diederich)
Date: Mon Aug 16 07:43:03 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <000b01c48352$0ed1bd80$e841fea9@oemcomputer>
References: <1f7befae0408151905594f785c@mail.gmail.com>
	<000b01c48352$0ed1bd80$e841fea9@oemcomputer>
Message-ID: <20040816054258.GT23725@performancedrivers.com>

On Mon, Aug 16, 2004 at 01:29:52AM -0400, Raymond Hettinger wrote:
> > I agree it would help to say more about threading pragmatics.  It's
> > hard to say anything true, though <wink>. 
> 
> After more thought, I'm leaving the existing tour mostly as-is and just
> making minor fixups such as expanding the example.  The tour is supposed
> to provide a brief introduction and not try to cover all that is known.
> 
> Several of the topics exceed my experience and some really need the
> collective experiences of various implementers, teachers, etc.  So, it
> might be worthwhile to have a wiki to collect the useful thoughts and
> use that as a basis for improving the rest of the docs.

One of many GUI examples, lyntin uses a Queue.Queue for synchronization
between the tk/wx/whatever GUI thread, the telnet thread, and execution 
including interactive debugger goodness.  http://sf.net/projects/lyntin/

-Jack
From fumanchu at amor.org  Mon Aug 16 09:48:31 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Mon Aug 16 09:53:46 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def),
	with	implementation
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DFC@exchange.hqamor.amorhq.net>

Barry Warsaw wrote:
> On Sun, 2004-08-15 at 19:58, Nick Coghlan wrote:
> 
> > What about the variant which makes the block optional when 
> > there is only one decorator?
> 
> Better, but I'm still not crazy about it, mostly because of the new
> keyword.  I think you'd have to add an __future__ to get it 
> into Python 2.4.

...which Guido hasn't completely ruled out (unlike most of the other
'potential' alternatives):
http://mail.python.org/pipermail/python-dev/2004-August/047001.html .
Personally, I feel decorators are powerful enough to warrant a more
structured change process like __future__; it seemed to work quite well
for generators. Perhaps any new feature which ends in "-rators" warrants
it. ;)


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From gjc at inescporto.pt  Mon Aug 16 12:35:33 2004
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Mon Aug 16 12:35:41 2004
Subject: [Python-Dev] Find out whether PyEval_InitThreads has been called?
Message-ID: <1092652532.1820.8.camel@emperor>

Short version:

  Can I add a variable to python to indicate whether PyEval_InitThreads
has been called?  Something like:

	extern int PyEval_ThreadsInitialized;

Motivation:

  I have discovered that, in pygtk, performance decreases a lot (50-90%
higher exectution time) when we use python locking.  We have an internal
boolean flag in pygtk to indicate whether pygtk threading was enabled or
not, thus avoiding the GIL locking functions in single threaded
programs.

  The problem is that this flag is local to pygtk.  If another module
activates threading in python, we would like to know it in order to
start acquiring the GIL.

  This should be a very simple patch, with zero impact in python core,
but it will help tremendously.  Is likely to be accepted for python 2.4?

  Regards.

PS: I already asked python-help about this, sorry for the duplicate
mails.

-- 
Gustavo J. A. M. Carneiro
<gjc@inescporto.pt> <gustavo@users.sourceforge.net>
The universe is always one step beyond logic

From gjc at inescporto.pt  Mon Aug 16 12:43:09 2004
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Mon Aug 16 12:43:23 2004
Subject: [Python-Dev] Find out whether PyEval_InitThreads has been called?
In-Reply-To: <1092652532.1820.8.camel@emperor>
References: <1092652532.1820.8.camel@emperor>
Message-ID: <1092652989.1814.13.camel@emperor>

A Seg, 2004-08-16 ?s 11:35, Gustavo J. A. M. Carneiro escreveu:
> Short version:
> 
>   Can I add a variable to python to indicate whether PyEval_InitThreads
> has been called?  Something like:
> 
> 	extern int PyEval_ThreadsInitialized;
> 
> Motivation:
> 
>   I have discovered that, in pygtk, performance decreases a lot (50-90%
> higher exectution time) when we use python locking.  We have an internal
> boolean flag in pygtk to indicate whether pygtk threading was enabled or
> not, thus avoiding the GIL locking functions in single threaded
> programs.
> 
>   The problem is that this flag is local to pygtk.  If another module
> activates threading in python, we would like to know it in order to
> start acquiring the GIL.
> 
>   This should be a very simple patch, with zero impact in python core,
> but it will help tremendously.  Is likely to be accepted for python 2.4?

  Forget it.  Why not go one step further and make all locking
operations in python no-ops if PyEval_InitThreads has never been
called?  Currently, we risk python crashing if we call such functions
without calling PyEval_InitThreads.  This is also trivial to implement.

  Regards.

-- 
Gustavo J. A. M. Carneiro
<gjc@inescporto.pt> <gustavo@users.sourceforge.net>
The universe is always one step beyond logic

From Michael.Sparks at rd.bbc.co.uk  Mon Aug 16 12:55:57 2004
From: Michael.Sparks at rd.bbc.co.uk (Michael Sparks)
Date: Mon Aug 16 13:01:37 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def),
	with implementation
Message-ID: <200408161155.57554.Michael.Sparks@rd.bbc.co.uk>

Nick Coghlan> [make block optional for one decorator ?]
Barry Warsaw> [Better, but requires keyword, and hence (?) __future__ ]

Robert Brewer wrote:
> ...which Guido hasn't completely ruled out (unlike most of the other
> 'potential' alternatives):
> http://mail.python.org/pipermail/python-dev/2004-August/047001.html .

It was (partly) reading this post that made me think it worthwhile 
implementing this syntax. I actually prefer keyword inside the function 
block, but I know that's ruled out, and can see why. As a result for me this 
is my second choice. Sure, I can live with the @pie syntax, but *personally* 
I prefer having a keyword.

As previously noted the arguments against the length of the decorator in the 
single case, I view as advantages when showing code to new users or 
maintainers.

As for whether it's the right keyword or not, I'm not sure I'm qualified to 
say, but decorators seem much more like declarations of function properties 
in all the use cases I've seen.

> Personally, I feel decorators are powerful enough to warrant a more
> structured change process like __future__; it seemed to work quite well
> for generators. Perhaps any new feature which ends in "-rators" warrants
> it. ;)

As a circular use case, I realised that the generator implementation in 
test3.py from Armin Rigo's greenlets package would allow one to write:

decorate:
   generator
def some function(bla):
   for i in xrange(1):
      Yield(X)

So you *could* argue the case for "swapping" keywords... ;-) (not serious :)

More seriously though, if there is interest I'll see what I can do about 
cleaning up the patch to resolve the scoping issue.

Also I just realised that I didn't introduce myself (as per list 
instructions...). I don't know if this is still in vogue or not, but here 
goes...

I've been using python for 2 years now, and prior to that used perl/C/C++
for around 4 years as my main languages. I've worked on large scale 
network systems for most of my working life, but also worked on 3D 
graphics, and translation between a high level language to asynchronous
hardware (handshake circuits) code generator. I've also contributed to 
a variety of F/OSS projects (in different ways!) in the past including squid,
linux virtual servers, and Vorbis, largely because I think it's one of the
nicest way of saying "thank you" :)

I also like B5, Cordwainer Smith and I'm scared of spiders :)

Regards,


Michael.
-- 
Michael Sparks, Senior R&D Engineer, Digital Media Group
Michael.Sparks@rd.bbc.co.uk,
British Broadcasting Corporation, Research and Development
Kingswood Warren, Surrey KT20 6NP

This e-mail may contain personal views which are not the views of the BBC.
From arigo at tunes.org  Mon Aug 16 13:29:16 2004
From: arigo at tunes.org (Armin Rigo)
Date: Mon Aug 16 13:35:36 2004
Subject: [Python-Dev] Re: Another test_compiler mystery
In-Reply-To: <1f7befae0408151950361f0cb4@mail.gmail.com>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
	<20040812185521.GA2277@vicky.ecs.soton.ac.uk>
	<1f7befae04081212414007274f@mail.gmail.com>
	<20040812204431.GA31884@vicky.ecs.soton.ac.uk>
	<1f7befae0408151950361f0cb4@mail.gmail.com>
Message-ID: <20040816112916.GA19969@vicky.ecs.soton.ac.uk>

Hello Tim,

On Sun, Aug 15, 2004 at 10:50:25PM -0400, Tim Peters wrote:
> > While I'm in a hacking mood, there might be a way to prevent PyErr_Clear()
> > clear away "asynchronuous" exceptions like RuntimeError, MemoryError, and
> > KeyboardInterrupt:  (...)
> 
> Threads probably complicate that too.  It's dreadful that serious
> problems can get transformed into bogus KeyErrors

Yes.  Here is a patch attempting to do what I described:
 http://www.python.org/sf/1009929

It's an extension of the asynchronous exception mecanism used to signal
between threads.  PyErr_Clear() can send some exceptions to its own thread
using this mecanism.  (So it is thread-safe.)


Armin
From arigo at tunes.org  Mon Aug 16 13:33:57 2004
From: arigo at tunes.org (Armin Rigo)
Date: Mon Aug 16 13:38:44 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: <001501c48310$bf15aa40$e841fea9@oemcomputer>
References: <001501c48310$bf15aa40$e841fea9@oemcomputer>
Message-ID: <20040816113357.GB19969@vicky.ecs.soton.ac.uk>

Hello Raymond,

On Sun, Aug 15, 2004 at 05:41:26PM -0400, Raymond Hettinger wrote:
> The atexit module does attempt to co-exist by introducing code to
> register a function in sys.exitfunc if it were defined before "import
> atexit" was called.  However, this is unreliable because it depends on
> import order and the library is free to introduce an earlier "import
> atexit" which would break code relying on the co-existance mechanism.

Multiple well-written modules can use sys.exitfunc in a way that is
compatible.  atexit is merely a glue to discourage non-well-written modules.  
I'd say that atexit should be encouraged but there is nothing wrong with using
sys.exitfunc directly, if one is aware of the common patching pattern:
"save-old-value, replace-with-my-function, upon-call-invoke-saved-old-value".


A bient?t,

Armin.
From mcherm at mcherm.com  Mon Aug 16 14:37:47 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Aug 16 14:35:53 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
Message-ID: <1092659867.4120aa9ba1239@mcherm.com>

> > > > if the only reason for it is to use isinstance?
> > >
> > > So that an extension author *could* write an int-like type deriving
> > > from it?
> >
> > But didn't you just say that people shouldn't be
> > deriving their own int-like types from baseinteger?
>
> Indeed, in general they shouldn't.  But for specialized purposes it
> might be needed (that's why I emphasized *could*).

I call YAGNI. We're talking about creating the class baseinteger
which might be useful ONLY for people creating new kinds of integers
in Python which will NOT extend int or long but WILL need to be
treated just like integers. Who is really likely to do that? And if
in the process we introduce a new class which won't be needed in
the long run (ie Python 3000 has just one type, called "int" and has
no need for baseinteger). So I maintain that it's not needed (and
is, in fact, confusing to users) unless someone has a real use case.

-- Michael Chermside

From mwh at python.net  Mon Aug 16 14:40:50 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug 16 14:40:52 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules
	readline.c, 2.72, 2.73
In-Reply-To: <E1BwM3N-0007X0-6P@sc8-pr-cvs1.sourceforge.net>
	(montanaro@users.sourceforge.net's
	message of "Sun, 15 Aug 2004 07:32:09 -0700")
References: <E1BwM3N-0007X0-6P@sc8-pr-cvs1.sourceforge.net>
Message-ID: <2md61rcl2l.fsf@starship.python.net>

montanaro@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Modules
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28578/Modules
>
> Modified Files:
> 	readline.c 
> Log Message:
> Add get_history_item and replace_history_item functions to the readline
> module.  Closes patch #675551.  My apologies to Michal Vitecek for taking so
> long to process this.

A few small quibbles...

>
> Index: readline.c
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Modules/readline.c,v
> retrieving revision 2.72
> retrieving revision 2.73
> diff -C2 -d -r2.72 -r2.73
> *** readline.c	8 Jul 2004 15:28:18 -0000	2.72
> --- readline.c	15 Aug 2004 14:31:57 -0000	2.73
> ***************
> *** 295,298 ****
> --- 295,363 ----
>   set the readline word delimiters for tab-completion");
>   
> + static PyObject *
> + py_remove_history(PyObject *self, PyObject *args)
> + {
> +         int entry_number;
> +         HIST_ENTRY *entry;
> + 
> +         if (!PyArg_ParseTuple(args, "i:remove_history", &entry_number))
> +                 return NULL;
> +         entry = remove_history(entry_number);
> +         if (!entry) {
> +                 char buf[80];
> +                 PyOS_snprintf(buf, sizeof(buf),
> +                               "No history item at position %i",
> +                               entry_number);
> +                 PyErr_SetString(PyExc_ValueError, buf);
> +                 return NULL;
> +         }

What's wrong with PyErr_Format()?

> +         /* free memory allocated for the history entry */
> +         if (entry->line)
> +                 free(entry->line);
> +         if (entry->data)
> +                 free(entry->data);
> +         free(entry);
> + 
> +         Py_INCREF(Py_None);
> +         return Py_None;
> + }
> + 
> + PyDoc_STRVAR(doc_remove_history,
> + "remove_history(pos) -> None\n\
     ^^^^^^^^^^^^^^
This isn't the name in the PyMethodDef at the end...

[snippity]

The same comments apply to the other function, too.

>   /* Add a line to the history buffer */
> ***************
> *** 494,497 ****
> --- 559,564 ----
>   	 METH_VARARGS, doc_set_completer_delims},
>   	{"add_history", py_add_history, METH_VARARGS, doc_add_history},
> +         {"remove_history_item", py_remove_history, METH_VARARGS, doc_remove_history},
              ^^^^^^^^^^^^^^^^^^^
> +         {"replace_history_item", py_replace_history, METH_VARARGS, doc_replace_history},
>   	{"get_completer_delims", get_completer_delims,
>   	 METH_NOARGS, doc_get_completer_delims},

Cheers,
mwh

-- 
  <dash> it's like a bicycle
  <dash> but with internet                      -- from Twisted.Quotes
From mwh at python.net  Mon Aug 16 15:05:38 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug 16 15:05:40 2004
Subject: [Python-Dev] More concerns about decorators
In-Reply-To: <1BE57B0C-EF40-11D8-8D77-000A95686CD8@redivi.com> (Bob
	Ippolito's message of "Mon, 16 Aug 2004 00:52:43 -0400")
References: <Pine.LNX.4.58.0408160833450.1972@rnd.onego.ru>
	<1BE57B0C-EF40-11D8-8D77-000A95686CD8@redivi.com>
Message-ID: <2m8ycfcjx9.fsf@starship.python.net>

Bob Ippolito <bob@redivi.com> writes:

>> Is it possible to change function inplace? Is it even an option?
>
> Not sure what you mean.  Functions have a __doc__ slot and a __dict__
> slot that are read/write.  __name__ is read-only.

__name__ is writable as of a few days ago.

Cheers,
mwh

-- 
  I believe C++ instills fear in programmers, fear that the
  interaction of some details causes unpredictable results.
                                        -- Erik Naggum, comp.lang.lisp
From fumanchu at amor.org  Mon Aug 16 17:19:10 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Mon Aug 16 17:24:33 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def),
	with implementation
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022DFD@exchange.hqamor.amorhq.net>

Michael Sparks wrote:
> Nick Coghlan> [make block optional for one decorator ?]
> Barry Warsaw> [Better, but requires keyword, and hence (?) 
> __future__ ]
> 
> As previously noted the arguments against the length of the 
> decorator in the 
> single case, I view as advantages when showing code to new users or 
> maintainers.

If you allow the keyword and the single decorator on the same line, this
argument comes down to "how long is the keyword?"

>>> try: a = 3
... except Exception, x:
... 	print "Error", x

Nobody seems to have a problem typing "try:" again and again. I'm not
sure why "decorate:" is receiving such pessimistic scrutiny. But then, I
wasn't hanging about when try/except came into being--perhaps it had
similar difficulties.

> As for whether it's the right keyword or not, I'm not sure 
> I'm qualified to 
> say, but decorators seem much more like declarations of 
> function properties 
> in all the use cases I've seen.

The proposal, in effect, creates a declarative suite as opposed to the
more traditional imperative suite. Perhaps "declare:" would be more
appropriate (but give your brain a week to mull it over ;).

> More seriously though, if there is interest I'll see what I 
> can do about cleaning up the patch to resolve the scoping issue.

I'm certainly interested; thanks for churning out an implementation.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From anthony at interlink.com.au  Mon Aug 16 17:31:11 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Aug 16 17:31:45 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com>
References: <002001c48325$436d9460$e841fea9@oemcomputer>	<20040816034817.GA25038@panix.com>	<0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com>	<20040816035936.GB7737@panix.com>
	<77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com>
Message-ID: <4120D33F.8080900@interlink.com.au>

Bob Ippolito wrote:
> 
> Maybe I don't understand what you're saying.. but I don't know any 
> platform that forces you to work with files from one and only one 
> specific thread.  But I definitely do know at least one platform where 
> you may only perform most GUI operations from the main thread of execution.

There's a number of GUIs in this boat. It's not too much hassle - you
just have to make sure you handle events to and from the GUI correctly
through some sort of intermediary that posts the events to the correct
thread. An example of this is wx and twisted - the only sane path is
to run each in it's own thread. wx also won't work unless the "main"
thread is the one running the GUI.

all-guis-suck,
Anthony


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From pje at telecommunity.com  Mon Aug 16 17:38:58 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug 16 17:39:37 2004
Subject: [Python-Dev] Re: Decorator order implemented backwards?
In-Reply-To: <117756C5-EF3E-11D8-8449-000A95A50FB2@fuhm.net>
References: <20040816034543.GR23725@performancedrivers.com>
	<B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost>
	<411F0F2A.2070709@ocf.berkeley.edu>
	<1092583647.1670.18.camel@localhost>
	<eppstein-11FD65.10110815082004@sea.gmane.org>
	<1092599162.29945.51.camel@localhost>
	<20040816034543.GR23725@performancedrivers.com>
Message-ID: <5.1.1.6.0.20040816113536.01ee7260@mail.telecommunity.com>

At 12:38 AM 8/16/04 -0400, James Y Knight wrote:
>On Aug 15, 2004, at 11:45 PM, Jack Diederich wrote:
>>My patch to add support for class decorators defines func before the
>>decorators as a side effect of moving Mark's decorator code out of
>>com_funcdef (and yes, that was a plug).
>
>That's definitely worse. It causes a possibly incorrect temporary value to 
>be bound, even if only for a short amount of time. The current behavior of 
>only doing the store after executing the decorators is cleaner.

Not only that, it's the behavior *required* by the PEP.  Note that it 
states "*without* the intermediate assignment to the variable 
func".  (Emphasis added.)

This choice of semantics is intentional, to allow for things like Ed 
Loper's property_getter/property_setter and generic functions to be 
possible.  That is, it's intended to allow for decorators that construct an 
object from multiple function definitions.  Such decorators need to be able 
to get the previous object bound to that name, and that's not possible if 
the new function has been saved under that name.

There should probably be some kind of test added to the test suite to 
verify this behavior, and adding an explanation to the PEP is probably in 
order as well.

From bob at redivi.com  Mon Aug 16 17:44:37 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug 16 17:45:30 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <4120D33F.8080900@interlink.com.au>
References: <002001c48325$436d9460$e841fea9@oemcomputer>	<20040816034817.GA25038@panix.com>	<0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com>	<20040816035936.GB7737@panix.com>
	<77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com>
	<4120D33F.8080900@interlink.com.au>
Message-ID: <2D8666C4-EF9B-11D8-A746-000A95686CD8@redivi.com>


On Aug 16, 2004, at 11:31 AM, Anthony Baxter wrote:

> Bob Ippolito wrote:
>> Maybe I don't understand what you're saying.. but I don't know any 
>> platform that forces you to work with files from one and only one 
>> specific thread.  But I definitely do know at least one platform 
>> where you may only perform most GUI operations from the main thread 
>> of execution.
>
> There's a number of GUIs in this boat. It's not too much hassle - you
> just have to make sure you handle events to and from the GUI correctly
> through some sort of intermediary that posts the events to the correct
> thread. An example of this is wx and twisted - the only sane path is
> to run each in it's own thread. wx also won't work unless the "main"
> thread is the one running the GUI.

But in those cases, Twisted only needs the mainloop to do two things: 
fire timers, watch file descriptors.  It's possible to write a reactor 
for many or most GUIs that does that while still remaining single 
threaded (as far as Python is concerned).  For GUIs that don't natively 
"watch sockets" as part of their runloop, you can start a thread that 
does that and only that and send asynchronous notifications to the GUI 
mainloop (which is what Cocoa/CoreFoundation does behind your back with 
their networking APIs).

-bob
From anthony at interlink.com.au  Mon Aug 16 17:52:45 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Aug 16 17:52:56 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <2D8666C4-EF9B-11D8-A746-000A95686CD8@redivi.com>
References: <002001c48325$436d9460$e841fea9@oemcomputer>	<20040816034817.GA25038@panix.com>	<0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com>	<20040816035936.GB7737@panix.com>
	<77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com>
	<4120D33F.8080900@interlink.com.au>
	<2D8666C4-EF9B-11D8-A746-000A95686CD8@redivi.com>
Message-ID: <4120D84D.70703@interlink.com.au>

Bob Ippolito wrote:
> But in those cases, Twisted only needs the mainloop to do two things: 
> fire timers, watch file descriptors.  It's possible to write a reactor 
> for many or most GUIs that does that while still remaining single 
> threaded (as far as Python is concerned).  For GUIs that don't natively 
> "watch sockets" as part of their runloop, you can start a thread that 
> does that and only that and send asynchronous notifications to the GUI 
> mainloop (which is what Cocoa/CoreFoundation does behind your back with 
> their networking APIs).

Unfortunately in the wx case, the timer granularity sucks. The wx docs
guarantee "no worse than 1s granularity". It's not _that_ bad, but on
Windows at least, it's pretty terrible - at best, only 20-30Hz. I'm
depressingly sure that there's other GUI toolkits with similar levels
of suck.

(Short (off-topic) summary - there's two ways to make the twisted and
wx event loops cooperate - either the wx event loop runs everything,
and fires timers to let the twisted event loop do it's thing - see
above. The second is to let the twisted event loop regularly call
the wx event loop to let it iterate any outstanding events. When
a popup or menu comes up, wx creates a new sub-event-loop and takes
control until that's dismissed... so we're back to needing to run
the two event loops alongside each other. And yes, Tim's comment
about insanity resulting from using asyncore and threads together
also apply to twisted - the only sane way to do it is to have all
the event loop stuff handled in the same thread. Madness follows,
otherwise.)

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From rnd at onego.ru  Mon Aug 16 17:39:54 2004
From: rnd at onego.ru (Roman Suzi)
Date: Mon Aug 16 18:04:17 2004
Subject: [Python-Dev] More concerns about decorators
In-Reply-To: <2m8ycfcjx9.fsf@starship.python.net>
References: <Pine.LNX.4.58.0408160833450.1972@rnd.onego.ru>
	<1BE57B0C-EF40-11D8-8D77-000A95686CD8@redivi.com>
	<2m8ycfcjx9.fsf@starship.python.net>
Message-ID: <Pine.LNX.4.58.0408161912070.2981@rnd.onego.ru>

On Mon, 16 Aug 2004, Michael Hudson wrote:

>Bob Ippolito <bob@redivi.com> writes:
>
>>> Is it possible to change function inplace? Is it even an option?
>>
>> Not sure what you mean.  Functions have a __doc__ slot and a __dict__
>> slot that are read/write.  __name__ is read-only.
>
>__name__ is writable as of a few days ago.

I mean that decorators are... decorators. They by their meaning must not
change the function but add some twists to their behaviour. Many of decorator
examples are destructing function attributes.

Good example of non-destructing decorator is staticmethod:

>>> class A:
...     def m(x):
...          """I am method m"""
...     m.__security__ = "low"
...     m = staticmethod(m)
...
>>> A.m.__doc__
'I am method m'
>>> A.m.__security__
'low'

So, my concern was that it requires additional efforts to save
function attributes. My thought was that probably decorators could
work directly on the function, not changing id of the function?
(staticmethod changes id, of course). Why? Because the function
is not accessible any more. while the old way it was possible to store
intermediate result:

class A:
     def m(x):
        """I am method m"""
     mm = m
     m = staticmethod(m)


And use it somehow. But with decorator function changes. Yes, it is possible
that some decorators could store function internally for the purpose of
calling it, but others will not.

The transfer of function attributes is on the developer.  And he/she could be
clueless that docstrings, etc, could be lost in transformation.


Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From skip at pobox.com  Mon Aug 16 17:51:06 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 16 18:09:59 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
Message-ID: <16672.55274.689761.740267@montanaro.dyndns.org>


I think I might have reported this previously (long time ago), but I don't
recall if it was ever resolved.  (Apparently not, since I'm still seeing
it.)

I'm seeing a test failure in test_tempfile on Mac OSX (I'm running
10.2.8/Jaguar) with a unix (not framework) build.  I've whittled the tests
down to sundry, tcl and tempfile.  Here's the full run:

    % ./python.exe Lib/test/regrtest.py -f testrun 
    test_sundry
    test_tcl
    test_tempfile
    *** malloc[17153]: error for object 0x2d70808: Pointer being reallocated was not allocated
    [5999 refs]
    test test_tempfile failed -- Traceback (most recent call last):
      File "/Users/skip/tmp/python/dist/src/Lib/test/test_tempfile.py", line 155, in test_wanted_dirs
        os.environ[envname] = os.path.abspath(envname)
      File "/Users/skip/tmp/python/dist/src/Lib/os.py", line 447, in __setitem__
        putenv(key, item)
    OSError: (0, 'Error')

    2 tests OK.
    1 test failed:
        test_tempfile
    [39478 refs]

This is a debug build linked with /usr/lib/libMallocDebug.a.  Running under
gdb's control with a breakpoint in malloc_printf yields this traceback info:

    (gdb) bt 10
    #0  0x90070ddc in malloc_printf ()
    #1  0x9000de5c in szone_realloc ()
    #2  0x9000dd70 in malloc_zone_realloc ()
    #3  0x9000dce8 in realloc ()
    #4  0x90010c24 in setenv ()
    #5  0x90010a74 in putenv ()
    #6  0x00131018 in posix_putenv (self=0x0, args=0x12ac678) at Modules/posixmodule.c:5695
    #7  0x000db4fc in PyCFunction_Call (func=0x3e25f8, arg=0x12ac678, kw=0x0) at Objects/methodobject.c:73
    #8  0x000ce2f4 in call_function (pp_stack=0xbfff94f0, oparg=2) at Python/ceval.c:3566
    #9  0x000c8624 in PyEval_EvalFrame (f=0xa05108) at Python/ceval.c:2170
    ...

In frame 6, putenv() is being called:

    (gdb) fr 6
    #6  0x00131018 in posix_putenv (self=0x0, args=0x12ac678) at Modules/posixmodule.c:5695
    5695            if (putenv(new)) {
    (gdb) p new
    $1 = 0x109a5e4 "TMPDIR=/Users/skip/tmp/python/dist/src/TMPDIR"

This looks reasonable to me, so it doesn't look like the problem is within
Python.  Should I just skip the tempfile test on Mac OSX?  Can someone check
this out on 10.3/Panther?

Skip
From skip at pobox.com  Mon Aug 16 18:00:22 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 16 18:10:05 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules
	readline.c, 2.72, 2.73
In-Reply-To: <2md61rcl2l.fsf@starship.python.net>
References: <E1BwM3N-0007X0-6P@sc8-pr-cvs1.sourceforge.net>
	<2md61rcl2l.fsf@starship.python.net>
Message-ID: <16672.55830.787676.951897@montanaro.dyndns.org>


    Michael> A few small quibbles...
    >> + PyErr_SetString(PyExc_ValueError, buf);

    Michael> What's wrong with PyErr_Format()?

Probably nothing.  Michal's original patch was against 2.2.something.  I
don't know if that makes a difference or not.  I'll fix it.

    >> + PyDoc_STRVAR(doc_remove_history,
    >> + "remove_history(pos) -> None\n\
    Michael> ^^^^^^^^^^^^^^ This isn't the name in the PyMethodDef at the
    Michael> end...

That too.  Michal had originally called them remove_history and
replace_history.  I added "_item" to the end of each so they were
structurally more like the preexisting get_history_item, then conveniently
missed the doc strings.

Skip
From mwh at python.net  Mon Aug 16 18:19:15 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug 16 18:19:17 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules
	readline.c, 2.72, 2.73
In-Reply-To: <16672.55830.787676.951897@montanaro.dyndns.org> (Skip
	Montanaro's message of "Mon, 16 Aug 2004 11:00:22 -0500")
References: <E1BwM3N-0007X0-6P@sc8-pr-cvs1.sourceforge.net>
	<2md61rcl2l.fsf@starship.python.net>
	<16672.55830.787676.951897@montanaro.dyndns.org>
Message-ID: <2mzn4vawe4.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

>     Michael> A few small quibbles...
>     >> + PyErr_SetString(PyExc_ValueError, buf);
>
>     Michael> What's wrong with PyErr_Format()?
>
> Probably nothing.  Michal's original patch was against 2.2.something.  I
> don't know if that makes a difference or not.  I'll fix it.

Thanks.

>     >> + PyDoc_STRVAR(doc_remove_history,
>     >> + "remove_history(pos) -> None\n\
>     Michael> ^^^^^^^^^^^^^^ This isn't the name in the PyMethodDef at the
>     Michael> end...
>
> That too.  Michal had originally called them remove_history and
> replace_history.  I added "_item" to the end of each so they were
> structurally more like the preexisting get_history_item, then conveniently
> missed the doc strings.

Fast work!

Cheers,
mwh

-- 
  Never meddle in the affairs of NT. It is slow to boot and quick to
  crash.                                             -- Stephen Harris
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
From bob at redivi.com  Mon Aug 16 18:19:09 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug 16 18:19:50 2004
Subject: [Python-Dev] More concerns about decorators
In-Reply-To: <Pine.LNX.4.58.0408161912070.2981@rnd.onego.ru>
References: <Pine.LNX.4.58.0408160833450.1972@rnd.onego.ru>
	<1BE57B0C-EF40-11D8-8D77-000A95686CD8@redivi.com>
	<2m8ycfcjx9.fsf@starship.python.net>
	<Pine.LNX.4.58.0408161912070.2981@rnd.onego.ru>
Message-ID: <008A3A96-EFA0-11D8-A746-000A95686CD8@redivi.com>

On Aug 16, 2004, at 11:39 AM, Roman Suzi wrote:

> On Mon, 16 Aug 2004, Michael Hudson wrote:
>
>> Bob Ippolito <bob@redivi.com> writes:
>>
>>>> Is it possible to change function inplace? Is it even an option?
>>>
>>> Not sure what you mean.  Functions have a __doc__ slot and a __dict__
>>> slot that are read/write.  __name__ is read-only.
>>
>> __name__ is writable as of a few days ago.
>
> I mean that decorators are... decorators. They by their meaning must 
> not
> change the function but add some twists to their behaviour. Many of 
> decorator
> examples are destructing function attributes.

Some decorators are "decorators", other decorators are transformations 
:)  staticmethod and classmethod certainly aren't "decorators" as they 
change what the function does and return a different object, from an 
implementation standpoint anyway.

> Good example of non-destructing decorator is staticmethod:

Not all decorators are non-destructive.

> So, my concern was that it requires additional efforts to save
> function attributes. My thought was that probably decorators could
> work directly on the function, not changing id of the function?
> (staticmethod changes id, of course). Why? Because the function
> is not accessible any more. while the old way it was possible to store
> intermediate result:

Most, if not all, useful decorators will need to change the id of the 
function.  Even a decorator that optimizes bytecode would have to 
return a new object.  The only things you can do to a function without 
changing its identity (IIRC) are changing the slots __name__, __doc__, 
and __dict__.

> class A:
>      def m(x):
>         """I am method m"""
>      mm = m
>      m = staticmethod(m)
>
>
> And use it somehow. But with decorator function changes. Yes, it is 
> possible
> that some decorators could store function internally for the purpose of
> calling it, but others will not.

I think that most decorators are going to store the function object one 
way or another.  I'm not sure why that matters though?  I've never seen 
production code that uses this pattern (save out an instance method and 
a staticmethod of separate names), except for properties (which is 
somewhat different).

> The transfer of function attributes is on the developer.  And he/she 
> could be
> clueless that docstrings, etc, could be lost in transformation.

That already happens if you do decoration "manually" the pre-2.4 way.  
As long as its documented, it shouldn't be a problem.  A "carryover" 
function as I offered up in my last email (modified to also bring over 
__name__) would help considerably, I think.

-bob
From bob at redivi.com  Mon Aug 16 18:47:06 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug 16 18:47:16 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <16672.55274.689761.740267@montanaro.dyndns.org>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
Message-ID: <E8985B48-EFA3-11D8-A746-000A95686CD8@redivi.com>


On Aug 16, 2004, at 11:51 AM, Skip Montanaro wrote:

>
> I think I might have reported this previously (long time ago), but I 
> don't
> recall if it was ever resolved.  (Apparently not, since I'm still 
> seeing
> it.)
>
> I'm seeing a test failure in test_tempfile on Mac OSX (I'm running
> 10.2.8/Jaguar) with a unix (not framework) build.  I've whittled the 
> tests
> down to sundry, tcl and tempfile.  Here's the full run:
--
> This looks reasonable to me, so it doesn't look like the problem is 
> within
> Python.  Should I just skip the tempfile test on Mac OSX?  Can someone 
> check
> this out on 10.3/Panther?

% sw_vers
ProductName:    Mac OS X
ProductVersion: 10.3.5
BuildVersion:   7M34
% mkdir _framework
% cd _framework
% ../configure --enable-framework
% make
% setenv DYLD_FRAMEWORK_PATH .
% ./python.exe ../Lib/test/regrtest.py -f testrun
test_sundry
test_tcl
test_tempfile
All 3 tests OK.

-bob
From mwh at python.net  Mon Aug 16 18:52:53 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug 16 18:52:56 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <16672.55274.689761.740267@montanaro.dyndns.org> (Skip
	Montanaro's message of "Mon, 16 Aug 2004 10:51:06 -0500")
References: <16672.55274.689761.740267@montanaro.dyndns.org>
Message-ID: <2mvffjauu2.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

> I think I might have reported this previously (long time ago), but I don't
> recall if it was ever resolved.  (Apparently not, since I'm still seeing
> it.)
>
> I'm seeing a test failure in test_tempfile on Mac OSX (I'm running
> 10.2.8/Jaguar) with a unix (not framework) build.  I've whittled the tests
> down to sundry, tcl and tempfile.  Here's the full run:

It's odd that test_tcl is on the list.  Where is your tcl from?  Fink?

Even so, it's hard to see how it could be interfering.  I don't see
this behaviour on a similar system, but then I don't have tcl
installed.

Cheers,
mwh

-- 
  All obscurity will buy you is time enough to contract venereal
  diseases.                                  -- Tim Peters, python-dev
From michel at dialnetwork.com  Mon Aug 16 13:35:06 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Mon Aug 16 19:16:27 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <20040816162011.424A11E4005@bag.python.org>
References: <20040816162011.424A11E4005@bag.python.org>
Message-ID: <20040816043506.0bb8542f.michel@dialnetwork.com>


> (Short (off-topic) summary - there's two ways to make the twisted and
> wx event loops cooperate - either the wx event loop runs everything,
> and fires timers to let the twisted event loop do it's thing - see
> above. The second is to let the twisted event loop regularly call
> the wx event loop to let it iterate any outstanding events. When
> a popup or menu comes up, wx creates a new sub-event-loop and takes
> control until that's dismissed... so we're back to needing to run
> the two event loops alongside each other. And yes, Tim's comment
> about insanity resulting from using asyncore and threads together
> also apply to twisted - the only sane way to do it is to have all
> the event loop stuff handled in the same thread. Madness follows,
> otherwise.)

What if asyncore encapsulated the socket map in a selection object (like
Java NIO) instead of a thread-global, module-level dictionary?.
Selections could then be made by calling methods on an object (again,
like NIO's select() (blocking), selectNow() (nonblocking) and
select(timeout)).

This doesn't really address the wx/event granularity issue, but it would
reduce the thread madness considerably and allow programmers to see
beyond asyncore's mainloop() by rolling their own, possible one taking
into consideration other event loops like GUIs.

Another benefit would be more than one selection loop per process,
allowing multiple threads to handle multiple selection sets.

-Michel
From bac at OCF.Berkeley.EDU  Mon Aug 16 19:28:22 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Aug 16 19:28:32 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <16672.55274.689761.740267@montanaro.dyndns.org>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
Message-ID: <4120EEB6.6040505@ocf.berkeley.edu>

Skip Montanaro wrote:
[SNIP - test_tempfile failure]
> This looks reasonable to me, so it doesn't look like the problem is within
> Python.  Should I just skip the tempfile test on Mac OSX?  Can someone check
> this out on 10.3/Panther?
> 

drifty@Bretts-Computer>./python.exe Lib/test/regrtest.py test_sundry 
test_tcl test_tempfile
test_sundry
test_tcl
test_tempfile
[7731 refs]
All 3 tests OK.
[31661 refs]


This UNIX pydebug build was compiled on Aug 14 for OS X 10.3.5 .

-Brett
From skip at pobox.com  Mon Aug 16 19:54:17 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 16 19:54:36 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <2mvffjauu2.fsf@starship.python.net>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
	<2mvffjauu2.fsf@starship.python.net>
Message-ID: <16672.62665.797132.507635@montanaro.dyndns.org>


    Michael> It's odd that test_tcl is on the list.  Where is your tcl from?
    Michael> Fink?

No, it's the TclAqua thing I think:

    _tkinter.so:
            /Library/Frameworks/Tcl.framework/Versions/8.4/Tcl (compatibility version 8.4.0, current version 8.4.0)
            /Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility version 8.4.0, current version 8.4.0)
            /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 63.0.0)

    Michael> Even so, it's hard to see how it could be interfering.  I don't
    Michael> see this behaviour on a similar system, but then I don't have
    Michael> tcl installed.

Dunno.  It was required to provoke the failure for me.  I was encouraged
that Bob's box didn't fail.  How is your _tkinter.so linked?

Skip
From mwh at python.net  Mon Aug 16 19:56:02 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug 16 19:56:04 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <16672.62665.797132.507635@montanaro.dyndns.org> (Skip
	Montanaro's message of "Mon, 16 Aug 2004 12:54:17 -0500")
References: <16672.55274.689761.740267@montanaro.dyndns.org>
	<2mvffjauu2.fsf@starship.python.net>
	<16672.62665.797132.507635@montanaro.dyndns.org>
Message-ID: <2mr7q7arwt.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

>     Michael> It's odd that test_tcl is on the list.  Where is your tcl from?
>     Michael> Fink?
>
> No, it's the TclAqua thing I think:
>
>     _tkinter.so:
>             /Library/Frameworks/Tcl.framework/Versions/8.4/Tcl (compatibility version 8.4.0, current version 8.4.0)
>             /Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility version 8.4.0, current version 8.4.0)
>             /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 63.0.0)
>
>     Michael> Even so, it's hard to see how it could be interfering.  I don't
>     Michael> see this behaviour on a similar system, but then I don't have
>     Michael> tcl installed.
>
> Dunno.  It was required to provoke the failure for me.  I was encouraged
> that Bob's box didn't fail.  How is your _tkinter.so linked?

It's not, I don't have one :-)  test_tcl skips on my system.

Cheers,
mwh

-- 
  One of the great skills in using any language is knowing what not
  to use, what not to say.  ... There's that simplicity thing again.
                                                       -- Ron Jeffries
From skip at pobox.com  Mon Aug 16 19:57:27 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 16 19:57:42 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <4120EEB6.6040505@ocf.berkeley.edu>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
	<4120EEB6.6040505@ocf.berkeley.edu>
Message-ID: <16672.62855.287221.180403@montanaro.dyndns.org>


    Brett> drifty@Bretts-Computer>./python.exe Lib/test/regrtest.py test_sundry 
    Brett> test_tcl test_tempfile
    Brett> test_sundry
    Brett> test_tcl
    Brett> test_tempfile
    Brett> [7731 refs]
    Brett> All 3 tests OK.
    Brett> [31661 refs]

    Brett> This UNIX pydebug build was compiled on Aug 14 for OS X 10.3.5 .

Thanks.  I'm going to let it slide on the assumption that it's a fixed bug
in Jaguar.  On the outside chance someone wants to truly mimic my build
environment, here's how I configured:

    ./configure '--with-pydebug' '--with-libs=/usr/lib/libMallocDebug.a'

I suppose it's possible other folks trying to reproduce this problem
encounter the same behavior but aren't linked with MallocDebug, so the
problem passes silently.

Skip
From skip at pobox.com  Mon Aug 16 20:07:32 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 16 20:07:48 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <2mr7q7arwt.fsf@starship.python.net>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
	<2mvffjauu2.fsf@starship.python.net>
	<16672.62665.797132.507635@montanaro.dyndns.org>
	<2mr7q7arwt.fsf@starship.python.net>
Message-ID: <16672.63460.103743.36579@montanaro.dyndns.org>


    >> Dunno.  It was required to provoke the failure for me.  I was
    >> encouraged that Bob's box didn't fail.  How is your _tkinter.so
    >> linked?

    Michael> It's not, I don't have one :-) test_tcl skips on my system.

Sorry about that.  I meant, "Bob, how is your _tkinter.so linked?"...

Skip
From bac at OCF.Berkeley.EDU  Mon Aug 16 20:10:33 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Aug 16 20:10:43 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <16672.62855.287221.180403@montanaro.dyndns.org>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
	<4120EEB6.6040505@ocf.berkeley.edu>
	<16672.62855.287221.180403@montanaro.dyndns.org>
Message-ID: <4120F899.7090008@ocf.berkeley.edu>

Skip Montanaro wrote:

>     Brett> drifty@Bretts-Computer>./python.exe Lib/test/regrtest.py test_sundry 
>     Brett> test_tcl test_tempfile
>     Brett> test_sundry
>     Brett> test_tcl
>     Brett> test_tempfile
>     Brett> [7731 refs]
>     Brett> All 3 tests OK.
>     Brett> [31661 refs]
> 
>     Brett> This UNIX pydebug build was compiled on Aug 14 for OS X 10.3.5 .
> 
> Thanks.  I'm going to let it slide on the assumption that it's a fixed bug
> in Jaguar.  On the outside chance someone wants to truly mimic my build
> environment, here's how I configured:
> 
>     ./configure '--with-pydebug' '--with-libs=/usr/lib/libMallocDebug.a'
> 
> I suppose it's possible other folks trying to reproduce this problem
> encounter the same behavior but aren't linked with MallocDebug, so the
> problem passes silently.
> 

And just to be thorough with my build environment (maybe we should have 
some command-line switch that outputs everything about the build of the 
interpreter?):

CONFIG_ARGS=    '--prefix=/Users/drifty/Code/compiled_CVS' 
'--disable-framework' '        --disable-toolbox-glue' '--without-cxx' 
'--with-pydebug' '-enable-unicode=ucs2' 'CPPFLAGS=-I/sw/include 
-I/opt/include' 'LDFLAGS=-L/sw/lib -L/opt/lib'



-Brett
From bob at redivi.com  Mon Aug 16 20:17:12 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug 16 20:17:53 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <16672.63460.103743.36579@montanaro.dyndns.org>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
	<2mvffjauu2.fsf@starship.python.net>
	<16672.62665.797132.507635@montanaro.dyndns.org>
	<2mr7q7arwt.fsf@starship.python.net>
	<16672.63460.103743.36579@montanaro.dyndns.org>
Message-ID: <7EE39EE7-EFB0-11D8-A746-000A95686CD8@redivi.com>


On Aug 16, 2004, at 2:07 PM, Skip Montanaro wrote:

>
>>> Dunno.  It was required to provoke the failure for me.  I was
>>> encouraged that Bob's box didn't fail.  How is your _tkinter.so
>>> linked?
>
>     Michael> It's not, I don't have one :-) test_tcl skips on my 
> system.
>
> Sorry about that.  I meant, "Bob, how is your _tkinter.so linked?"...

% otool -L build/lib.darwin-7.3.0-Power_Macintosh-2.4/_tkinter.so
build/lib.darwin-7.3.0-Power_Macintosh-2.4/_tkinter.so:
         /Library/Frameworks/Tcl.framework/Versions/8.4/Tcl 
(compatibility version 8.4.0, current version 8.4.0)
         /Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility 
version 8.4.0, current version 8.4.0)
         /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, 
current version 71.0.0)

presumably this is:
-bundle -undefined dynamic_lookup -framework Tcl -framework Tk

(using TclTkAqua framework builds...)

-undefined dynamic_lookup is new to Mac OS X 10.3, and allows bundles 
to use Python without linking directly to it.  10.2 would look largely 
the same except it would probably have a -F. -framework Python in 
there.

-bob
From mal at egenix.com  Mon Aug 16 20:27:50 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon Aug 16 20:27:52 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <411DC3DD.3080900@v.loewis.de>
References: <1092258146.411a896246d1a@mcherm.com>	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>	<16666.56033.65067.241794@montanaro.dyndns.org>	<411AFBD4.5080903@v.loewis.de><411B0DA0.4030109@v.loewis.de>	<411BB50C.8000804@v.loewis.de>	<16667.54746.535997.120878@montanaro.dyndns.org>	<411C85B6.2060104@egenix.com>
	<411DC3DD.3080900@v.loewis.de>
Message-ID: <4120FCA6.2070104@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
> 
>> If we switch the binding of 'yyy' to mean unicode('yyy')
>> some day, why can't we just continue to use the existing implementation
>> for 8-bit strings for b'xxx' (the current implementation is already
>> doing the right thing, meaning that it is 8-bit safe regardeless
>> of the source code encoding) ?
> 
> 
> Not exactly - the current implementation is not safe with respect to
> re-encoding source in a different encoding.

It is if you stick to writing your binary data using an ASCII
compatible encoding -- I wouldn't expect any other encoding for
binary data anyway. The most common are ASCII + escape sequences,
base64 or hex, all of which are ASCII compatible.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 16 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From skip at pobox.com  Mon Aug 16 21:14:38 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 16 21:14:55 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <4120F899.7090008@ocf.berkeley.edu>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
	<4120EEB6.6040505@ocf.berkeley.edu>
	<16672.62855.287221.180403@montanaro.dyndns.org>
	<4120F899.7090008@ocf.berkeley.edu>
Message-ID: <16673.1950.317156.261551@montanaro.dyndns.org>


    Brett> ... (maybe we should have some command-line switch that outputs
    Brett> everything about the build of the interpreter?):

The time machine strikes again:

    % egrep 'exec /bin/sh ./configure' config.status 
      exec /bin/sh ./configure '--with-pydebug' '--with-libs=/usr/lib/libMallocDebug.a' $ac_configure_extra_args --no-create --no-recursion

though in this case I think it may be Dennis Ritchie's or Ken Thompson's
time machines, not Guido's. <wink>

Skip
From bob at redivi.com  Mon Aug 16 21:23:01 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon Aug 16 21:23:39 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <16673.1950.317156.261551@montanaro.dyndns.org>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
	<4120EEB6.6040505@ocf.berkeley.edu>
	<16672.62855.287221.180403@montanaro.dyndns.org>
	<4120F899.7090008@ocf.berkeley.edu>
	<16673.1950.317156.261551@montanaro.dyndns.org>
Message-ID: <B08B55D6-EFB9-11D8-8F75-000A95686CD8@redivi.com>


On Aug 16, 2004, at 3:14 PM, Skip Montanaro wrote:

>
>     Brett> ... (maybe we should have some command-line switch that 
> outputs
>     Brett> everything about the build of the interpreter?):
>
> The time machine strikes again:
>
>     % egrep 'exec /bin/sh ./configure' config.status
>       exec /bin/sh ./configure '--with-pydebug' 
> '--with-libs=/usr/lib/libMallocDebug.a' $ac_configure_extra_args 
> --no-create --no-recursion
>
> though in this case I think it may be Dennis Ritchie's or Ken 
> Thompson's
> time machines, not Guido's. <wink>

Even better (and I think this is what he was actually asking for):

% python -c 'import distutils.sysconfig; print 
distutils.sysconfig.get_config_var("CONFIG_ARGS")'
'--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' 
'--enable-ipv6' '--with-threads' 
'--enable-framework=/System/Library/Frameworks' '--enable-toolbox-glue' 
'CFLAGS=-g -Os -pipe -fno-common -Wno-long-double -mno-fused-madd 
-pipe' 'LDFLAGS=-Wl,-F.'

% ./python.exe -c 'import distutils.sysconfig; print 
distutils.sysconfig.get_config_var("CONFIG_ARGS")'
'--enable-framework'

-bob

From phil at riverbankcomputing.co.uk  Mon Aug 16 21:57:41 2004
From: phil at riverbankcomputing.co.uk (Phil Thompson)
Date: Mon Aug 16 21:57:43 2004
Subject: [Python-Dev] Problem with PyGILState_Ensure() and the thread Module
Message-ID: <200408162057.41113.phil@riverbankcomputing.co.uk>

I'm having a problem calling functions in a C extension module that uses 
PyGILState_Ensure() and PyGILState_Release() from a thread created with 
thread.start_new_thread().

A deadlock occurs in PyGILState_Ensure() because it doesn't know anything 
about the thread state created by start_new_thread(). Therefore it creates a 
new one and then calls PyEval_RestoreThread() - but it already holds the GIL 
and so deadlocks.

I would expect the threading PyGILState_Ensure() to know about all thread 
states - not just the ones it creates itself.

Is this a bug, feature, or misunderstanding on my part?

This is with Python 2.4a2.

Thanks,
Phil
From aahz at pythoncraft.com  Mon Aug 16 23:19:07 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Aug 16 23:19:09 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com>
References: <002001c48325$436d9460$e841fea9@oemcomputer>
	<20040816034817.GA25038@panix.com>
	<0ED3DE85-EF38-11D8-8D77-000A95686CD8@redivi.com>
	<20040816035936.GB7737@panix.com>
	<77B2C60E-EF3D-11D8-8D77-000A95686CD8@redivi.com>
Message-ID: <20040816211907.GA1471@panix.com>

On Mon, Aug 16, 2004, Bob Ippolito wrote:
> On Aug 15, 2004, at 11:59 PM, Aahz wrote:
>>On Sun, Aug 15, 2004, Bob Ippolito wrote:
>>>
>>>Many platform GUIs require that all or most all GUI activities be
>>>consolidated to one and only one thread.  I don't think any of them
>>>'care' if another thread is also running, but it can't communicate
>>>directly with the GUI.
>>
>>That's true, but it also applies to other external resources, such as
>>files.
> 
> Maybe I don't understand what you're saying.. but I don't know any 
> platform that forces you to work with files from one and only one 
> specific thread.  But I definitely do know at least one platform where 
> you may only perform most GUI operations from the main thread of 
> execution.

For any one file handle, you use only one thread.  It just happens in
the case of GUIs that there is only one GUI handle.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From skip at pobox.com  Mon Aug 16 23:55:41 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 16 23:55:55 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <B08B55D6-EFB9-11D8-8F75-000A95686CD8@redivi.com>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
	<4120EEB6.6040505@ocf.berkeley.edu>
	<16672.62855.287221.180403@montanaro.dyndns.org>
	<4120F899.7090008@ocf.berkeley.edu>
	<16673.1950.317156.261551@montanaro.dyndns.org>
	<B08B55D6-EFB9-11D8-8F75-000A95686CD8@redivi.com>
Message-ID: <16673.11613.356961.874826@montanaro.dyndns.org>


    Bob> Even better (and I think this is what he was actually asking for):

    Bob> % python -c 'import distutils.sysconfig; print 
    Bob> distutils.sysconfig.get_config_var("CONFIG_ARGS")'

Thanks.  Not as easy to remember as "grep configure config.status", but
useful nonetheless.

Skip
From bob at redivi.com  Tue Aug 17 00:09:25 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Aug 17 00:10:34 2004
Subject: [Python-Dev] test_tempfile failure on Mac OSX
In-Reply-To: <16673.11613.356961.874826@montanaro.dyndns.org>
References: <16672.55274.689761.740267@montanaro.dyndns.org>
	<4120EEB6.6040505@ocf.berkeley.edu>
	<16672.62855.287221.180403@montanaro.dyndns.org>
	<4120F899.7090008@ocf.berkeley.edu>
	<16673.1950.317156.261551@montanaro.dyndns.org>
	<B08B55D6-EFB9-11D8-8F75-000A95686CD8@redivi.com>
	<16673.11613.356961.874826@montanaro.dyndns.org>
Message-ID: <EF47F946-EFD0-11D8-AA0D-000A95686CD8@redivi.com>


On Aug 16, 2004, at 5:55 PM, Skip Montanaro wrote:

>
>     Bob> Even better (and I think this is what he was actually asking 
> for):
>
>     Bob> % python -c 'import distutils.sysconfig; print
>     Bob> distutils.sysconfig.get_config_var("CONFIG_ARGS")'
>
> Thanks.  Not as easy to remember as "grep configure config.status", but
> useful nonetheless.

Yeah, but I don't have a hard time remembering that the Makefile (that 
gets distributed w/ Python) knows everything, and that distutils knows 
how to read the Makefile.  It's hard to remember the names of the make 
variables and the distutils incantation, but neither are hard to look 
up (if you've been through it a couple times anyway).

I prefer not to depend on your solution because it's only useful if you 
compiled Python and still have access to that build tree (and remember 
which one it was).  The first CONFIG_ARGS I spit out was the one used 
to build Python 2.3.0 on Mac OS X 10.3, and I certainly didn't do that 
here :)

-bob
From martin at v.loewis.de  Tue Aug 17 01:04:13 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 17 01:04:16 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4120FCA6.2070104@egenix.com>
References: <1092258146.411a896246d1a@mcherm.com>	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>	<16666.56033.65067.241794@montanaro.dyndns.org>	<411AFBD4.5080903@v.loewis.de><411B0DA0.4030109@v.loewis.de>	<411BB50C.8000804@v.loewis.de>	<16667.54746.535997.120878@montanaro.dyndns.org>	<411C85B6.2060104@egenix.com>
	<411DC3DD.3080900@v.loewis.de> <4120FCA6.2070104@egenix.com>
Message-ID: <41213D6D.50302@v.loewis.de>

M.-A. Lemburg wrote:
> It is if you stick to writing your binary data using an ASCII
> compatible encoding -- I wouldn't expect any other encoding for
> binary data anyway. The most common are ASCII + escape sequences,
> base64 or hex, all of which are ASCII compatible.

We probably have a different notion of "ASCII compatible" then.
I would define it as:

  An encoding E is "ASCII compatbible" if strings that only consist
  of ASCII characters use the same byte representation in E that
  they use in ASCII.

In that sense, ISO-8859-1 and UTF-8 are also ASCII compatible. Notice
that this is also the definition that PEP 263 assumes.

However, byte strings used in source code are not "safe" if they
are encoded in ISO-8859-1 under recoding: If the source code is
converted to UTF-8 (including the encoding declaration), then
the length of the strings changes, as do the byte values inside
the string.

Regards,
Martin

From ncoghlan at iinet.net.au  Tue Aug 17 01:15:48 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue Aug 17 01:15:51 2004
Subject: [Python-Dev] Decorator syntax J2 (decorate..def),
	with implementation
In-Reply-To: <3A81C87DC164034AA4E2DDFE11D258E3022DFD@exchange.hqamor.amorhq.net>
References: <3A81C87DC164034AA4E2DDFE11D258E3022DFD@exchange.hqamor.amorhq.net>
Message-ID: <41214024.2030005@iinet.net.au>

Robert Brewer wrote:

> Michael Sparks wrote:
> 
>>As for whether it's the right keyword or not, I'm not sure 
>>I'm qualified to 
>>say, but decorators seem much more like declarations of 
>>function properties 
>>in all the use cases I've seen.
> 
> The proposal, in effect, creates a declarative suite as opposed to the
> more traditional imperative suite. Perhaps "declare:" would be more
> appropriate (but give your brain a week to mull it over ;).

I'm still a fan of 'predef:' if a keyword is going to be involved. The 
only thing I've seen so far that is common to all decorators is that 
they are predefining things about a 'def' we haven't encountered yet :)

Cheers,
Nick.

-- 
Nick Coghlan               |  Eugene, Oregon
Email: ncoghlan@email.com  |             USA
From foom at fuhm.net  Tue Aug 17 02:07:14 2004
From: foom at fuhm.net (James Y Knight)
Date: Tue Aug 17 02:07:25 2004
Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compiler
	mystery] 
In-Reply-To: <20040816112916.GA19969@vicky.ecs.soton.ac.uk>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
	<20040812185521.GA2277@vicky.ecs.soton.ac.uk>
	<1f7befae04081212414007274f@mail.gmail.com>
	<20040812204431.GA31884@vicky.ecs.soton.ac.uk>
	<1f7befae0408151950361f0cb4@mail.gmail.com>
	<20040816112916.GA19969@vicky.ecs.soton.ac.uk>
Message-ID: <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net>

On Aug 16, 2004, at 7:29 AM, Armin Rigo wrote:
> Yes.  Here is a patch attempting to do what I described:
>  http://www.python.org/sf/1009929


 From the patch description:
>  Some more thinking about [the exception hierarchy] would be welcome. 
> Maybe AsynchronousException and a few others should not subclass 
> Exception at all, so that "except Exception" statements don't catch 
> them. Anyway, this patch alreaddy improves the situation, because you 
> can catch and re-raise AsynchronousException (instead of, say, just 
> KeyboardInterrupt).

It seems to me that something similar to what Java has would be a good 
idea. Namely, a new top level exception (from which all others would 
derive) called "Raisable", analogous to Java's Throwable. This then has 
two subclasses: "Exception", and "FatalError". I'm not sure FatalError 
is a good name, but *some* new name needs to be thought up for Java's 
"Error" class, because lots of python exceptions end in "Error" but 
belong under the "Exception" hierarchy (for example "KeyError").

The criteria for whether a given exception should go under "Exception" 
or "FatalError" is whether users' code should generally catch the 
exception. Thus, at least "SystemExit", "KeyboardInterrupt", and 
"MemoryError" should go under "FatalError". Catching those is nearly 
never what you wanted to do when you write "except Exception:". There's 
likely more -- I have not gone through all the exceptions in Python to 
classify them.

One issue is that creating a new category of Exceptions doesn't help 
people who do "except:" instead of "except Exception:". It is unlikely 
that person meant to catch things like MemoryError, rather, they were 
just being lazy. I suppose that syntax could be deprecated, at least in 
example code and documentation, in favor of "except Exception" for the 
usual case, and "except Raisable" for the cases where you do actually 
want to catch everything*.

James

* Except, of course, old string exceptions which have been deprecated 
for ages.

From bob at redivi.com  Tue Aug 17 02:33:51 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Aug 17 02:34:27 2004
Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compiler
	mystery] 
In-Reply-To: <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
	<20040812185521.GA2277@vicky.ecs.soton.ac.uk>
	<1f7befae04081212414007274f@mail.gmail.com>
	<20040812204431.GA31884@vicky.ecs.soton.ac.uk>
	<1f7befae0408151950361f0cb4@mail.gmail.com>
	<20040816112916.GA19969@vicky.ecs.soton.ac.uk>
	<64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net>
Message-ID: <1CC8AA0A-EFE5-11D8-AA0D-000A95686CD8@redivi.com>


On Aug 16, 2004, at 8:07 PM, James Y Knight wrote:

> On Aug 16, 2004, at 7:29 AM, Armin Rigo wrote:
>> Yes.  Here is a patch attempting to do what I described:
>>  http://www.python.org/sf/1009929
>
>
> From the patch description:
>>  Some more thinking about [the exception hierarchy] would be welcome. 
>> Maybe AsynchronousException and a few others should not subclass 
>> Exception at all, so that "except Exception" statements don't catch 
>> them. Anyway, this patch alreaddy improves the situation, because you 
>> can catch and re-raise AsynchronousException (instead of, say, just 
>> KeyboardInterrupt).
>
> It seems to me that something similar to what Java has would be a good 
> idea. Namely, a new top level exception (from which all others would 
> derive) called "Raisable", analogous to Java's Throwable. This then 
> has two subclasses: "Exception", and "FatalError". I'm not sure 
> FatalError is a good name, but *some* new name needs to be thought up 
> for Java's "Error" class, because lots of python exceptions end in 
> "Error" but belong under the "Exception" hierarchy (for example 
> "KeyError").
>
> The criteria for whether a given exception should go under "Exception" 
> or "FatalError" is whether users' code should generally catch the 
> exception. Thus, at least "SystemExit", "KeyboardInterrupt", and 
> "MemoryError" should go under "FatalError". Catching those is nearly 
> never what you wanted to do when you write "except Exception:". 
> There's likely more -- I have not gone through all the exceptions in 
> Python to classify them.
>
> One issue is that creating a new category of Exceptions doesn't help 
> people who do "except:" instead of "except Exception:". It is unlikely 
> that person meant to catch things like MemoryError, rather, they were 
> just being lazy. I suppose that syntax could be deprecated, at least 
> in example code and documentation, in favor of "except Exception" for 
> the usual case, and "except Raisable" for the cases where you do 
> actually want to catch everything*.
>
> James
>
> * Except, of course, old string exceptions which have been deprecated 
> for ages.

* basestr could inherit from "Raisable" ;)

The big problem with "Raisable" is that both raiseable and raisable 
seem to be correct spellings, and I don't think either are in many 
abridged dictionaries (the OS X spell checker doesn't like either, for 
example).

-bob
From aahz at pythoncraft.com  Tue Aug 17 03:01:56 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Aug 17 03:02:12 2004
Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compiler
	mystery]
In-Reply-To: <64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
	<20040812185521.GA2277@vicky.ecs.soton.ac.uk>
	<1f7befae04081212414007274f@mail.gmail.com>
	<20040812204431.GA31884@vicky.ecs.soton.ac.uk>
	<1f7befae0408151950361f0cb4@mail.gmail.com>
	<20040816112916.GA19969@vicky.ecs.soton.ac.uk>
	<64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net>
Message-ID: <20040817010156.GA23702@panix.com>

On Mon, Aug 16, 2004, James Y Knight wrote:
>
> The criteria for whether a given exception should go under "Exception" 
> or "FatalError" is whether users' code should generally catch the 
> exception. Thus, at least "SystemExit", "KeyboardInterrupt", and 
> "MemoryError" should go under "FatalError". Catching those is nearly 
> never what you wanted to do when you write "except Exception:". There's 
> likely more -- I have not gone through all the exceptions in Python to 
> classify them.
> 
> One issue is that creating a new category of Exceptions doesn't help 
> people who do "except:" instead of "except Exception:". It is unlikely 
> that person meant to catch things like MemoryError, rather, they were 
> just being lazy. I suppose that syntax could be deprecated, at least in 
> example code and documentation, in favor of "except Exception" for the 
> usual case, and "except Raisable" for the cases where you do actually 
> want to catch everything*.

We've already got StandardError; I think it makes more sense to
rearrange the exception hierarchy a bit to support your suggestion
rather than creating a whole new base class.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From guido at python.org  Tue Aug 17 03:06:19 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 03:06:40 2004
Subject: [Python-Dev] Decorator order implemented backwards?
In-Reply-To: Your message of "Sun, 15 Aug 2004 13:05:28 PDT."
	<411FC208.3040800@ocf.berkeley.edu> 
References: <B20BF049-EA7B-11D8-8D12-000A95A50FB2@fuhm.net>
	<4118566B.2090907@interlink.com.au>
	<41185844.80906@interlink.com.au>
	<1092172040.1681.9.camel@localhost>
	<411F0F2A.2070709@ocf.berkeley.edu>
	<1092583647.1670.18.camel@localhost> 
	<411FC208.3040800@ocf.berkeley.edu> 
Message-ID: <200408170106.i7H16J614260@guido.python.org>

> 728 emails to summarize on top of any sent in today so I will be swamped 
> with that for the rest of the week.  I am sure someone else can step up 
> and applying it if Guido okays the application order.

Since Mark ended up agreeing with your order, the OK is given by
default. :-)


--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug 17 03:08:56 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 03:09:08 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: Your message of "Sun, 15 Aug 2004 17:41:26 EDT."
	<001501c48310$bf15aa40$e841fea9@oemcomputer> 
References: <001501c48310$bf15aa40$e841fea9@oemcomputer> 
Message-ID: <200408170108.i7H18vs14287@guido.python.org>

> The documentation for the atexit module (introduced in Py2.0) states:
> 
> """Note: This module is unlikely to work correctly when used with other
> code that sets sys.exitfunc. In particular, other core Python modules
> are free to use atexit without the programmer's knowledge. Authors who
> use sys.exitfunc should convert their code to use atexit instead. The
> simplest way to convert code that sets sys.exitfunc is to import atexit
> and register the function that had been bound to sys.exitfunc."""
> 
> Can we strengthen this by deprecating sys.exitfunc?
> 
> The atexit module does attempt to co-exist by introducing code to
> register a function in sys.exitfunc if it were defined before "import
> atexit" was called.  However, this is unreliable because it depends on
> import order and the library is free to introduce an earlier "import
> atexit" which would break code relying on the co-existance mechanism.

+1

(I think the quoted words were written before we had introduced a
formal concept of deprecation -- atexit is quite old. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tdelaney at avaya.com  Tue Aug 17 03:11:17 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Aug 17 03:11:25 2004
Subject: [Python-Dev] Exception hierarchy [was Re: Another
	test_compilermystery] 
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01D588B3@au3010avexu1.global.avaya.com>

James Y Knight wrote:

> One issue is that creating a new category of Exceptions doesn't help
> people who do "except:" instead of "except Exception:". It is unlikely
> that person meant to catch things like MemoryError, rather, they were
> just being lazy. I suppose that syntax could be deprecated, at least
> in 
> example code and documentation, in favor of "except Exception" for the
> usual case, and "except Raisable" for the cases where you do actually
> want to catch everything*.

Personally, I would be in favour of a bare except being equivalent in
new code to 'except Exception'. Although this does violate 'in the
presence of ambiguity, resist the temptation to guess'.

Tim Delaney
From guido at python.org  Tue Aug 17 03:11:34 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 03:11:42 2004
Subject: [Python-Dev] Classes that claim to be defined in __builtin__ but
	aren't
In-Reply-To: Your message of "Mon, 16 Aug 2004 13:09:31 +1200."
	<200408160109.i7G19VHN019296@cosc353.cosc.canterbury.ac.nz> 
References: <200408160109.i7G19VHN019296@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408170111.i7H1BYJ14320@guido.python.org>

> The real question seems to be whether __builtin__ should contain *all*
> built-in types, including internal ones, or only those intended for
> public use. Do you have an opinion about that, Guido?
> 
> I suppose it's reasonable to put them all in __builtin__, since as you
> say, they can be useful for type checking even if you can't or rarely
> want to instantiate them yourself.

That's where I stand, at about +0.5.  I don't think it's super
important that all builtins be particularly well-known or useful --
it's easy enough to have a section in the documentation for
"lesser-known builtins".

--Guido van Rossum (home page: http://www.python.org/~guido/)
From greg at cosc.canterbury.ac.nz  Tue Aug 17 03:13:32 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug 17 03:13:37 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41213D6D.50302@v.loewis.de>
Message-ID: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>

> However, byte strings used in source code are not "safe" if they
> are encoded in ISO-8859-1 under recoding: If the source code is
> converted to UTF-8 (including the encoding declaration), then
> the length of the strings changes, as do the byte values inside
> the string.

This suggests that byte string literals should be
restricted to ASCII characters and \x escapes.
Would that be safe enough?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From guido at python.org  Tue Aug 17 03:14:47 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 03:14:55 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: Your message of "Sun, 15 Aug 2004 23:44:23 EDT."
	<900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com> 
References: <002001c48325$436d9460$e841fea9@oemcomputer>
	<87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp> 
	<900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com> 
Message-ID: <200408170114.i7H1Elk14352@guido.python.org>

> > (Season to taste, fix technical inaccuracies.)  I see no reason to
> > mention the GIL, an implementation detail, at all.
> 
> I think it's worth mentioning.  It's absolutely necessary to know all 
> about the GIL when writing an extension that can take advantage of 
> threading or embed Python correctly in a threaded application.

But does that mention beling in a threading *tutorial*?

(In addition, I find that the GIL often causes people to worry
unnecessarily about the GIL.  Sort of like worrying about relativistic
effects when driving a car.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug 17 03:21:55 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 03:22:02 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: Your message of "Mon, 16 Aug 2004 12:33:57 BST."
	<20040816113357.GB19969@vicky.ecs.soton.ac.uk> 
References: <001501c48310$bf15aa40$e841fea9@oemcomputer>  
	<20040816113357.GB19969@vicky.ecs.soton.ac.uk> 
Message-ID: <200408170121.i7H1Ltn14417@guido.python.org>

> Multiple well-written modules can use sys.exitfunc in a way that is
> compatible.  atexit is merely a glue to discourage non-well-written
> modules.  I'd say that atexit should be encouraged but there is
> nothing wrong with using sys.exitfunc directly, if one is aware of
> the common patching pattern: "save-old-value,
> replace-with-my-function, upon-call-invoke-saved-old-value".

But why offer two ways of doing the same thing?  Apart from b/w
compatibility, what's the advantage of using sys.exitfunc directly?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug 17 03:32:59 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 03:33:08 2004
Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compiler
	mystery]
In-Reply-To: Your message of "Mon, 16 Aug 2004 20:07:14 EDT."
	<64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net> 
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
	<20040812185521.GA2277@vicky.ecs.soton.ac.uk>
	<1f7befae04081212414007274f@mail.gmail.com>
	<20040812204431.GA31884@vicky.ecs.soton.ac.uk>
	<1f7befae0408151950361f0cb4@mail.gmail.com>
	<20040816112916.GA19969@vicky.ecs.soton.ac.uk> 
	<64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net> 
Message-ID: <200408170133.i7H1WxC14538@guido.python.org>

> From the patch description:
> > Some more thinking about [the exception hierarchy] would be welcome. 
> > Maybe AsynchronousException and a few others should not subclass 
> > Exception at all, so that "except Exception" statements don't catch 
> > them. Anyway, this patch alreaddy improves the situation, because you 
> > can catch and re-raise AsynchronousException (instead of, say, just 
> > KeyboardInterrupt).
> 
> It seems to me that something similar to what Java has would be a good 
> idea. Namely, a new top level exception (from which all others would 
> derive) called "Raisable", analogous to Java's Throwable. This then has 
> two subclasses: "Exception", and "FatalError". I'm not sure FatalError 
> is a good name, but *some* new name needs to be thought up for Java's 
> "Error" class, because lots of python exceptions end in "Error" but 
> belong under the "Exception" hierarchy (for example "KeyError").

Hm, Java needs the distinction because some exceptions must be
declared and others mustn't.  But Python doesn't have that
distinction.  I'm not sure that you can always treat the same set of
exceptions as fatal.  E.g. in many situations, AttributeError,
TypeError and NameError are all indicative of programming bugs
(typically typos), but in other contexts these are recoverable.  So
rather than giving an arbitrary definition of fatality, let's refrain
from defining the concept.

> The criteria for whether a given exception should go under "Exception" 
> or "FatalError" is whether users' code should generally catch the 
> exception. Thus, at least "SystemExit", "KeyboardInterrupt", and 
> "MemoryError" should go under "FatalError". Catching those is nearly 
> never what you wanted to do when you write "except Exception:". There's 
> likely more -- I have not gone through all the exceptions in Python to 
> classify them.

Calling SystemExit and KeyboardInterrupt fatal strikes me as
particularly odd, as I routinely catch these.

> One issue is that creating a new category of Exceptions doesn't help 
> people who do "except:" instead of "except Exception:". It is unlikely 
> that person meant to catch things like MemoryError, rather, they were 
> just being lazy. I suppose that syntax could be deprecated, at least in 
> example code and documentation, in favor of "except Exception" for the 
> usual case, and "except Raisable" for the cases where you do actually 
> want to catch everything*.
> 
> James
> 
> * Except, of course, old string exceptions which have been deprecated 
> for ages.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug 17 03:34:13 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 03:34:19 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: Your message of "Tue, 17 Aug 2004 13:13:32 +1200."
	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> 
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408170134.i7H1YDR14560@guido.python.org>

> This suggests that byte string literals should be
> restricted to ASCII characters and \x escapes.
> Would that be safe enough?

Make that printable ASCII and I'm +0.  (I'm still not sold on the
concept of bytes literals at all.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From foom at fuhm.net  Tue Aug 17 04:52:35 2004
From: foom at fuhm.net (James Y Knight)
Date: Tue Aug 17 04:52:40 2004
Subject: [Python-Dev] Exception hierarchy [was Re: Another test_compiler
	mystery]
In-Reply-To: <200408170133.i7H1WxC14538@guido.python.org>
References: <002d01c48083$9a89a6c0$5229c797@oemcomputer>
	<1f7befae040812091754035bcb@mail.gmail.com>
	<20040812185521.GA2277@vicky.ecs.soton.ac.uk>
	<1f7befae04081212414007274f@mail.gmail.com>
	<20040812204431.GA31884@vicky.ecs.soton.ac.uk>
	<1f7befae0408151950361f0cb4@mail.gmail.com>
	<20040816112916.GA19969@vicky.ecs.soton.ac.uk>
	<64D757B8-EFE1-11D8-8449-000A95A50FB2@fuhm.net>
	<200408170133.i7H1WxC14538@guido.python.org>
Message-ID: <7E3ABDEE-EFF8-11D8-8449-000A95A50FB2@fuhm.net>


On Aug 16, 2004, at 9:32 PM, Guido van Rossum wrote:

> Hm, Java needs the distinction because some exceptions must be
> declared and others mustn't.  But Python doesn't have that
> distinction.  I'm not sure that you can always treat the same set of
> exceptions as fatal.  E.g. in many situations, AttributeError,
> TypeError and NameError are all indicative of programming bugs
> (typically typos), but in other contexts these are recoverable.  So
> rather than giving an arbitrary definition of fatality, let's refrain
> from defining the concept.

Well, actually, java has three categories of exceptions:
1) Error - not necessary to declare as thrown, only *serious* errors. 
Contains:
     a) things that can "never happen" with a program compiled by the 
java compiler (no such field, unknown class file format, etc),
     b) machine errors (out of memory, stack overflow, etc)
     c) assert() failure
     d) ThreadDeath (similar to a KeyboardInterrupt/SystemExit)
2) Exception - normal run of the mill exception, needs to be declared 
as thrown.
3) RuntimeException - subclass of Exception, does not need to be 
declared as thrown. (e.g. IndexOutOfBoundsException, 
NoSuchFieldException, ClassCastException)

The distinction you refer to above is really the difference between 
Exception and RuntimeException. Translated to java, AttributeError, 
TypeError and NameError would be RuntimeExceptions. So, I agree with 
you - I don't believe python needs that distinction. I do believe 
python needs the distinction between Exception and Error.

> Calling SystemExit and KeyboardInterrupt fatal strikes me as
> particularly odd, as I routinely catch these.

I'll agree: I don't think the name "FatalError" is particularly great. 
However, I hope it gets the idea across better than 
"XXXErrorXXXRenameMeXXX" which was my other choice of name. ;) I do 
think the categorization is correct. While you may sometimes catch 
KeyboardInterrupt/SystemExit, most of the time you really do not want 
to, even if you are catching "everything". If you do want to catch 
KeyboardInterrupt, you are also likely to be catching it explicitly by 
its name, anyhow.

James

From martin at v.loewis.de  Tue Aug 17 08:30:05 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 17 08:30:13 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
Message-ID: <4121A5ED.9030001@v.loewis.de>

Greg Ewing wrote:
> This suggests that byte string literals should be
> restricted to ASCII characters and \x escapes.
> Would that be safe enough?

Yes, that should work fine (Guidos restriction to
*printable* characters is also useful, as line endings
are easily changed, too, when moving from one system
to another).

Regards,
Martin
From anthony at interlink.com.au  Tue Aug 17 08:54:35 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug 17 08:54:54 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <200408170134.i7H1YDR14560@guido.python.org>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
Message-ID: <4121ABAB.9060605@interlink.com.au>

Guido van Rossum wrote:
> (I'm still not sold on the
> concept of bytes literals at all.)

Ok. Here's a case - in shtoom, I generate audio data. Lots
of audio data. This is broken into packets, then gets a small
header put onto each RTP packet. Right now, I'm using strings
for this. If there was a 'byte literal', I'd use it. This isn't
a huge problem right now, because strings are good enough. But
if we end up in an 'all the strings are unicode', I'll need
_some_ way to construct these packets.

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From martin at v.loewis.de  Tue Aug 17 09:06:23 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 17 09:06:35 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4121ABAB.9060605@interlink.com.au>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
Message-ID: <4121AE6F.4090904@v.loewis.de>

Anthony Baxter wrote:
> Ok. Here's a case - in shtoom, I generate audio data. Lots
> of audio data. This is broken into packets, then gets a small
> header put onto each RTP packet. Right now, I'm using strings
> for this. If there was a 'byte literal', I'd use it. This isn't
> a huge problem right now, because strings are good enough. But
> if we end up in an 'all the strings are unicode', I'll need
> _some_ way to construct these packets.

Maybe you are missing the point here, maybe not: there is no
debate that Python should always have a byte string type (although
there is debate on whether that type should be mutable).

The current question is whether you want to denote objects of
the byte string type *in source code*. I.e. do you have the "Lots
of audio data" stored in .py files?

Regards,
Martin

From anthony at interlink.com.au  Tue Aug 17 09:17:13 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug 17 09:17:30 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4121AE6F.4090904@v.loewis.de>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au> <4121AE6F.4090904@v.loewis.de>
Message-ID: <4121B0F9.4080309@interlink.com.au>

Martin v. L?wis wrote:
> The current question is whether you want to denote objects of
> the byte string type *in source code*. I.e. do you have the "Lots
> of audio data" stored in .py files?

Generally, no - with the exception of test cases. In that case, I
often end up with byte literals in the source code. (To check that
a particular en/de coding operation Did The Right Thing).

Anthony
From t-meyer at ihug.co.nz  Tue Aug 17 09:24:26 2004
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Tue Aug 17 09:24:33 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E89625B6@its-xchg4.massey.ac.nz>
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E86C36D8@its-xchg4.massey.ac.nz>

[Martin v. L?wis]
> Maybe you are missing the point here, maybe not: there is no 
> debate that Python should always have a byte string type 
> (although there is debate on whether that type should be mutable).
> 
> The current question is whether you want to denote objects of 
> the byte string type *in source code*. I.e. do you have the 
> "Lots of audio data" stored in .py files?

I don't know about shtoom, but Mike Fletcher's resourcepackage does.  (It
packages up non-*.py files as *.py files).  I believe that the main point of
resourcepackage is for ease of distribution (but could be wrong), so perhaps
better distribution tools make this unnecessary.  It is an example use-case,
though.

=Tony Meyer

From mal at egenix.com  Tue Aug 17 09:32:38 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue Aug 17 09:32:33 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41213D6D.50302@v.loewis.de>
References: <1092258146.411a896246d1a@mcherm.com>	<200408120156.i7C1ugqF011681@cosc353.cosc.canterbury.ac.nz>	<16666.56033.65067.241794@montanaro.dyndns.org>	<411AFBD4.5080903@v.loewis.de><411B0DA0.4030109@v.loewis.de>	<411BB50C.8000804@v.loewis.de>	<16667.54746.535997.120878@montanaro.dyndns.org>	<411C85B6.2060104@egenix.com>	<411DC3DD.3080900@v.loewis.de>
	<4120FCA6.2070104@egenix.com> <41213D6D.50302@v.loewis.de>
Message-ID: <4121B496.6050207@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
> 
>> It is if you stick to writing your binary data using an ASCII
>> compatible encoding -- I wouldn't expect any other encoding for
>> binary data anyway. The most common are ASCII + escape sequences,
>> base64 or hex, all of which are ASCII compatible.
> 
> 
> We probably have a different notion of "ASCII compatible" then.
> I would define it as:
> 
>  An encoding E is "ASCII compatbible" if strings that only consist
>  of ASCII characters use the same byte representation in E that
>  they use in ASCII.
> 
> In that sense, ISO-8859-1 and UTF-8 are also ASCII compatible. Notice
> that this is also the definition that PEP 263 assumes.

Sorry, wrong wording on my part: I meant a string literal
that only uses ASCII characters for the literal definition,
i.e. literaldefinition.decode('ascii').encode('ascii') ==
literaldefinition.

> However, byte strings used in source code are not "safe" if they
> are encoded in ISO-8859-1 under recoding: If the source code is
> converted to UTF-8 (including the encoding declaration), then
> the length of the strings changes, as do the byte values inside
> the string.

Agreed.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 17 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From guido at python.org  Tue Aug 17 12:45:26 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 12:45:38 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: Your message of "Tue, 17 Aug 2004 16:54:35 +1000."
	<4121ABAB.9060605@interlink.com.au> 
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org> 
	<4121ABAB.9060605@interlink.com.au> 
Message-ID: <200408171045.i7HAjQZ15411@guido.python.org>

> Guido van Rossum wrote:
> > (I'm still not sold on the concept of bytes literals at all.)

[Anthony]
> Ok. Here's a case - in shtoom, I generate audio data. Lots
> of audio data. This is broken into packets, then gets a small
> header put onto each RTP packet. Right now, I'm using strings
> for this. If there was a 'byte literal', I'd use it. This isn't
> a huge problem right now, because strings are good enough. But
> if we end up in an 'all the strings are unicode', I'll need
> _some_ way to construct these packets.

I see that as a huge case for a bytes type, which I've proposed
myself; but what's the use case for bytes literals, assuming you can
write bytes("foo")?  Does b"foo" really make much of a difference?  Is
it so hard to have to write bytes([0x66, 0x6f, 0x6f]) instead of
b"\x66\x6f\x6f"?

IOW, how many *literal* packet fragments are in shtoom?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From anthony at interlink.com.au  Tue Aug 17 13:13:29 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug 17 13:13:45 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <200408171045.i7HAjQZ15411@guido.python.org>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
Message-ID: <4121E859.8000908@interlink.com.au>

Guido van Rossum wrote:
> I see that as a huge case for a bytes type, which I've proposed
> myself; but what's the use case for bytes literals, assuming you can
> write bytes("foo")?  Does b"foo" really make much of a difference?  Is
> it so hard to have to write bytes([0x66, 0x6f, 0x6f]) instead of
> b"\x66\x6f\x6f"?

It's a pretty marginal case for it. I just played with it a bit, and
I think after playing with it, I actually prefer the non b'' case.

A big +1 for a bytes() type, though. I'm not sure on the details,
but it'd be nice if it was possible to pass a bytes() object to,
for instance, write() directly.

From arigo at tunes.org  Tue Aug 17 13:16:22 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue Aug 17 13:21:02 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: <200408170121.i7H1Ltn14417@guido.python.org>
References: <001501c48310$bf15aa40$e841fea9@oemcomputer>
	<20040816113357.GB19969@vicky.ecs.soton.ac.uk>
	<200408170121.i7H1Ltn14417@guido.python.org>
Message-ID: <20040817111622.GA12954@vicky.ecs.soton.ac.uk>

Hello,

On Mon, Aug 16, 2004 at 06:21:55PM -0700, Guido van Rossum wrote:
> But why offer two ways of doing the same thing?  Apart from b/w
> compatibility, what's the advantage of using sys.exitfunc directly?

Well, none, really.  But let's not change the name sys.exitfunc just for the
sake of deprecation, because it will probably break existing and well-behaving
modules (not just non-well-behaving ones).


Armin
From gmccaughan at synaptics-uk.com  Tue Aug 17 14:19:24 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Tue Aug 17 14:20:07 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4121E859.8000908@interlink.com.au>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
Message-ID: <200408171319.24084.gmccaughan@synaptics-uk.com>

On Tuesday 2004-08-17 12:13, Anthony Baxter wrote:
> Guido van Rossum wrote:
> > I see that as a huge case for a bytes type, which I've proposed
> > myself; but what's the use case for bytes literals, assuming you can
> > write bytes("foo")?  Does b"foo" really make much of a difference?  Is
> > it so hard to have to write bytes([0x66, 0x6f, 0x6f]) instead of
> > b"\x66\x6f\x6f"?
> 
> It's a pretty marginal case for it. I just played with it a bit, and
> I think after playing with it, I actually prefer the non b'' case.

Another option, with pros and cons of its own: something along
the lines of b"666f6f".

-- 
g

From mwh at python.net  Tue Aug 17 14:49:21 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug 17 14:49:23 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4121E859.8000908@interlink.com.au> (Anthony Baxter's message
	of "Tue, 17 Aug 2004 21:13:29 +1000")
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
Message-ID: <2mbrhaaq0e.fsf@starship.python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

> Guido van Rossum wrote:
>> I see that as a huge case for a bytes type, which I've proposed
>> myself; but what's the use case for bytes literals, assuming you can
>> write bytes("foo")?  Does b"foo" really make much of a difference?  Is
>> it so hard to have to write bytes([0x66, 0x6f, 0x6f]) instead of
>> b"\x66\x6f\x6f"?
>
> It's a pretty marginal case for it. I just played with it a bit, and
> I think after playing with it, I actually prefer the non b'' case.

Is this getting to (hopefully uncontroverisal!) PEP time?

Is there any consensus forming on whether bytes() instances are
mutable or not?

> A big +1 for a bytes() type, though. I'm not sure on the details,
> but it'd be nice if it was possible to pass a bytes() object to,
> for instance, write() directly.

If bytes() doesn't implement the read buffer interface, someone
somewhere is going to need shooting :-)

Cheers,
mwh

-- 
  <Yosomono> rasterman is the millionth monkey
                                                -- from Twisted.Quotes
From barry at python.org  Tue Aug 17 16:25:52 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 17 16:25:28 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E86C36D8@its-xchg4.massey.ac.nz>
References: <ECBA357DDED63B4995F5C1F5CBE5B1E86C36D8@its-xchg4.massey.ac.nz>
Message-ID: <1092752752.8622.32.camel@localhost>

On Tue, 2004-08-17 at 03:24, Tony Meyer wrote:

> I don't know about shtoom, but Mike Fletcher's resourcepackage does.  (It
> packages up non-*.py files as *.py files).  I believe that the main point of
> resourcepackage is for ease of distribution (but could be wrong), so perhaps
> better distribution tools make this unnecessary.  It is an example use-case,
> though.

I'd think that in most cases, huge globs of binary byte data would
better live in non-py data files.  I personally don't see much of a use
case for byte literals (although, +1 on a bytes type).  Byte literals
just aren't going to be commonly written by people I think, and any
programmatic generation of byte data should just as easily live outside
py files.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040817/3586cfdb/attachment.pgp
From skip at pobox.com  Tue Aug 17 15:26:31 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 17 16:41:41 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <2mbrhaaq0e.fsf@starship.python.net>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
Message-ID: <16674.1927.230587.346903@montanaro.dyndns.org>


    Michael> Is this getting to (hopefully uncontroverisal!) PEP time?

One would hope.  I sent in a skeletal PEP last week asking for a number but
haven't heard back.

    Michael> Is there any consensus forming on whether bytes() instances are
    Michael> mutable or not?

ISTR Guido thought mutable was the way to go.  I don't think that efficiency
concerns will be a huge deal since it won't be used all over the place the
way strings are.  Will they need to be used as dict keys?  Doesn't seem
likely to me.

Skip
From anthony at interlink.com.au  Tue Aug 17 16:49:54 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug 17 16:50:09 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <16674.1927.230587.346903@montanaro.dyndns.org>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<200408170134.i7H1YDR14560@guido.python.org>	<4121ABAB.9060605@interlink.com.au>	<200408171045.i7HAjQZ15411@guido.python.org>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>
	<16674.1927.230587.346903@montanaro.dyndns.org>
Message-ID: <41221B12.4080808@interlink.com.au>


> ISTR Guido thought mutable was the way to go.  I don't think that efficiency
> concerns will be a huge deal since it won't be used all over the place the
> way strings are.  Will they need to be used as dict keys?  Doesn't seem
> likely to me.

Mutable is far far more useful. I can't see them being used commonly
for dict keys, but I do know of a lot of protocols where you have
to go back and patch a 'length' field in a packet header after you've
finished the packet construction.

Maybe we could use the @ symbol for the byte literal? <wink>

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From mwh at python.net  Tue Aug 17 16:57:15 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug 17 16:57:17 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <16674.1927.230587.346903@montanaro.dyndns.org> (Skip
	Montanaro's message of "Tue, 17 Aug 2004 08:26:31 -0500")
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
	<16674.1927.230587.346903@montanaro.dyndns.org>
Message-ID: <2mvffhak38.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

>     Michael> Is this getting to (hopefully uncontroverisal!) PEP time?
>
> One would hope.  I sent in a skeletal PEP last week asking for a number but
> haven't heard back.

Goody.

>     Michael> Is there any consensus forming on whether bytes() instances are
>     Michael> mutable or not?
>
> ISTR Guido thought mutable was the way to go.

OK.

> I don't think that efficiency concerns will be a huge deal since it
> won't be used all over the place the way strings are.

Who cares? :-)

> Will they need to be used as dict keys?  Doesn't seem likely to me.

This is indeed the question.  I agree it's unlikely, and you can
presumably do something like convert a bytes() instance into a tuple
of small integers if you really want to key into a dict.

Cheers,
mwh

-- 
  Clue: You've got the appropriate amount of hostility for the
  Monastery, however you are metaphorically getting out of the
  safari jeep and kicking the lions.                         -- coonec
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
From guido at python.org  Tue Aug 17 17:27:20 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 17:27:26 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: Your message of "Tue, 17 Aug 2004 21:13:29 +1000."
	<4121E859.8000908@interlink.com.au> 
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org> 
	<4121E859.8000908@interlink.com.au> 
Message-ID: <200408171527.i7HFRKV16082@guido.python.org>

> A big +1 for a bytes() type, though. I'm not sure on the details,
> but it'd be nice if it was possible to pass a bytes() object to,
> for instance, write() directly.

That should already possible with the array module, but somehow it
doesn't quite work, even though the array type appears to support the
buffer API.  (Does anybody understand why not?)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug 17 17:28:03 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 17:28:08 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: Your message of "Tue, 17 Aug 2004 12:16:22 BST."
	<20040817111622.GA12954@vicky.ecs.soton.ac.uk> 
References: <001501c48310$bf15aa40$e841fea9@oemcomputer>
	<20040816113357.GB19969@vicky.ecs.soton.ac.uk>
	<200408170121.i7H1Ltn14417@guido.python.org> 
	<20040817111622.GA12954@vicky.ecs.soton.ac.uk> 
Message-ID: <200408171528.i7HFS3t16093@guido.python.org>

> > But why offer two ways of doing the same thing?  Apart from b/w
> > compatibility, what's the advantage of using sys.exitfunc directly?
> 
> Well, none, really.  But let's not change the name sys.exitfunc just
> for the sake of deprecation, because it will probably break existing
> and well-behaving modules (not just non-well-behaving ones).

That's never been a reason not to deprecate something.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug 17 17:31:30 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 17:31:36 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: Your message of "Tue, 17 Aug 2004 13:49:21 BST."
	<2mbrhaaq0e.fsf@starship.python.net> 
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au> 
	<2mbrhaaq0e.fsf@starship.python.net> 
Message-ID: <200408171531.i7HFVUx16139@guido.python.org>

> Is this getting to (hopefully uncontroverisal!) PEP time?

Sure.

> Is there any consensus forming on whether bytes() instances are
> mutable or not?

Mutable!

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug 17 17:33:07 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 17:33:13 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: Your message of "Tue, 17 Aug 2004 15:57:15 BST."
	<2mvffhak38.fsf@starship.python.net> 
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
	<16674.1927.230587.346903@montanaro.dyndns.org> 
	<2mvffhak38.fsf@starship.python.net> 
Message-ID: <200408171533.i7HFX7416165@guido.python.org>

> This is indeed the question.  I agree it's unlikely, and you can
> presumably do something like convert a bytes() instance into a tuple
> of small integers if you really want to key into a dict.

Or even a Unicode string using the Latin-1 encoding. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From pje at telecommunity.com  Tue Aug 17 17:48:04 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug 17 17:49:11 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <200408171531.i7HFVUx16139@guido.python.org>
References: <Your message of "Tue,
	17 Aug 2004 13:49:21 BST." <2mbrhaaq0e.fsf@starship.python.net>
	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
Message-ID: <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>

At 08:31 AM 8/17/04 -0700, Guido van Rossum wrote:
> > Is this getting to (hopefully uncontroverisal!) PEP time?
>
>Sure.
>
> > Is there any consensus forming on whether bytes() instances are
> > mutable or not?
>
>Mutable!


So, how will it be different from:

     from array import array

     def bytes(*initializer):
         return array('B',*initializer)

Even if it's desirable for 'bytes' to be an actual type (e.g. subclassing 
ArrayType), it might help the definition process to describe the difference 
between the new type and a byte array.

From mal at egenix.com  Tue Aug 17 18:20:30 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue Aug 17 18:20:34 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <2mbrhaaq0e.fsf@starship.python.net>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<200408170134.i7H1YDR14560@guido.python.org>	<4121ABAB.9060605@interlink.com.au>	<200408171045.i7HAjQZ15411@guido.python.org>	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
Message-ID: <4122304E.1070907@egenix.com>

Michael Hudson wrote:
> Anthony Baxter <anthony@interlink.com.au> writes:
>>A big +1 for a bytes() type, though. I'm not sure on the details,
>>but it'd be nice if it was possible to pass a bytes() object to,
>>for instance, write() directly.
> 
> If bytes() doesn't implement the read buffer interface, someone
> somewhere is going to need shooting :-)

Is there any reason you cannot use buffer() ?!

It already implements all the necessary things and has been
available for many years.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 17 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From arigo at tunes.org  Tue Aug 17 18:18:33 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue Aug 17 18:23:05 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: <200408171528.i7HFS3t16093@guido.python.org>
References: <001501c48310$bf15aa40$e841fea9@oemcomputer>
	<20040816113357.GB19969@vicky.ecs.soton.ac.uk>
	<200408170121.i7H1Ltn14417@guido.python.org>
	<20040817111622.GA12954@vicky.ecs.soton.ac.uk>
	<200408171528.i7HFS3t16093@guido.python.org>
Message-ID: <20040817161833.GA29635@vicky.ecs.soton.ac.uk>

Hello Guido,

On Tue, Aug 17, 2004 at 08:28:03AM -0700, Guido van Rossum wrote:
> > Well, none, really.  But let's not change the name sys.exitfunc just
> > for the sake of deprecation, because it will probably break existing
> > and well-behaving modules (not just non-well-behaving ones).
> 
> That's never been a reason not to deprecate something.

Sorry.  Sure.  What I was opposing here is Raymond's original claim in this 
thread, which was apparently the reason he wanted to deprecate sys.exitfunc:

"""
The atexit module does attempt to co-exist by introducing code to
register a function in sys.exitfunc if it were defined before "import
atexit" was called.  However, this is unreliable because it depends on
import order and the library is free to introduce an earlier "import
atexit" which would break code relying on the co-existance mechanism.
"""

I claim that there is nothing unreliable or depending on import order here, as
long as all concerned parties do the right thing.  Now if there are other good
reasons to deprecate sys.exitfunc, like it being another way of doing
something for which a better interface is provided, then fine.


Armin
From skip at pobox.com  Tue Aug 17 18:23:10 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 17 18:23:25 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: <200408171528.i7HFS3t16093@guido.python.org>
References: <001501c48310$bf15aa40$e841fea9@oemcomputer>
	<20040816113357.GB19969@vicky.ecs.soton.ac.uk>
	<200408170121.i7H1Ltn14417@guido.python.org>
	<20040817111622.GA12954@vicky.ecs.soton.ac.uk>
	<200408171528.i7HFS3t16093@guido.python.org>
Message-ID: <16674.12526.711803.480295@montanaro.dyndns.org>


    >> > But why offer two ways of doing the same thing?  Apart from b/w
    >> > compatibility, what's the advantage of using sys.exitfunc directly?
    >> 
    >> Well, none, really.  But let's not change the name sys.exitfunc just
    >> for the sake of deprecation, because it will probably break existing
    >> and well-behaving modules (not just non-well-behaving ones).

    Guido> That's never been a reason not to deprecate something.

Care to reword that without the double negative?  I tried and came up with

    That's sometimes a reason to deprecate something.

which seems like you're saying that breaking existing well-behaving modules
is sometimes a good reason to deprecate something. 

not-sure-if-i-should-wink-ly, y'rs,

Skip
From mcherm at mcherm.com  Tue Aug 17 18:40:02 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Aug 17 18:37:57 2004
Subject: [Python-Dev] Decorator order implemented backwards?
Message-ID: <1092760802.412234e2cb4d5@mcherm.com>

Mark writes:
> Your patch results in the evaluation order:
>
>     evalname1 evalargs1 makedec1
>     evalname2 evalargs2 makedec2
>     evalname3 evalargs3 makedec3
>     calldec3 calldec2 calldec1
>
> Mine (#1009560) gives:
>
>     evalname3 evalargs3 makedec3 calldec3
>     evalname2 evalargs2 makedec2 calldec2
>     evalname1 evalargs1 makedec1 calldec1

Guido writes:
> Since Mark ended up agreeing with your order, the OK is given by
> default. :-)

Wait... I'm confused. What was the final decision? I favor

     evalname1 evalargs1 makedec1
     evalname2 evalargs2 makedec2
     evalname3 evalargs3 makedec3
     calldec3 calldec2 calldec1

becase of left-to-right-top-to-bottom evaluation. Is this what it
actually does? I imagine this:

      >>> # Warning - not actual code
      >>> def dummyDecorator1(x):
      ...     print 'one'
      ...     return x
      ...
      >>> def dummyDecorator2(x):
      ...     print 'two'
      ...     return x
      ...
      >>> @dummyDecorator1
      >>> @dummyDecorator2
      >>> def func(): pass
      two
      one

which somehow seems wrong to me.

-- Michael Chermside

From python at rcn.com  Tue Aug 17 18:38:15 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug 17 18:39:38 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: <20040817161833.GA29635@vicky.ecs.soton.ac.uk>
Message-ID: <002f01c48478$98b112e0$e841fea9@oemcomputer>

[Armin]
> I claim that there is nothing unreliable or depending on import order
> here, as
> long as all concerned parties do the right thing.

Right.  It's only unreliable if a party takes an obvious, but not wise
approach.


>  Now if there are other
> good
> reasons to deprecate sys.exitfunc, like it being another way of doing
> something for which a better interface is provided, then fine.

That is the case.  The API for at exit accepts function arguments and
automatically handles multiple exit functions.  It is careful to handle
the multiple exits using pop() in case another exit function alters the
function list or in case multiple threads are using atexit.  So, yes
there is a better API that should be the one preferred way to do it.


Raymond

From mcherm at mcherm.com  Tue Aug 17 18:47:37 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Aug 17 18:45:27 2004
Subject: [Python-Dev] Decorator order implemented backwards?
Message-ID: <1092761257.412236a9e7f9f@mcherm.com>

I wrote:
> Wait... I'm confused. What was the final decision? I favor
     [...]

Never mind. I somehow missed Mark's second email.

-- Michael Chermside

From pje at telecommunity.com  Tue Aug 17 18:52:59 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug 17 18:53:56 2004
Subject: [Python-Dev] Decorator order implemented backwards?
In-Reply-To: <1092760802.412234e2cb4d5@mcherm.com>
Message-ID: <5.1.1.6.0.20040817124720.02d28640@mail.telecommunity.com>

At 09:40 AM 8/17/04 -0700, Michael Chermside wrote:
>Mark writes:
> > Your patch results in the evaluation order:
> >
> >     evalname1 evalargs1 makedec1
> >     evalname2 evalargs2 makedec2
> >     evalname3 evalargs3 makedec3
> >     calldec3 calldec2 calldec1
> >
> > Mine (#1009560) gives:
> >
> >     evalname3 evalargs3 makedec3 calldec3
> >     evalname2 evalargs2 makedec2 calldec2
> >     evalname1 evalargs1 makedec1 calldec1
>
>Guido writes:
> > Since Mark ended up agreeing with your order, the OK is given by
> > default. :-)
>
>Wait... I'm confused. What was the final decision? I favor
>
>      evalname1 evalargs1 makedec1
>      evalname2 evalargs2 makedec2
>      evalname3 evalargs3 makedec3
>      calldec3 calldec2 calldec1
>
>becase of left-to-right-top-to-bottom evaluation. Is this what it
>actually does? I imagine this:

Hm.  Consider this code:

     func = a(b)(c(d)(e(func)))

Per the PEP, it's the equivalent of:

     @a(b)
     @c(d)
     @e
     def func(...):
         ...

Here's an annotated disassembly:

 >>> dis.dis(lambda: a(b)(c(d)(e(func))))
           0 SET_LINENO               1
           3 LOAD_GLOBAL              0 (a)  #evalname1
           6 LOAD_GLOBAL              1 (b)  #evalargs1
           9 CALL_FUNCTION            1      #makedec1
          12 LOAD_GLOBAL              2 (c)  #evalname2
          15 LOAD_GLOBAL              3 (d)  #evalargs2
          18 CALL_FUNCTION            1      #makedec2
          21 LOAD_GLOBAL              4 (e)  #evalname3
          24 LOAD_GLOBAL              5 (func)#defun
          27 CALL_FUNCTION            1       #calldec3
          30 CALL_FUNCTION            1       #calldec2
          33 CALL_FUNCTION            1       #calldec1
          36 RETURN_VALUE

Therefore, this is the evaluation order currently prescribed by the PEP.

From nas at arctrix.com  Tue Aug 17 19:00:05 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue Aug 17 19:00:10 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: <16674.12526.711803.480295@montanaro.dyndns.org>
References: <001501c48310$bf15aa40$e841fea9@oemcomputer>
	<20040816113357.GB19969@vicky.ecs.soton.ac.uk>
	<200408170121.i7H1Ltn14417@guido.python.org>
	<20040817111622.GA12954@vicky.ecs.soton.ac.uk>
	<200408171528.i7HFS3t16093@guido.python.org>
	<16674.12526.711803.480295@montanaro.dyndns.org>
Message-ID: <20040817170005.GA11751@mems-exchange.org>

On Tue, Aug 17, 2004 at 11:23:10AM -0500, Skip Montanaro wrote:
> Care to reword that without the double negative?  I tried and came
> up with
> 
>     That's sometimes a reason to deprecate something.
> 
> which seems like you're saying that breaking existing
> well-behaving modules is sometimes a good reason to deprecate
> something. 

I think Guido's point was that deprecating something is not the same
as making a backwards incompatible change.  It probably would be
good to deprecate features that will be removed in 3.0.

IMO, the sys.exitfunc should not be renamed to sys._exitfunc before
3.0.

  Neil
From amk at amk.ca  Tue Aug 17 19:16:22 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Tue Aug 17 19:17:11 2004
Subject: [Python-Dev] Python 3.0 list of goals
Message-ID: <20040817171622.GA10200@rogue.amk.ca>

The thread about bytes() is about a Python 3.0 feature.  Guido's
presentations have mentioned various changes he'd like to make in 3.0,
but there's no master list of things that would change.  

I think it would be useful to have such a list, because it would focus
our language development effort on ones that are steps to 3.0, and
maybe people can invent ways to smooth the transition.  I've started a
list in the Wiki at http://www.python.org/moin/Python3.0 , but should
it be a PEP?  (PEP 3000, perhaps?)

--amk

From mcherm at mcherm.com  Tue Aug 17 19:19:37 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Aug 17 19:17:26 2004
Subject: [Python-Dev] Find out whether PyEval_InitThreads has been called?
Message-ID: <1092763177.41223e29382cb@mcherm.com>

> Why not go one step further and make all locking
> operations in python no-ops if PyEval_InitThreads has never been
> called?  Currently, we risk python crashing if we call such functions
> without calling PyEval_InitThreads.

+1. Seems like it can't possibly hurt, and as you say, it's trivial
to implement.

-- Michael Chermside

From bac at OCF.Berkeley.EDU  Tue Aug 17 19:24:35 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Aug 17 19:24:44 2004
Subject: [Python-Dev] Python 3.0 list of goals
In-Reply-To: <20040817171622.GA10200@rogue.amk.ca>
References: <20040817171622.GA10200@rogue.amk.ca>
Message-ID: <41223F53.5060501@ocf.berkeley.edu>

A.M. Kuchling wrote:

> The thread about bytes() is about a Python 3.0 feature.  Guido's
> presentations have mentioned various changes he'd like to make in 3.0,
> but there's no master list of things that would change.  
> 
> I think it would be useful to have such a list, because it would focus
> our language development effort on ones that are steps to 3.0, and
> maybe people can invent ways to smooth the transition.  I've started a
> list in the Wiki at http://www.python.org/moin/Python3.0 , but should
> it be a PEP?  (PEP 3000, perhaps?)
> 

I say a PEP.  While it can say that these features are just 
possibilities, I would rather have a controlled place to list this stuff 
so it is a little more official than having someone randomly come in and 
start throwing in everything they wished Python 3.0 will have.

When it is decided where all of this info will end up I will go through 
the python-dev Summaries and see if I can find anything there (tried to 
make it a habit to doc stuff that Guido said would be a Python 3.0 feature.

There is also another wiki page on Python 3.0 at 
http://www.python.org/cgi-bin/moinmoin/PythonThreeDotOh .  I also have 
an initial list going.  At PyCON Michael McLay and I discussed getting a 
PEP written in hopes of getting grant funding from outside sources 
(read: not the PSF) to implement some of the features.  And we figured a 
PEP was probably as official as we could get short of a specific grant 
proposal and thus be more acceptable to people considering providing funds.

-Brett
From bac at OCF.Berkeley.EDU  Tue Aug 17 19:27:46 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Aug 17 19:27:56 2004
Subject: [Python-Dev] Python 3.0 list of goals
In-Reply-To: <41223F53.5060501@ocf.berkeley.edu>
References: <20040817171622.GA10200@rogue.amk.ca>
	<41223F53.5060501@ocf.berkeley.edu>
Message-ID: <41224012.9050500@ocf.berkeley.edu>

Brett C. wrote:

> A.M. Kuchling wrote:
> 
>> The thread about bytes() is about a Python 3.0 feature.  Guido's
>> presentations have mentioned various changes he'd like to make in 3.0,
>> but there's no master list of things that would change. 
>> I think it would be useful to have such a list, because it would focus
>> our language development effort on ones that are steps to 3.0, and
>> maybe people can invent ways to smooth the transition.  I've started a
>> list in the Wiki at http://www.python.org/moin/Python3.0 , but should
>> it be a PEP?  (PEP 3000, perhaps?)
>>
> 
> I say a PEP.  While it can say that these features are just 
> possibilities, I would rather have a controlled place to list this stuff 
> so it is a little more official than having someone randomly come in and 
> start throwing in everything they wished Python 3.0 will have.

And I forgot to mention I would like to help out with this.

-Brett
From rnd at onego.ru  Tue Aug 17 19:43:50 2004
From: rnd at onego.ru (Roman Suzi)
Date: Tue Aug 17 19:42:53 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4122304E.1070907@egenix.com>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com>
Message-ID: <Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>

On Tue, 17 Aug 2004, M.-A. Lemburg wrote:

>Michael Hudson wrote:
>> Anthony Baxter <anthony@interlink.com.au> writes:
>>>A big +1 for a bytes() type, though. I'm not sure on the details,
>>>but it'd be nice if it was possible to pass a bytes() object to,
>>>for instance, write() directly.
>>
>> If bytes() doesn't implement the read buffer interface, someone
>> somewhere is going to need shooting :-)
>
>Is there any reason you cannot use buffer() ?!

Is it mutable?
My guess: no:

>>> d = u'123124'
>>> ddd[0]
'1'
>>> ddd[1]
'\x00'
>>> ddd[1] = '1'
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: buffer is read-only

>It already implements all the necessary things and has been
>available for many years.

It was in the shadows because we had byte-strings.

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From janssen at parc.com  Tue Aug 17 21:09:22 2004
From: janssen at parc.com (Bill Janssen)
Date: Tue Aug 17 21:10:48 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python 
In-Reply-To: Your message of "Tue, 17 Aug 2004 03:45:26 PDT."
	<200408171045.i7HAjQZ15411@guido.python.org> 
Message-ID: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com>

> Does b"foo" really make much of a difference?

Yes.  My guess is that if you leave it out, you'll see

  var = u"foo".encode("ASCII")

all over the place (assuming that encode() will produce a bytes type).
Wouldn't b"foo" be more readable all around?

> Is it so hard to have to write bytes([0x66, 0x6f, 0x6f]) instead of
> b"\x66\x6f\x6f"?

No, that's true.  But if you have a bytes literal syntax, might as
well allow \x in it.

Bill
From barry at python.org  Tue Aug 17 21:16:29 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 17 21:16:06 2004
Subject: [Python-Dev] Python 3.0 list of goals
In-Reply-To: <20040817171622.GA10200@rogue.amk.ca>
References: <20040817171622.GA10200@rogue.amk.ca>
Message-ID: <1092770189.9170.89.camel@localhost>

On Tue, 2004-08-17 at 13:16, A.M. Kuchling wrote:

> it be a PEP?  (PEP 3000, perhaps?)

+1

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040817/060d410e/attachment.pgp
From martin at v.loewis.de  Tue Aug 17 22:07:46 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 17 22:07:48 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com>
References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com>
Message-ID: <41226592.1040109@v.loewis.de>

Bill Janssen wrote:
> Yes.  My guess is that if you leave it out, you'll see
> 
>   var = u"foo".encode("ASCII")
> 
> all over the place (assuming that encode() will produce a bytes type).

If you also had

    var = bytes(u"foo")

then I guess people would prefer that. People who want to save typing
can do

    b = bytes

and, given that the u prefix will be redundant, write

   var = b("foo")

Regards,
Martin
From bob at redivi.com  Tue Aug 17 22:20:49 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Aug 17 22:21:30 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41226592.1040109@v.loewis.de>
References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com>
	<41226592.1040109@v.loewis.de>
Message-ID: <EE2A4060-F08A-11D8-BEC0-000A95686CD8@redivi.com>


On Aug 17, 2004, at 4:07 PM, Martin v. L?wis wrote:

> Bill Janssen wrote:
>> Yes.  My guess is that if you leave it out, you'll see
>>   var = u"foo".encode("ASCII")
>> all over the place (assuming that encode() will produce a bytes type).
>
> If you also had
>
>    var = bytes(u"foo")
>
> then I guess people would prefer that. People who want to save typing
> can do
>
>    b = bytes
>
> and, given that the u prefix will be redundant, write
>
>   var = b("foo")

How would you embed raw bytes if the string was unicode?  Maybe there 
should be something roughly equivalent to this:

bytesvar = r"delimited packet\x00".decode("string_escape")

"string_escape" would probably be a bad name for it, of course.

-bob
From pgimpelj at sympatico.ca  Tue Aug 17 22:41:56 2004
From: pgimpelj at sympatico.ca (Paul Gimpelj)
Date: Tue Aug 17 22:41:11 2004
Subject: [Python-Dev] multiple instances of python on XP
Message-ID: <029201c4849a$a43763e0$c901010a@zoom>

hi,

I was wondering how to have multiple instances of python running on XP.

I have ZOPE and TSW installed, but ZOPE's python is grabbing paths from TSW's python.

specifically I

 changed to dir "...\Zope-2.7.2-0\bin"
and in cmd window  ran python

did help() , sys
and executable= r'...\Zope-2.7.2-0\bin\python.exe'
but exec_prefix = r'...\TSW\Apache2\python'
path_importer_cache = ...\TSW|Apache2\python...

TSW's pyhon is ver 2.3.4
ZOPE python is 2.3.3

so Is there a way to prevent this mixup?
I would like to have ZOPE installed as a service, and that means a python is running as a service.

I would like to multiple python services running, and have them stay in their own installed areas.

Is there a way to define the internal paths, and not to use XP's search  PATH

Would I have to rename python to python_1, Python_2, etc?

Thanks.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040817/296e39b1/attachment.htm
From martin at v.loewis.de  Tue Aug 17 23:11:06 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 17 23:11:24 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <EE2A4060-F08A-11D8-BEC0-000A95686CD8@redivi.com>
References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com>	<41226592.1040109@v.loewis.de>
	<EE2A4060-F08A-11D8-BEC0-000A95686CD8@redivi.com>
Message-ID: <4122746A.2010106@v.loewis.de>

Bob Ippolito wrote:
> How would you embed raw bytes if the string was unicode?

The most direct notation would be

    bytes("delimited packet\x00")

However, people might not understand what is happening, and
Guido doesn't like it if the bytes are >127.

Regards,
Martin
From martin at v.loewis.de  Tue Aug 17 23:14:56 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 17 23:14:55 2004
Subject: [Python-Dev] multiple instances of python on XP
In-Reply-To: <029201c4849a$a43763e0$c901010a@zoom>
References: <029201c4849a$a43763e0$c901010a@zoom>
Message-ID: <41227550.6030907@v.loewis.de>

Paul Gimpelj wrote:
> I was wondering how to have multiple instances of python running on XP.

python-dev is for the development *of* Python, not for the development
*with* Python; please post your question to an appropriate forum.

I see that you have also posted the question to python-help. I
personally think it is impolite to cross-post questions without
indicating so - people might try to research answers for you when
you already might have found them elsewhere.

Regards,
Martin
From bob at redivi.com  Tue Aug 17 23:18:38 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Aug 17 23:19:15 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4122746A.2010106@v.loewis.de>
References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com>	<41226592.1040109@v.loewis.de>
	<EE2A4060-F08A-11D8-BEC0-000A95686CD8@redivi.com>
	<4122746A.2010106@v.loewis.de>
Message-ID: <01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com>


On Aug 17, 2004, at 5:11 PM, Martin v. L?wis wrote:

> Bob Ippolito wrote:
>> How would you embed raw bytes if the string was unicode?
>
> The most direct notation would be
>
>    bytes("delimited packet\x00")
>
> However, people might not understand what is happening, and
> Guido doesn't like it if the bytes are >127.

I guess that was a bad example, what if the delimiter was \xff?  I know 
that map(ord, u'delimited packet\xff') would get correct results.. but 
I don't think I like that either.

-bob
From martin at v.loewis.de  Tue Aug 17 23:31:09 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 17 23:31:16 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com>
References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com>	<41226592.1040109@v.loewis.de>
	<EE2A4060-F08A-11D8-BEC0-000A95686CD8@redivi.com>
	<4122746A.2010106@v.loewis.de>
	<01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com>
Message-ID: <4122791D.7050003@v.loewis.de>

Bob Ippolito wrote:
> I guess that was a bad example, what if the delimiter was \xff?

I would write

    bytes("delimited packet\xff")

Repeating myself: Guido wouldn't.

Regards,
Martin
From guido at python.org  Tue Aug 17 23:33:54 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 23:34:01 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: Your message of "Tue, 17 Aug 2004 11:48:04 EDT."
	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> 
References: <Your message of "Tue,
	17 Aug 2004 13:49:21 BST." <2mbrhaaq0e.fsf@starship.python.net>
	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net> 
	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com> 
Message-ID: <200408172133.i7HLXsS16658@guido.python.org>

> So, how will it be different from:
> 
>      from array import array
> 
>      def bytes(*initializer):
>          return array('B',*initializer)
> 
> Even if it's desirable for 'bytes' to be an actual type (e.g. subclassing 
> ArrayType), it might help the definition process to describe the difference 
> between the new type and a byte array.

Not a whole lot different, except for the ability to use a string as
alternate argument to the constructor, and the fact that it's going to
be an actual type, and that it should support the buffer API (which
array mysteriously doesn't?).

The string argument support may not even be necessary -- an
alternative way to spell that would be to let s.decode() return a
bytes object, which has the advantage of being explicit about the
encoding; there's even a base64 encoding already!  But it would be a
bigger incompatibility, more likely to break existing code using
decode() and expecting to get a string.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From pgimpelj at sympatico.ca  Tue Aug 17 23:48:47 2004
From: pgimpelj at sympatico.ca (Paul Gimpelj)
Date: Tue Aug 17 23:48:00 2004
Subject: [Python-Dev] multiple instances of python on XP
References: <029201c4849a$a43763e0$c901010a@zoom>
	<41227550.6030907@v.loewis.de>
Message-ID: <02e001c484a3$faed7c20$c901010a@zoom>

I have searched all the documentation online and google.

I thought that developers would interested in public input.

and I got email back that everybody is on vacation till sept 6.

Since it is open source,

I would think that this an import issue since a python advocate  would want
python's user base to increase.

And if python , by design, is incapable of coexisting with all the other
user programs and itself,
what use is it? on the most popular operating system in the entire world.

I do work with linux.


regards,
Paul


----- Original Message -----
From: "Martin v. L?wis" <martin@v.loewis.de>
To: "Paul Gimpelj" <pgimpelj@sympatico.ca>
Cc: <Python-Dev@python.org>
Sent: Tuesday, August 17, 2004 5:14 PM
Subject: Re: [Python-Dev] multiple instances of python on XP


> Paul Gimpelj wrote:
> > I was wondering how to have multiple instances of python running on XP.
>
> python-dev is for the development *of* Python, not for the development
> *with* Python; please post your question to an appropriate forum.
>
> I see that you have also posted the question to python-help. I
> personally think it is impolite to cross-post questions without
> indicating so - people might try to research answers for you when
> you already might have found them elsewhere.
>
> Regards,
> Martin
>

From guido at python.org  Tue Aug 17 23:48:35 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 23:48:42 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: Your message of "Tue, 17 Aug 2004 17:18:33 BST."
	<20040817161833.GA29635@vicky.ecs.soton.ac.uk> 
References: <001501c48310$bf15aa40$e841fea9@oemcomputer>
	<20040816113357.GB19969@vicky.ecs.soton.ac.uk>
	<200408170121.i7H1Ltn14417@guido.python.org>
	<20040817111622.GA12954@vicky.ecs.soton.ac.uk>
	<200408171528.i7HFS3t16093@guido.python.org> 
	<20040817161833.GA29635@vicky.ecs.soton.ac.uk> 
Message-ID: <200408172148.i7HLmZx16688@guido.python.org>

> On Tue, Aug 17, 2004 at 08:28:03AM -0700, Guido van Rossum wrote:
> > > Well, none, really.  But let's not change the name sys.exitfunc just
> > > for the sake of deprecation, because it will probably break existing
> > > and well-behaving modules (not just non-well-behaving ones).
> > 
> > That's never been a reason not to deprecate something.
> 
> Sorry.  Sure.  What I was opposing here is Raymond's original claim in this 
> thread, which was apparently the reason he wanted to deprecate sys.exitfunc:
> 
> """
> The atexit module does attempt to co-exist by introducing code to
> register a function in sys.exitfunc if it were defined before "import
> atexit" was called.  However, this is unreliable because it depends on
> import order and the library is free to introduce an earlier "import
> atexit" which would break code relying on the co-existance mechanism.
> """
> 
> I claim that there is nothing unreliable or depending on import
> order here, as long as all concerned parties do the right thing.
> Now if there are other good reasons to deprecate sys.exitfunc, like
> it being another way of doing something for which a better interface
> is provided, then fine.

Raymond may have overstated his case somewhat.  But the fact is that
it is easy to abuse sys.exitfunc with a poorly written handler, and
the atexit module avoids this.  This in itself to me looks like a
classic "deprecate the old solution, recommend the new way".  TOOWTDI!

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Tue Aug 17 23:50:13 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 17 23:50:20 2004
Subject: [Python-Dev] Deprecate sys.exitfunc?
In-Reply-To: Your message of "Tue, 17 Aug 2004 13:00:05 EDT."
	<20040817170005.GA11751@mems-exchange.org> 
References: <001501c48310$bf15aa40$e841fea9@oemcomputer>
	<20040816113357.GB19969@vicky.ecs.soton.ac.uk>
	<200408170121.i7H1Ltn14417@guido.python.org>
	<20040817111622.GA12954@vicky.ecs.soton.ac.uk>
	<200408171528.i7HFS3t16093@guido.python.org>
	<16674.12526.711803.480295@montanaro.dyndns.org> 
	<20040817170005.GA11751@mems-exchange.org> 
Message-ID: <200408172150.i7HLoEL16716@guido.python.org>

> IMO, the sys.exitfunc should not be renamed to sys._exitfunc before
> 3.0.

Agreed.  But it can be deprecated right away.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From mal at egenix.com  Tue Aug 17 23:58:27 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue Aug 17 23:58:31 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>
	<4122304E.1070907@egenix.com>
	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>
Message-ID: <41227F83.4060403@egenix.com>

Roman Suzi wrote:
> On Tue, 17 Aug 2004, M.-A. Lemburg wrote:
> 
> 
>>Michael Hudson wrote:
>>
>>>Anthony Baxter <anthony@interlink.com.au> writes:
>>>
>>>>A big +1 for a bytes() type, though. I'm not sure on the details,
>>>>but it'd be nice if it was possible to pass a bytes() object to,
>>>>for instance, write() directly.
>>>
>>>If bytes() doesn't implement the read buffer interface, someone
>>>somewhere is going to need shooting :-)
>>
>>Is there any reason you cannot use buffer() ?!
> 
> Is it mutable?
> My guess: no:

The buffer object itself can be read-only or read-write.
Unfortunately, the buffer() built-in always returns
read-only buffers. At C level it is easy to create a buffer
object from a read-write capable object.

>>>>d = u'123124'
>>>>ddd[0]
> 
> '1'
> 
>>>>ddd[1]
> 
> '\x00'
> 
>>>>ddd[1] = '1'
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: buffer is read-only
> 
> 
>>It already implements all the necessary things and has been
>>available for many years.
> 
> 
> It was in the shadows because we had byte-strings.

Right, so why not revive it ?!

Anyway, this whole discussion about a new bytes type doesn't
really solve the problem that the b'...' literal was
intended for: that of having a nice way to define (read-only)
8-bit binary string literals.

We already have a number of read-write types for storing binary
data, e.g. arrays, cStringIO and buffers. Inventing yet another
way to spell binary data won't make life easier.

However, what will be missing is a nice way to spell read-only
binary data.

Since 'tada' will return a Unicode object in Py3k, I think we
should reuse the existing 8-bit string object under the new
literal constructor b'tada\x00' (and apply the same source code
encoding semantics we apply today for 'tada\x00').

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 17 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From bob at redivi.com  Tue Aug 17 23:59:19 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Aug 17 23:59:58 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <200408172133.i7HLXsS16658@guido.python.org>
References: <Your message of "Tue,
	17 Aug 2004 13:49:21 BST." <2mbrhaaq0e.fsf@starship.python.net>
	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>
	<200408172133.i7HLXsS16658@guido.python.org>
Message-ID: <B0627378-F098-11D8-933F-000A95686CD8@redivi.com>


On Aug 17, 2004, at 5:33 PM, Guido van Rossum wrote:

>> So, how will it be different from:
>>
>>      from array import array
>>
>>      def bytes(*initializer):
>>          return array('B',*initializer)
>>
>> Even if it's desirable for 'bytes' to be an actual type (e.g. 
>> subclassing
>> ArrayType), it might help the definition process to describe the 
>> difference
>> between the new type and a byte array.
>
> Not a whole lot different, except for the ability to use a string as
> alternate argument to the constructor, and the fact that it's going to
> be an actual type, and that it should support the buffer API (which
> array mysteriously doesn't?).
>
> The string argument support may not even be necessary -- an
> alternative way to spell that would be to let s.decode() return a
> bytes object, which has the advantage of being explicit about the
> encoding; there's even a base64 encoding already!  But it would be a
> bigger incompatibility, more likely to break existing code using
> decode() and expecting to get a string.

IMHO current uses of decode and encode are really confusing.  Many 
decodes are from str -> unicode, and many encodes are from unicode -> 
str (or str -> unicode -> str implicitly, which is usually going to 
fail miserably)... while yet others like zlib, base64, etc. are str <-> 
str.  Technically unicode.decode(base64) should certainly work, but it 
doesn't because unicode doesn't have a decode method.

I don't have a proposed solution at the moment, but perhaps these 
operations should either be outside of the data types altogether (i.e. 
use codecs only) or there should be separate methods for doing separate 
things (character translations versus data->data transformations).

-bob
From guido at python.org  Wed Aug 18 00:01:12 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug 18 00:01:21 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: Your message of "Tue, 17 Aug 2004 17:18:38 EDT."
	<01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com> 
References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com>
	<41226592.1040109@v.loewis.de>
	<EE2A4060-F08A-11D8-BEC0-000A95686CD8@redivi.com>
	<4122746A.2010106@v.loewis.de> 
	<01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com> 
Message-ID: <200408172201.i7HM1CT16805@guido.python.org>

> >> How would you embed raw bytes if the string was unicode?
> >
> > The most direct notation would be
> >
> >    bytes("delimited packet\x00")
> >
> > However, people might not understand what is happening, and
> > Guido doesn't like it if the bytes are >127.
> 
> I guess that was a bad example, what if the delimiter was \xff?  I know 
> that map(ord, u'delimited packet\xff') would get correct results.. but 
> I don't think I like that either.

Maybe the constructor could be bytes(<string>[, <encoding>])?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From foom at fuhm.net  Wed Aug 18 00:04:45 2004
From: foom at fuhm.net (James Y Knight)
Date: Wed Aug 18 00:04:49 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com>
References: <04Aug17.120927pdt."58612"@synergy1.parc.xerox.com>	<41226592.1040109@v.loewis.de>
	<EE2A4060-F08A-11D8-BEC0-000A95686CD8@redivi.com>
	<4122746A.2010106@v.loewis.de>
	<01C28AC3-F093-11D8-933F-000A95686CD8@redivi.com>
Message-ID: <72B8535D-F099-11D8-8449-000A95A50FB2@fuhm.net>


On Aug 17, 2004, at 5:18 PM, Bob Ippolito wrote:

> On Aug 17, 2004, at 5:11 PM, Martin v. L?wis wrote:
>
>> Bob Ippolito wrote:
>>> How would you embed raw bytes if the string was unicode?
>>
>> The most direct notation would be
>>
>>    bytes("delimited packet\x00")
>>
>> However, people might not understand what is happening, and
>> Guido doesn't like it if the bytes are >127.
>
> I guess that was a bad example, what if the delimiter was \xff?

Indeed, if all strings are unicode, the question becomes: what encoding 
does bytes() use to translate unicode characters to bytes. Two 
alternatives have been proposed so far:
1) ASCII (translate chars as their codepoint if  < 128, else error)
2) ISO-8859-1 (translate chars as their codepoint if < 256, else error)

I think I'd choose #2, myself.

> I know that map(ord, u'delimited packet\xff') would get correct 
> results.. but I don't think I like that either.

Why would you consider that wrong? ord(u'\xff') *should* return 255. 
Just as ord(u'\u1000') returns 4096. There's nothing mysterious there.

James
From mal at egenix.com  Wed Aug 18 00:25:21 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 00:25:27 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <411D1EB4.10103@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>
	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>
	<411D1EB4.10103@livinglogic.de>
Message-ID: <412285D1.7010907@egenix.com>

Walter D?rwald wrote:
> M.-A. Lemburg wrote:
>> Overall, I don't like the idea of adding extra
>> APIs breaking the existing codec API.
> 
> Adding a final argument that defaults to False doesn't
> break the API for the callers, only for the implementors.
> 
> And if we drop the final argument, the API is completely
> backwards compatible both for users and implementors.
> The only thing that gets added is the feed() method,
> that implementors don't have to overwrite.
> 
>> I believe that we can
>> extend stream codecs to also work in a feed mode without
>> breaking the API.
> 
> Abandoning the final argument and adding a feed() method
> would IMHO be the simplest way to do this.
> 
> But then there's no way to make sure that every byte from
> the input stream is really consumed.

I've thought about this some more. Perhaps I'm still missing
something, but wouldn't it be possible to add a feeding
mode to the existing stream codecs by creating a new queue
data type (much like the queue you have in the test cases of
your patch) and using the stream codecs on these ?

I think such a queue would be generally useful in other
contexts as well, e.g. for implementing fast character based
pipes between threads, non-Unicode feeding parsers, etc.
Using such a type you could potentially add a feeding
mode to stream or file-object API based algorithms very
easily.

We could then have a new class, e.g. FeedReader, which
wraps the above in a nice API, much like StreamReaderWriter
and StreamRecoder.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 17 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From bac at OCF.Berkeley.EDU  Wed Aug 18 00:29:25 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Aug 18 00:29:33 2004
Subject: [Python-Dev] multiple instances of python on XP
In-Reply-To: <02e001c484a3$faed7c20$c901010a@zoom>
References: <029201c4849a$a43763e0$c901010a@zoom>	<41227550.6030907@v.loewis.de>
	<02e001c484a3$faed7c20$c901010a@zoom>
Message-ID: <412286C5.9010802@ocf.berkeley.edu>

Paul Gimpelj wrote:

> I have searched all the documentation online and google.
> 
> I thought that developers would interested in public input.
> 

We do care about public input, but not on possible bugs delivered this 
way.  It is easier for everyone if bugs are reported through the 
SourceForge bug  tracker at http://www.sf.net/projects/python .

> and I got email back that everybody is on vacation till sept 6.
> 

Don't know who that is from but is most likely a single person who 
forgot to turn off their mailing list subscriptions.

> Since it is open source,
> 
> I would think that this an import issue since a python advocate  would want
> python's user base to increase.
> 

I am going to assume you meant "important" instead of "import".

Yes, we would like to see usage grow, but we just don't have time to try 
to personally fix every bug through python-dev.  That is why we have the 
SF bug tracker.

> And if python , by design, is incapable of coexisting with all the other
> user programs and itself,
> what use is it?

Python can coexist, just don't expect it to come out in a hemp-based 
tie-die shirt holding a bunch of posies.  =)  I would wait and see if 
python-help can turn up some leads.  If not you can try comp.lang.python 
since I suspect this is just a configuration error.

-Brett
From mal at egenix.com  Wed Aug 18 00:30:27 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 00:30:29 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <B0627378-F098-11D8-933F-000A95686CD8@redivi.com>
References: <Your message of "Tue, 17 Aug 2004 13:49:21 BST."
	<2mbrhaaq0e.fsf@starship.python.net>	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<200408170134.i7H1YDR14560@guido.python.org>	<4121ABAB.9060605@interlink.com.au>	<200408171045.i7HAjQZ15411@guido.python.org>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>	<200408172133.i7HLXsS16658@guido.python.org>
	<B0627378-F098-11D8-933F-000A95686CD8@redivi.com>
Message-ID: <41228703.40207@egenix.com>

Bob Ippolito wrote:
> 
> On Aug 17, 2004, at 5:33 PM, Guido van Rossum wrote:
> 
>>> So, how will it be different from:
>>>
>>>      from array import array
>>>
>>>      def bytes(*initializer):
>>>          return array('B',*initializer)
>>>
>>> Even if it's desirable for 'bytes' to be an actual type (e.g. 
>>> subclassing
>>> ArrayType), it might help the definition process to describe the 
>>> difference
>>> between the new type and a byte array.
>>
>>
>> Not a whole lot different, except for the ability to use a string as
>> alternate argument to the constructor, and the fact that it's going to
>> be an actual type, and that it should support the buffer API (which
>> array mysteriously doesn't?).
>>
>> The string argument support may not even be necessary -- an
>> alternative way to spell that would be to let s.decode() return a
>> bytes object, which has the advantage of being explicit about the
>> encoding; there's even a base64 encoding already!  But it would be a
>> bigger incompatibility, more likely to break existing code using
>> decode() and expecting to get a string.
> 
> 
> IMHO current uses of decode and encode are really confusing.  Many 
> decodes are from str -> unicode, and many encodes are from unicode -> 
> str (or str -> unicode -> str implicitly, which is usually going to fail 
> miserably)... while yet others like zlib, base64, etc. are str <-> str.  
> Technically unicode.decode(base64) should certainly work, but it doesn't 
> because unicode doesn't have a decode method.

They do in 2.4. Note that in 2.4 .decode() and .encode() guarantee that
you get a basestring instance. If you want more flexibility in
terms of return type, the new codecs.encode() and codecs.decode()
will allow arbitrary types as return value.

> I don't have a proposed solution at the moment, but perhaps these 
> operations should either be outside of the data types altogether (i.e. 
> use codecs only) or there should be separate methods for doing separate 
> things (character translations versus data->data transformations).

It all depends on whether you are discussing placing binary
data into the Python source file (by some means of using
literals) or just working with bytes you got from a file,
generator, socket, etc.

The current discussion is mixing these contexts a bit too
much, I believe, which is probably why people keep misunderstanding
each other (at least that's how I perceive the debate).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 17 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From tim.peters at gmail.com  Wed Aug 18 00:39:22 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Aug 18 00:39:31 2004
Subject: [Python-Dev] multiple instances of python on XP
In-Reply-To: <029201c4849a$a43763e0$c901010a@zoom>
References: <029201c4849a$a43763e0$c901010a@zoom>
Message-ID: <1f7befae0408171539138aa997@mail.gmail.com>

[Paul Gimpelj]
> I was wondering how to have multiple instances of python running on
> XP.

Multiple instances of Python are routinely used on all flavors of
Windows without problems (at the moment, I have 4 on my XP box, and
often run 3 simultaneously).

> I have ZOPE and TSW installed, but ZOPE's python is grabbing paths
> from TSW's python.

Then you need to ask on Zope and TSW (unsure what that is) lists about
why those applications are fighting with each other.  Asking here is
like asking on a bash developer's list because you run two apps from
the shell that insist on having different PATHs.  There's nothing bash
can do about that -- and the question is better addressed to a bash
user's list even if you think there should be.

> specifically I
> 
>  changed to dir "...\Zope-2.7.2-0\bin"
> and in cmd window  ran python
> 
> did help() , sys
> and executable= r'...\Zope-2.7.2-0\bin\python.exe'
> but exec_prefix = r'...\TSW\Apache2\python'
> path_importer_cache = ....\TSW|Apache2\python...

Well, Python isn't forcing the use of TSW, and Zope certainly isn't,
so if I were you I'd ask the TSW developers what they're up to.  I
have no idea how they manage to change Zope's Python's idea of what
exec_prefix should be.  Zope sets no environment variables or registry
entries related to Python, and Zope runs happily without interfering
with a user-installed Python, so TSW is your hangup.
From janssen at parc.com  Wed Aug 18 01:13:21 2004
From: janssen at parc.com (Bill Janssen)
Date: Wed Aug 18 01:14:00 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python 
In-Reply-To: Your message of "Tue, 17 Aug 2004 13:07:46 PDT."
	<41226592.1040109@v.loewis.de> 
Message-ID: <04Aug17.161328pdt."58612"@synergy1.parc.xerox.com>

Yes, that works for me.

Martin writes:
> If you also had
> 
>     var = bytes(u"foo")
> 
> then I guess people would prefer that. People who want to save typing
> can do
> 
>     b = bytes
> 
> and, given that the u prefix will be redundant, write
> 
>    var = b("foo")

Bill
From guido at python.org  Wed Aug 18 01:21:06 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug 18 01:21:13 2004
Subject: [Python-Dev] Python 3.0 list of goals
In-Reply-To: Your message of "Tue, 17 Aug 2004 10:24:35 PDT."
	<41223F53.5060501@ocf.berkeley.edu> 
References: <20040817171622.GA10200@rogue.amk.ca>  
	<41223F53.5060501@ocf.berkeley.edu> 
Message-ID: <200408172321.i7HNL6L17005@guido.python.org>

> A.M. Kuchling wrote:
> 
> > The thread about bytes() is about a Python 3.0 feature.  Guido's
> > presentations have mentioned various changes he'd like to make in 3.0,
> > but there's no master list of things that would change.  
> > 
> > I think it would be useful to have such a list, because it would focus
> > our language development effort on ones that are steps to 3.0, and
> > maybe people can invent ways to smooth the transition.  I've started a
> > list in the Wiki at http://www.python.org/moin/Python3.0 , but should
> > it be a PEP?  (PEP 3000, perhaps?)

Good idea to start collecting all this.  I'm currently too busy to
spend a lot of time thinking about all this, but eventually I will be
back and then it would be nice to find a list of things I said in the
past. :-)

> I say a PEP.  While it can say that these features are just 
> possibilities, I would rather have a controlled place to list this stuff 
> so it is a little more official than having someone randomly come in and 
> start throwing in everything they wished Python 3.0 will have.

Right, but the Wiki is a good start while you consider a PEP.

> When it is decided where all of this info will end up I will go through 
> the python-dev Summaries and see if I can find anything there (tried to 
> make it a habit to doc stuff that Guido said would be a Python 3.0 feature.
> 
> There is also another wiki page on Python 3.0 at 
> http://www.python.org/cgi-bin/moinmoin/PythonThreeDotOh .

Hrm.  That is mostly Mike McLay's wish list; best to ignore it.

> I also have an initial list going.  At PyCON Michael McLay and I
> discussed getting a PEP written in hopes of getting grant funding
> from outside sources (read: not the PSF) to implement some of the
> features.  And we figured a PEP was probably as official as we could
> get short of a specific grant proposal and thus be more acceptable
> to people considering providing funds.

A PEP along the lines of Mike's Wiki page doesn't really have my
support.  It's a long story that I'll have to save for another time,
but basically it seems to me that Mike's agenda is driven by the need
of a very specific subcommunity.  That's fine, but let's be clear
about what it is -- it's not my idea for Python 3000.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From skip at pobox.com  Wed Aug 18 04:03:03 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Aug 18 04:03:26 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <200408172133.i7HLXsS16658@guido.python.org>
References: <Your message of "Tue,
	17 Aug 2004 13:49:21 BST." <2mbrhaaq0e.fsf@starship.python.net>
	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>
	<200408172133.i7HLXsS16658@guido.python.org>
Message-ID: <16674.47319.319502.989563@montanaro.dyndns.org>


    Guido> The string argument support may not even be necessary -- an
    Guido> alternative way to spell that would be to let s.decode() return a
    Guido> bytes object, which has the advantage of being explicit about the
    Guido> encoding; there's even a base64 encoding already!  

I'm sorry folks, but I still don't understand all this discussion overlap
between unicode/string objects (which require explicit or implicit decoding)
and bytes objects (which clearly must not).  Everyone keeps talking about
decoding stuff into bytes objects and whether or not bytes literals would be
compatible with the current source encoding.  My understanding is that bytes
objects are just that, raw sequences of bytes in the range 0x00 to 0xff,
inclusive, with no interpretation of any type.

Skip
From paul at prescod.net  Wed Aug 18 04:15:54 2004
From: paul at prescod.net (Paul Prescod)
Date: Wed Aug 18 04:16:05 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41227F83.4060403@egenix.com>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>
	<41227F83.4060403@egenix.com>
Message-ID: <4122BBDA.4050401@prescod.net>

M.-A. Lemburg wrote:
>...
> 
> 
> We already have a number of read-write types for storing binary
> data, e.g. arrays, cStringIO and buffers. Inventing yet another
> way to spell binary data won't make life easier.

The point is canonicalize one and start to make APIs that expect it. 
Otherwise we will never make the leap from the bad habit of using 
strings as byte arrays. Can you pass arrays to the write() function? Can 
you decode buffers to strings? A byte array type would have a certain 
mix of functions and API compatibility that is missing from the plethora 
of similar thingees.

  Paul Prescod

From greg at cosc.canterbury.ac.nz  Wed Aug 18 05:01:34 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug 18 05:01:39 2004
Subject: [Python-Dev] Python 3.0 list of goals
In-Reply-To: <20040817171622.GA10200@rogue.amk.ca>
Message-ID: <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz>

"A.M. Kuchling" <amk@amk.ca>:

> I think it would be useful to have such a list, because it would focus
> our language development effort on ones that are steps to 3.0, and
> maybe people can invent ways to smooth the transition.  I've started a
> list in the Wiki at http://www.python.org/moin/Python3.0 , but should
> it be a PEP?  (PEP 3000, perhaps?)

As a point of interest, are Python 3.0 and Python 3000 actually the
same thing? Python 3000 is clearly some hypothetical version at some
unspecified time in the future, which may or may not ever exist. But
at the rate version numbers are going, we're going to actually get to
3.0 before very long. So, when we talk about 3.0, are we talking about
a wish version or a real version?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From guido at python.org  Wed Aug 18 05:16:46 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug 18 05:16:55 2004
Subject: [Python-Dev] Python 3.0 list of goals
In-Reply-To: Your message of "Wed, 18 Aug 2004 15:01:34 +1200."
	<200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz> 
References: <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408180316.i7I3Gk217306@guido.python.org>

> As a point of interest, are Python 3.0 and Python 3000 actually the
> same thing?

In my mind, yes.

> Python 3000 is clearly some hypothetical version at some
> unspecified time in the future, which may or may not ever exist. But
> at the rate version numbers are going, we're going to actually get to
> 3.0 before very long. So, when we talk about 3.0, are we talking about
> a wish version or a real version?

Actually, with one minor version number bump per 12-18 months, we
should have at least 5 years before we're at 2.9.  So I think we can
continue to equate 3.0 and 3000.  It would be nice PR to actually call
it "Python 3000" rather than "Python 3.0" when it comes out -- showing
we can play the marketing game as well as anybody without losing our
sense of humor.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Wed Aug 18 05:52:46 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Aug 18 05:52:49 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <200408170114.i7H1Elk14352@guido.python.org>
References: <002001c48325$436d9460$e841fea9@oemcomputer>
	<87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp> 
	<900DDB3A-EF36-11D8-8D77-000A95686CD8@redivi.com>
	<200408170114.i7H1Elk14352@guido.python.org>
Message-ID: <1f7befae040817205233d70086@mail.gmail.com>

[Guido]
> (In addition, I find that the GIL often causes people to worry
> unnecessarily about the GIL.  Sort of like worrying about relativistic
> effects when driving a car.)

LOL!  OTOH, well-endowed gentlemen have always driven as fast as
possible, in order to avoid frightening their admiring lady onlookers.
 To be effective, I have to scrunch over rather uncomfortably, though,
since Lorentz contraction only occurs along the direction of observed
motion.

done-right-it-only-takes-0.97c-to-give-the-appearance-of-6-inches-ly
    y'rs  - tim
From barry at python.org  Wed Aug 18 06:23:00 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 18 06:22:35 2004
Subject: [Python-Dev] Re: Re: Update PEP 292
In-Reply-To: <cfdbv6$pv5$1@sea.gmane.org>
References: <1092177771.21839.54.camel@localhost>
	<cfd23o$lla$1@sea.gmane.org> <1092232536.9945.21.camel@localhost>
	<cfdbv6$pv5$1@sea.gmane.org>
Message-ID: <1092802980.7810.29.camel@geddy.wooz.org>

On Wed, 2004-08-11 at 10:54, Fernando Perez wrote:

> I guess my post wasn't worded too clearly.  I did mean existing, in the sense
> that the error above occurs with current python (it's cut-and-paste from python
> 2.2).  My point was precisely to illustrate this failure with a simple example,
> to then ask whether the new scheme could be made, _as shipped_, to accept this
> kind of expansion:
> 
> print template('sys.platform is: $sys.platform') % locals()

Not as shipped.  You'd have you derive from the Template class and write
a regular expression (or adapt the existing pattern).

> It may be trivial to extend pep-292 for this, but not having to do it in every
> small utility you write/ship would be very useful, IMHO.  If there are no
> technical reasons to prevent it, is the 'too complex' argument really that
> strong (especially since you state it's a trivial amount of code)?

Well, it's about 3 lines of Python, but what would you call the class,
AttributeTemplate?  I wouldn't be opposed to adding the class, since I
think it would be pretty useful.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040818/62a441f5/attachment.pgp
From barry at python.org  Wed Aug 18 06:24:44 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 18 06:24:19 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <20040811151347.GA18312@rogue.amk.ca>
References: <1092177771.21839.54.camel@localhost>
	<20040811151347.GA18312@rogue.amk.ca>
Message-ID: <1092803084.7834.32.camel@geddy.wooz.org>

On Wed, 2004-08-11 at 11:13, A.M. Kuchling wrote:
> On Tue, Aug 10, 2004 at 06:42:51PM -0400, Barry Warsaw wrote:
> > The updated PEP is at:
> > http://www.python.org/peps/pep-0292.html
> 
> Are you sure that people aren't going to want to format numbers
> as they do with % (e.g. %4.2f)?  

I'm sure they will, but I think they should just use %-interpolation
then. ;)  I'm personally not interested in supporting that for (default)
$-strings.

> You should post the PEP to c.l.py.announce, too.  (Perhaps you have
> and it's still sitting in the queue.)

I hadn't, but good point.  I'll do that.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040818/5125de4b/attachment.pgp
From rnd at onego.ru  Wed Aug 18 06:30:58 2004
From: rnd at onego.ru (Roman Suzi)
Date: Wed Aug 18 06:30:01 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41227F83.4060403@egenix.com>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<4121ABAB.9060605@interlink.com.au> <4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com>
	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>
	<41227F83.4060403@egenix.com>
Message-ID: <Pine.LNX.4.58.0408180821240.1555@rnd.onego.ru>

On Tue, 17 Aug 2004, M.-A. Lemburg wrote:

>Roman Suzi wrote:
>> On Tue, 17 Aug 2004, M.-A. Lemburg wrote:

>> It was in the shadows because we had byte-strings.
>
>Right, so why not revive it ?!
>
>Anyway, this whole discussion about a new bytes type doesn't
>really solve the problem that the b'...' literal was
>intended for: that of having a nice way to define (read-only)
>8-bit binary string literals.

I think new _mutable_ bytes() type is better than old 8-bit binary strings
for binary data processing purposes.
Or do we need them for legacy text-procesing software?

>We already have a number of read-write types for storing binary
>data, e.g. arrays, cStringIO and buffers. Inventing yet another
>way to spell binary data won't make life easier.
>
>However, what will be missing is a nice way to spell read-only
>binary data.
>
>Since 'tada' will return a Unicode object in Py3k, I think we
>should reuse the existing 8-bit string object under the new
>literal constructor b'tada\x00' (and apply the same source code
>encoding semantics we apply today for 'tada\x00').
>
>

Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From martin at v.loewis.de  Wed Aug 18 06:57:02 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 06:57:04 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412285D1.7010907@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>
	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>
	<411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com>
Message-ID: <4122E19E.9050906@v.loewis.de>

M.-A. Lemburg wrote:
> I've thought about this some more. Perhaps I'm still missing
> something, but wouldn't it be possible to add a feeding
> mode to the existing stream codecs by creating a new queue
> data type (much like the queue you have in the test cases of
> your patch) and using the stream codecs on these ?

Here is the problem. In UTF-8, how does the actual algorithm
tell (the application) that the bytes it got on decoding provide
for three fully decodable characters, and that 2 bytes are left
undecoded, and that those bytes are not inherently ill-formed,
but lack a third byte to complete the multi-byte sequence?

On top of that, you can implement whatever queuing or streaming
APIs you want, but you *need* an efficient way to communicate
incompleteness.

Regards,
Martin
From greg at cosc.canterbury.ac.nz  Wed Aug 18 06:58:28 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug 18 06:58:33 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <1f7befae040817205233d70086@mail.gmail.com>
Message-ID: <200408180458.i7I4wS2a023568@cosc353.cosc.canterbury.ac.nz>

> done-right-it-only-takes-0.97c-to-give-the-appearance-of-6-inches-ly
>    y'rs  - tim

Hmmm, so Tim is claiming to have an actual length
of

>>> 6 / sqrt(1 - (0.97 * 0.97))
24.680702093691806

inches...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From martin at v.loewis.de  Wed Aug 18 07:01:50 2004
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed Aug 18 07:01:47 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41227F83.4060403@egenix.com>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>
	<41227F83.4060403@egenix.com>
Message-ID: <4122E2BE.1070101@v.loewis.de>

M.-A. Lemburg wrote:
> Anyway, this whole discussion about a new bytes type doesn't
> really solve the problem that the b'...' literal was
> intended for: that of having a nice way to define (read-only)
> 8-bit binary string literals.

But why do you need a way to spell 8-bit string literals?
You can always do

    "string".encode("L1")

If that is too much typing, do

    def b(s):return s.encode("L1")

    b("string")

Regards,
Martin
From martin at v.loewis.de  Wed Aug 18 07:02:45 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 07:02:42 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <B0627378-F098-11D8-933F-000A95686CD8@redivi.com>
References: <Your message of "Tue, 17 Aug 2004 13:49:21 BST."
	<2mbrhaaq0e.fsf@starship.python.net>	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<200408170134.i7H1YDR14560@guido.python.org>	<4121ABAB.9060605@interlink.com.au>	<200408171045.i7HAjQZ15411@guido.python.org>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>	<200408172133.i7HLXsS16658@guido.python.org>
	<B0627378-F098-11D8-933F-000A95686CD8@redivi.com>
Message-ID: <4122E2F5.5080508@v.loewis.de>

Bob Ippolito wrote:
>
> IMHO current uses of decode and encode are really confusing.

I completely agree. This overgeneralization is insane.

Regards,
Martin
From martin at v.loewis.de  Wed Aug 18 07:09:18 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 07:09:17 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <16674.47319.319502.989563@montanaro.dyndns.org>
References: <Your message of "Tue, 17 Aug 2004 13:49:21 BST."
	<2mbrhaaq0e.fsf@starship.python.net>	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<200408170134.i7H1YDR14560@guido.python.org>	<4121ABAB.9060605@interlink.com.au>	<200408171045.i7HAjQZ15411@guido.python.org>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>	<200408172133.i7HLXsS16658@guido.python.org>
	<16674.47319.319502.989563@montanaro.dyndns.org>
Message-ID: <4122E47E.1040806@v.loewis.de>

Skip Montanaro wrote:
> My understanding is that bytes
> objects are just that, raw sequences of bytes in the range 0x00 to 0xff,
> inclusive, with no interpretation of any type.

Yes, but your understanding is limited :-) This idea is good, but it
falls short once we talk about source code, because source code does
have an encoding. So if you don't want to incorporate the notion of
encodings into the byte string types, yet be able to declare them
in source code, you have to go for a numeric representation. I.e.
you write
    bytes(71,69, 84)
instead of
    b"GET"

As soon as you use some kind of string notation for denoting byte
code values, you immediately *have* to deal with encodings.

Regards,
Martin
From martin at v.loewis.de  Wed Aug 18 07:13:25 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 07:13:21 2004
Subject: [Python-Dev] Python 3.0 list of goals
In-Reply-To: <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz>
References: <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz>
Message-ID: <4122E575.9020108@v.loewis.de>

Greg Ewing wrote:
> As a point of interest, are Python 3.0 and Python 3000 actually the
> same thing? Python 3000 is clearly some hypothetical version at some
> unspecified time in the future, which may or may not ever exist. But
> at the rate version numbers are going, we're going to actually get to
> 3.0 before very long.

I can't see that :-) After 2.9, we get 2.10, then 2.11. After 2.99,
we get 2.100, then 2.101. At that rate, we won't reach 3.0 before
that heath death (sp?) of the universe.

Regards,
Martin
From allison at sumeru.stanford.EDU  Wed Aug 18 07:33:54 2004
From: allison at sumeru.stanford.EDU (Dennis Allison)
Date: Wed Aug 18 07:34:11 2004
Subject: [Python-Dev] Python 3.0 list of goals
In-Reply-To: <4122E575.9020108@v.loewis.de>
Message-ID: <Pine.LNX.4.10.10408172232250.10675-100000@sumeru.stanford.EDU>

Ahhhh...  Zeno's paradox again.  We'll get there eventually, for all
practical purposes.

On Wed, 18 Aug 2004, [ISO-8859-1] "Martin v. L=F6wis" wrote:

> Greg Ewing wrote:
> > As a point of interest, are Python 3.0 and Python 3000 actually the
> > same thing? Python 3000 is clearly some hypothetical version at some
> > unspecified time in the future, which may or may not ever exist. But
> > at the rate version numbers are going, we're going to actually get to
> > 3.0 before very long.
>=20
> I can't see that :-) After 2.9, we get 2.10, then 2.11. After 2.99,
> we get 2.100, then 2.101. At that rate, we won't reach 3.0 before
> that heath death (sp?) of the universe.
>=20
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/allison%40=
sumeru.stanford.edu
>=20

From mal at egenix.com  Wed Aug 18 10:36:06 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 10:36:09 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4122E19E.9050906@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de>
Message-ID: <412314F6.5060502@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
> 
>> I've thought about this some more. Perhaps I'm still missing
>> something, but wouldn't it be possible to add a feeding
>> mode to the existing stream codecs by creating a new queue
>> data type (much like the queue you have in the test cases of
>> your patch) and using the stream codecs on these ?
> 
> Here is the problem. In UTF-8, how does the actual algorithm
> tell (the application) that the bytes it got on decoding provide
> for three fully decodable characters, and that 2 bytes are left
> undecoded, and that those bytes are not inherently ill-formed,
> but lack a third byte to complete the multi-byte sequence?

This state can be stored in the stream codec instance,
e.g. by using a special state object that is stored in
the instance and passed to the encode/decode APIs of the
codec or by implementing the stream codec itself in C.

We do need to extend the API between the stream codec
and the encode/decode functions, no doubt about that.
However, this is an extension that is well hidden from
the user of the codec and won't break code.

> On top of that, you can implement whatever queuing or streaming
> APIs you want, but you *need* an efficient way to communicate
> incompleteness.

Agreed.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 18 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Wed Aug 18 10:41:36 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 10:41:39 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4122E2BE.1070101@v.loewis.de>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>
	<41227F83.4060403@egenix.com> <4122E2BE.1070101@v.loewis.de>
Message-ID: <41231640.2020706@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
> 
>> Anyway, this whole discussion about a new bytes type doesn't
>> really solve the problem that the b'...' literal was
>> intended for: that of having a nice way to define (read-only)
>> 8-bit binary string literals.
> 
> But why do you need a way to spell 8-bit string literals?
 >
> You can always do
> 
>    "string".encode("L1")
> 
> If that is too much typing, do
> 
>    def b(s):return s.encode("L1")
> 
>    b("string")

You need to think about the important use-case of having
to convert Py2 applications to Py3 style. In many cases,
the application can be made to run under Py3 be adding the
small 'b' in front of the used string literals.

Even better: if we add the b'xxx' notation now, we could start
working towards the switch-over by slowly converting the
Python standard library to actually work in -U mode (which
basically implements the switch-over by making 'abc' behave
as u'abc').

Since the code is already in place and the change is
minimal, I don't see any reason not to use it.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 18 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Wed Aug 18 10:46:55 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 10:46:57 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4122E47E.1040806@v.loewis.de>
References: <Your message of "Tue, 17 Aug 2004 13:49:21
	BST."	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<200408170134.i7H1YDR14560@guido.python.org>	<4121ABAB.9060605@interlink.com.au>	<200408171045.i7HAjQZ15411@guido.python.org>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>	<200408172133.i7HLXsS16658@guido.python.org>	<16674.47319.319502.989563@montanaro.dyndns.org>
	<4122E47E.1040806@v.loewis.de>
Message-ID: <4123177F.7050306@egenix.com>

Martin v. L?wis wrote:
> Skip Montanaro wrote:
> 
>> My understanding is that bytes
>> objects are just that, raw sequences of bytes in the range 0x00 to 0xff,
>> inclusive, with no interpretation of any type.
> 
> 
> Yes, but your understanding is limited :-) This idea is good, but it
> falls short once we talk about source code, because source code does
> have an encoding. So if you don't want to incorporate the notion of
> encodings into the byte string types, yet be able to declare them
> in source code, you have to go for a numeric representation. I.e.
> you write
>    bytes(71,69, 84)
> instead of
>    b"GET"
> 
> As soon as you use some kind of string notation for denoting byte
> code values, you immediately *have* to deal with encodings.

Of course you do, but aren't you making things too complicated,
Martin ?

If you write your string literal using just ASCII characters and
escapes, I don't see much of a problem with different source code
encodings. If it makes you feel better, we could even enforce
this by only allowing these characters in binary string literals.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 18 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Wed Aug 18 10:49:52 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 10:49:55 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <Pine.LNX.4.58.0408180821240.1555@rnd.onego.ru>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>
	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>	<41227F83.4060403@egenix.com>
	<Pine.LNX.4.58.0408180821240.1555@rnd.onego.ru>
Message-ID: <41231830.2090903@egenix.com>

Roman Suzi wrote:
> On Tue, 17 Aug 2004, M.-A. Lemburg wrote:
> 
> 
>>Roman Suzi wrote:
>>
>>>On Tue, 17 Aug 2004, M.-A. Lemburg wrote:
> 
> 
>>>It was in the shadows because we had byte-strings.
>>
>>Right, so why not revive it ?!
>>
>>Anyway, this whole discussion about a new bytes type doesn't
>>really solve the problem that the b'...' literal was
>>intended for: that of having a nice way to define (read-only)
>>8-bit binary string literals.
> 
> I think new _mutable_ bytes() type is better than old 8-bit binary strings
> for binary data processing purposes.
> Or do we need them for legacy text-procesing software?

Hmm, who ever said that we are going to drop the current 8-bit
string implementation ?

I'm only suggesting to look at what's there instead of
trying to redo everything in slightly different way, e.g.
you can already get the bytes() functionality from
buffer type at C level - it's just that this functionality
is not exposed at Python level.

>>We already have a number of read-write types for storing binary
>>data, e.g. arrays, cStringIO and buffers. Inventing yet another
>>way to spell binary data won't make life easier.
>>
>>However, what will be missing is a nice way to spell read-only
>>binary data.
>>
>>Since 'tada' will return a Unicode object in Py3k, I think we
>>should reuse the existing 8-bit string object under the new
>>literal constructor b'tada\x00' (and apply the same source code
>>encoding semantics we apply today for 'tada\x00').

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 18 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Wed Aug 18 11:02:40 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 11:02:41 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4122BBDA.4050401@prescod.net>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>
	<41227F83.4060403@egenix.com> <4122BBDA.4050401@prescod.net>
Message-ID: <41231B30.2060307@egenix.com>

Paul Prescod wrote:
> M.-A. Lemburg wrote:
> 
>> ...
>>
>>
>> We already have a number of read-write types for storing binary
>> data, e.g. arrays, cStringIO and buffers. Inventing yet another
>> way to spell binary data won't make life easier.
> 
> The point is canonicalize one and start to make APIs that expect it. 
> Otherwise we will never make the leap from the bad habit of using 
> strings as byte arrays.
 >
> Can you pass arrays to the write() function? Can 
> you decode buffers to strings? A byte array type would have a certain 
> mix of functions and API compatibility that is missing from the plethora 
> of similar thingees.

Wouldn't it be possible to extend the existing buffer type
to meet those standards ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 18 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From anthony at interlink.com.au  Wed Aug 18 08:50:49 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Aug 18 13:12:24 2004
Subject: [Python-Dev] Python 3.0 list of goals
In-Reply-To: <200408180316.i7I3Gk217306@guido.python.org>
References: <200408180301.i7I31YHW023388@cosc353.cosc.canterbury.ac.nz>
	<200408180316.i7I3Gk217306@guido.python.org>
Message-ID: <4122FC49.5040008@interlink.com.au>

Guido van Rossum wrote:
> Actually, with one minor version number bump per 12-18 months, we
> should have at least 5 years before we're at 2.9.  

And if we still need time, there's 2.X - just like 2.9, but with
multicoloured buttons!

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From gjc at inescporto.pt  Wed Aug 18 14:11:07 2004
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Wed Aug 18 14:11:18 2004
Subject: [Python-Dev] Find out whether PyEval_InitThreads has been called?
In-Reply-To: <1092763177.41223e29382cb@mcherm.com>
References: <1092763177.41223e29382cb@mcherm.com>
Message-ID: <1092831067.1944.1.camel@emperor>

A Ter, 2004-08-17 ?s 18:19, Michael Chermside escreveu:
> > Why not go one step further and make all locking
> > operations in python no-ops if PyEval_InitThreads has never been
> > called?  Currently, we risk python crashing if we call such functions
> > without calling PyEval_InitThreads.
> 
> +1. Seems like it can't possibly hurt, and as you say, it's trivial
> to implement.

  Thanks for the feedback.

  http://www.python.org/sf?id=1011380

  Regards.

-- 
Gustavo J. A. M. Carneiro
<gjc@inescporto.pt> <gustavo@users.sourceforge.net>
The universe is always one step beyond logic

From s.percivall at chello.se  Wed Aug 18 15:25:25 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Wed Aug 18 15:25:31 2004
Subject: [Python-Dev] tab in file tkFont.py
Message-ID: <10AB5C0F-F11A-11D8-873D-0003934AD54A@chello.se>

lib-tk/tkFont.py CVS HEAD has a tab character at line 94, breaking 
installation.

//Simon

From bac at OCF.Berkeley.EDU  Wed Aug 18 19:07:09 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Aug 18 19:07:19 2004
Subject: [Python-Dev] Python 3.0 list of goals
In-Reply-To: <200408172321.i7HNL6L17005@guido.python.org>
References: <20040817171622.GA10200@rogue.amk.ca>
	<41223F53.5060501@ocf.berkeley.edu>
	<200408172321.i7HNL6L17005@guido.python.org>
Message-ID: <41238CBD.1050908@ocf.berkeley.edu>

Guido van Rossum wrote:
>>A.M. Kuchling wrote:
>>
>>
>>>The thread about bytes() is about a Python 3.0 feature.  Guido's
>>>presentations have mentioned various changes he'd like to make in 3.0,
>>>but there's no master list of things that would change.  
>>>
>>>I think it would be useful to have such a list, because it would focus
>>>our language development effort on ones that are steps to 3.0, and
>>>maybe people can invent ways to smooth the transition.  I've started a
>>>list in the Wiki at http://www.python.org/moin/Python3.0 , but should
>>>it be a PEP?  (PEP 3000, perhaps?)
> 
> 
> Good idea to start collecting all this.  I'm currently too busy to
> spend a lot of time thinking about all this, but eventually I will be
> back and then it would be nice to find a list of things I said in the
> past. :-)
> 

=)  Prepping your time machine for ya.

> 
>>I say a PEP.  While it can say that these features are just 
>>possibilities, I would rather have a controlled place to list this stuff 
>>so it is a little more official than having someone randomly come in and 
>>start throwing in everything they wished Python 3.0 will have.
> 
> 
> Right, but the Wiki is a good start while you consider a PEP.
> 

OK.  I will start plugging in what I have on my personal collection of 
what you have said into the wiki after the next summary goes out (400 
emails to go).

> 
>>When it is decided where all of this info will end up I will go through 
>>the python-dev Summaries and see if I can find anything there (tried to 
>>make it a habit to doc stuff that Guido said would be a Python 3.0 feature.
>>
>>There is also another wiki page on Python 3.0 at 
>>http://www.python.org/cgi-bin/moinmoin/PythonThreeDotOh .
> 
> 
> Hrm.  That is mostly Mike McLay's wish list; best to ignore it.
> 

OK.  I will just go off what I have mentioned in the Summaries and what 
I can recollect from my own memory.

-Brett
From martin at v.loewis.de  Wed Aug 18 19:28:04 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 19:28:00 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41231640.2020706@egenix.com>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>
	<41227F83.4060403@egenix.com> <4122E2BE.1070101@v.loewis.de>
	<41231640.2020706@egenix.com>
Message-ID: <412391A4.4080700@v.loewis.de>

M.-A. Lemburg wrote:
> You need to think about the important use-case of having
> to convert Py2 applications to Py3 style. In many cases,
> the application can be made to run under Py3 be adding the
> small 'b' in front of the used string literals.

That is hard to tell, because Py3 is not implemented, yet.
It might be that in many cases, no change is necessary at
all, because the system default encoding will convert the
strings to bytes.

> Since the code is already in place and the change is
> minimal, I don't see any reason not to use it.

I do. It would mean that we commit to the b"" notation,
when there is no real need for that.

Regards,
Martin
From martin at v.loewis.de  Wed Aug 18 19:31:28 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 19:31:26 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4123177F.7050306@egenix.com>
References: <Your message of "Tue, 17 Aug 2004 13:49:21
	BST."	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<200408170134.i7H1YDR14560@guido.python.org>	<4121ABAB.9060605@interlink.com.au>	<200408171045.i7HAjQZ15411@guido.python.org>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>	<200408172133.i7HLXsS16658@guido.python.org>	<16674.47319.319502.989563@montanaro.dyndns.org>
	<4122E47E.1040806@v.loewis.de> <4123177F.7050306@egenix.com>
Message-ID: <41239270.5090100@v.loewis.de>

M.-A. Lemburg wrote:
> If you write your string literal using just ASCII characters and
> escapes, I don't see much of a problem with different source code
> encodings. 

That is correct. I personally have no problem if byte fields and
unicode strings are connected through some encoding; I personally
think making it fixed at Latin-1 might be best.

I was merely responding to Skip's question why an encoding comes
into play at all, as byte fields inherently have no encoding, and
might not even represent character data. I was responding that
this is mostly true, except for source code.

Regards,
Martin

From martin at v.loewis.de  Wed Aug 18 19:44:18 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 19:44:17 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41231B30.2060307@egenix.com>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>	<41227F83.4060403@egenix.com>
	<4122BBDA.4050401@prescod.net> <41231B30.2060307@egenix.com>
Message-ID: <41239572.8090808@v.loewis.de>

M.-A. Lemburg wrote:
> Wouldn't it be possible to extend the existing buffer type
> to meet those standards ?

Yes; you then need to change all codecs to return buffers
from .encode.

Regards,
Martin

From mal at egenix.com  Wed Aug 18 19:55:52 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 19:55:55 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41239572.8090808@v.loewis.de>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>	<41227F83.4060403@egenix.com>	<4122BBDA.4050401@prescod.net>
	<41231B30.2060307@egenix.com> <41239572.8090808@v.loewis.de>
Message-ID: <41239828.9000301@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
> 
>> Wouldn't it be possible to extend the existing buffer type
>> to meet those standards ?
> 
> Yes; you then need to change all codecs to return buffers
> from .encode.

I'm still not convinced that we can simply drop the existing
immutable 8-bit string type and replace it with a mutable
bytes or buffer type, e.g. would buffer.lower() work on the
buffer itself or return a lowered copy ?

However, if that's where Python will be heading, then you're
right (for most of the codecs: some might want to return
Unicode objects, e.g. unicode-escape).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 18 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From martin at v.loewis.de  Wed Aug 18 20:06:37 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 20:06:33 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41239828.9000301@egenix.com>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>	<41227F83.4060403@egenix.com>	<4122BBDA.4050401@prescod.net>
	<41231B30.2060307@egenix.com> <41239572.8090808@v.loewis.de>
	<41239828.9000301@egenix.com>
Message-ID: <41239AAD.2080604@v.loewis.de>

M.-A. Lemburg wrote:
> I'm still not convinced that we can simply drop the existing
> immutable 8-bit string type and replace it with a mutable
> bytes or buffer type, e.g. would buffer.lower() work on the
> buffer itself or return a lowered copy ?

The byte string type would not have a .lower method, as
"lowering" is not meaningful for bytes, only for characters.

Regards,
Martin

From fperez528 at yahoo.com  Wed Aug 18 20:07:59 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Wed Aug 18 20:08:14 2004
Subject: [Python-Dev] Re: Re: Re: Update PEP 292
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<1092232536.9945.21.camel@localhost> <cfdbv6$pv5$1@sea.gmane.org>
	<1092802980.7810.29.camel@geddy.wooz.org>
Message-ID: <cg05u5$qfh$1@sea.gmane.org>

Barry Warsaw wrote:

> On Wed, 2004-08-11 at 10:54, Fernando Perez wrote:

>> It may be trivial to extend pep-292 for this, but not having to do it in
>> every
>> small utility you write/ship would be very useful, IMHO.  If there are no
>> technical reasons to prevent it, is the 'too complex' argument really that
>> strong (especially since you state it's a trivial amount of code)?
> 
> Well, it's about 3 lines of Python, but what would you call the class,
> AttributeTemplate?  I wouldn't be opposed to adding the class, since I
> think it would be pretty useful.

I'd like to argue that this form may be the most useful for common tasks, so
you can mix and match "this is foo: $foo and this is foo.bar: $foo.bar"
without having to worry too much about which template class you are using.  

How about making a BaseTemplate class which does NOT allow $foo.bar, and having
the default Template implement attributes?  This would give us:

-  A default class covering what I think is a reasonable, common-case behavior. 
I'm a big fan of covering most _reasonable_ default behavior out-of-the-box,
and I think python's stdlib achieves this very well in most cases.  I'd argue
this is one of the reasons for its success, and I have the feeling that in
this case (PEP-292) the proposed default would be sub-par.

- A clean base class for user-defined subclasses which want to be very exacting
about what they want to implement or not.

This sounds like a reasonable compromise to me, but ymmv.

Best, and thanks for keeping an open mind about this issue (string
interpolation is one of the very few areas where python's syntax bugs me, even
after years of daily use).

f

From mal at egenix.com  Wed Aug 18 20:31:17 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 20:31:21 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41239AAD.2080604@v.loewis.de>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>	<41227F83.4060403@egenix.com>	<4122BBDA.4050401@prescod.net>
	<41231B30.2060307@egenix.com>	<41239572.8090808@v.loewis.de>
	<41239828.9000301@egenix.com> <41239AAD.2080604@v.loewis.de>
Message-ID: <4123A075.9010407@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
> 
>> I'm still not convinced that we can simply drop the existing
>> immutable 8-bit string type and replace it with a mutable
>> bytes or buffer type, e.g. would buffer.lower() work on the
>> buffer itself or return a lowered copy ?
> 
> The byte string type would not have a .lower method, as
> "lowering" is not meaningful for bytes, only for characters.

Indeed... and the same is true for almost all other methods
(except maybe .replace()).

Sounds like a lot of code will break.

OTOH, it will also enforce the notion of doing encoding and decoding
only at IO boundaries and being explicit about character sets which
is good in the long run.

Auto-conversion using the default encoding will get us
all ASCII character (Unicode) strings converted to buffers
without problems (and without having the need for an extra
b'something' modifier).

This leaves the question of how to deal with the byte range
0x80 - 0xFF. The straight forward solution would be to switch
to Latin-1 as default encoding and let the same magic take
care of that byte range as well.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 18 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From martin at v.loewis.de  Wed Aug 18 21:06:09 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 21:06:06 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4123A075.9010407@egenix.com>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>	<41227F83.4060403@egenix.com>	<4122BBDA.4050401@prescod.net>
	<41231B30.2060307@egenix.com>	<41239572.8090808@v.loewis.de>
	<41239828.9000301@egenix.com> <41239AAD.2080604@v.loewis.de>
	<4123A075.9010407@egenix.com>
Message-ID: <4123A8A1.9060908@v.loewis.de>

M.-A. Lemburg wrote:
> Indeed... and the same is true for almost all other methods
> (except maybe .replace()).
> 
> Sounds like a lot of code will break.

We will see. The default string type will be Unicode, so code
using .lower will continue to work in many cases.

The question is what open(path,"r").read() will return. It seems
that Guido wants users to specify "rb" if they want that to be
byte strings.

Regards,
Martin
From bac at OCF.Berkeley.EDU  Wed Aug 18 21:38:15 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Aug 18 21:38:28 2004
Subject: [Python-Dev] tab in file tkFont.py
In-Reply-To: <10AB5C0F-F11A-11D8-873D-0003934AD54A@chello.se>
References: <10AB5C0F-F11A-11D8-873D-0003934AD54A@chello.se>
Message-ID: <4123B027.7020405@ocf.berkeley.edu>

Simon Percivall wrote:

> lib-tk/tkFont.py CVS HEAD has a tab character at line 94, breaking 
> installation.
> 

Looks like it has been fixed now.

-Brett
From Kevin.Smith at theMorgue.org  Wed Aug 18 21:42:55 2004
From: Kevin.Smith at theMorgue.org (Kevin Smith)
Date: Wed Aug 18 21:42:27 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
Message-ID: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>

For what it's worth, I wrote the original PEP 318.  I probably wasn't 
qualified, but I just wanted a nice simple way to declare class methods 
without having to repeat the function name.  After submitting it to 
BDFL for approval, more work was needed and the discussion of PEP 318 
on python-dev increased rapidly.  It was evident that I was in over my 
head, so I asked more someone more experienced to take over.

I guess others had bigger plans for my proposal that I had planned.  It 
has turned into the "solution" to many problems: type checking (both 
arguments and returned values), metaclasses, metadata, interfaces, 
function attributes, etc.).  Unfortunately, in the process, this simple 
request for syntactical sugar has turned into a monstrosity.  In my 
opinion, none of the proposed syntaxes really seem Pythonic.  This PEP 
just seems to be trying to solve too many problems.

Bear with me, but I'd like to propose one more syntax that is simple, 
easy for newbies to understand,  and nowhere near as powerful as the 
current PEP's syntax.  However, it doesn't add incoherent, arbitrary 
syntax either.

def classmethod foo(x, y, z):
     pass

That's it.  One "decorator" that is a callable object that takes a 
method as it's only argument.  No expressions, lists, tuples, etc.  
Just one callable object.  Ok, if you absolutely must have more than 
one.

def classmethod synchronized foo(x, y, z):
     pass

Once again, no expressions.  I know that this isn't going to solve 
everyone's type-checking, metadata, and function attribute problems, 
but let's face it, using this PEP for all of those things just creates 
ugly syntax.  There must be more Pythonic ways to do those things in 
their own PEPs.

Kevin Smith
Kevin.Smith@themorgue.org

From walter at livinglogic.de  Wed Aug 18 21:46:10 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Aug 18 21:46:26 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412285D1.7010907@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>
	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>
	<411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com>
Message-ID: <4123B202.5000200@livinglogic.de>

M.-A. Lemburg wrote:

> Walter D?rwald wrote:
> 
>> M.-A. Lemburg wrote:
>>
>>> Overall, I don't like the idea of adding extra
>>> APIs breaking the existing codec API.
>>
>> Adding a final argument that defaults to False doesn't
>> break the API for the callers, only for the implementors.
>>
>> And if we drop the final argument, the API is completely
>> backwards compatible both for users and implementors.
>> The only thing that gets added is the feed() method,
>> that implementors don't have to overwrite.
>>
>>> I believe that we can
>>> extend stream codecs to also work in a feed mode without
>>> breaking the API.
>>
>> Abandoning the final argument and adding a feed() method
>> would IMHO be the simplest way to do this.
>>
>> But then there's no way to make sure that every byte from
>> the input stream is really consumed.
> 
> I've thought about this some more. Perhaps I'm still missing
> something, but wouldn't it be possible to add a feeding
> mode to the existing stream codecs by creating a new queue
> data type (much like the queue you have in the test cases of
> your patch) and using the stream codecs on these ?

No, because when the decode method encounters an incomplete
chunk (and so return a size that is smaller then size of the
input) read() would have to push the remaining bytes back into
the queue. This would be code similar in functionality
to the feed() method from the patch, with the difference that
the buffer lives in the queue not the StreamReader. So
we won't gain any code simplification by going this route.

> I think such a queue would be generally useful in other
> contexts as well, e.g. for implementing fast character based
> pipes between threads, non-Unicode feeding parsers, etc.
> Using such a type you could potentially add a feeding
> mode to stream or file-object API based algorithms very
> easily.

Yes, so we could put this Queue class into a module with
string utilities. Maybe string.py?

> We could then have a new class, e.g. FeedReader, which
> wraps the above in a nice API, much like StreamReaderWriter
> and StreamRecoder.

But why should we, when decode() does most of what we need,
and the rest has to be implemented in both versions?

Bye,
    Walter D?rwald


From bob at redivi.com  Wed Aug 18 21:51:14 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed Aug 18 21:51:49 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
Message-ID: <F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>


On Aug 18, 2004, at 3:42 PM, Kevin Smith wrote:

> For what it's worth, I wrote the original PEP 318.  I probably wasn't 
> qualified, but I just wanted a nice simple way to declare class 
> methods without having to repeat the function name.  After submitting 
> it to BDFL for approval, more work was needed and the discussion of 
> PEP 318 on python-dev increased rapidly.  It was evident that I was in 
> over my head, so I asked more someone more experienced to take over.
>
> I guess others had bigger plans for my proposal that I had planned.  
> It has turned into the "solution" to many problems: type checking 
> (both arguments and returned values), metaclasses, metadata, 
> interfaces, function attributes, etc.).  Unfortunately, in the 
> process, this simple request for syntactical sugar has turned into a 
> monstrosity.  In my opinion, none of the proposed syntaxes really seem 
> Pythonic.  This PEP just seems to be trying to solve too many 
> problems.
>
> Bear with me, but I'd like to propose one more syntax that is simple, 
> easy for newbies to understand,  and nowhere near as powerful as the 
> current PEP's syntax.  However, it doesn't add incoherent, arbitrary 
> syntax either.
>
> def classmethod foo(x, y, z):
>     pass
>
> That's it.  One "decorator" that is a callable object that takes a 
> method as it's only argument.  No expressions, lists, tuples, etc.  
> Just one callable object.  Ok, if you absolutely must have more than 
> one.
>
> def classmethod synchronized foo(x, y, z):
>     pass
>
> Once again, no expressions.  I know that this isn't going to solve 
> everyone's type-checking, metadata, and function attribute problems, 
> but let's face it, using this PEP for all of those things just creates 
> ugly syntax.  There must be more Pythonic ways to do those things in 
> their own PEPs.

The problem is that there is FAR FAR more use of type-checking, 
metadata, and function attributes in the wild (from what I've seen and 
written) than simple stuff like classmethod, staticmethod, and this 
hypothetical synchronized.  I don't think proposals such as this do 
very much for anyone.  It's too simple to be useful, and has too many 
side-effects on existing tools and people that try and read python 
source :)

Is @syntax really SO horrible that you propose that we make decorators 
near useless to most but more "aesthetically pleasing" to some?  Do you 
use a significant amount of decorators now?  Do you plan to use them 
much in the future?  It seems to me that most people who argue against 
the sensible and flexible syntaxes are the same people who will rarely 
if ever use decorators at all...

-bob
From mal at egenix.com  Wed Aug 18 22:07:15 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 22:07:18 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123B202.5000200@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de>
Message-ID: <4123B6F3.6060708@egenix.com>

Walter D?rwald wrote:
>> I've thought about this some more. Perhaps I'm still missing
>> something, but wouldn't it be possible to add a feeding
>> mode to the existing stream codecs by creating a new queue
>> data type (much like the queue you have in the test cases of
>> your patch) and using the stream codecs on these ?
> 
> No, because when the decode method encounters an incomplete
> chunk (and so return a size that is smaller then size of the
> input) read() would have to push the remaining bytes back into
> the queue. This would be code similar in functionality
> to the feed() method from the patch, with the difference that
> the buffer lives in the queue not the StreamReader. So
> we won't gain any code simplification by going this route.

Maybe not code simplification, but the APIs will be well-
separated.

If we require the queue type for feeding mode operation
we are free to define whatever APIs are needed to communicate
between the codec and the queue type, e.g. we could define
a method that pushes a few bytes back onto the queue end
(much like ungetc() in C).

>> I think such a queue would be generally useful in other
>> contexts as well, e.g. for implementing fast character based
>> pipes between threads, non-Unicode feeding parsers, etc.
>> Using such a type you could potentially add a feeding
>> mode to stream or file-object API based algorithms very
>> easily.
> 
> Yes, so we could put this Queue class into a module with
> string utilities. Maybe string.py?

Hmm, I think a separate module would be better since we
could then recode the implementation in C at some point
(and after the API has settled).

We'd only need a new name for it, e.g. StreamQueue or
something.

>> We could then have a new class, e.g. FeedReader, which
>> wraps the above in a nice API, much like StreamReaderWriter
>> and StreamRecoder.
> 
> But why should we, when decode() does most of what we need,
> and the rest has to be implemented in both versions?

To hide the details from the user. It should be possible
to instantiate one of these StreamQueueReaders (named
after the queue type) and simply use it in feeding
mode without having to bother about the details behind
the implementation.

StreamReaderWriter and StreamRecoder exist for the same
reason.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 18 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From martin at v.loewis.de  Wed Aug 18 22:07:46 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 22:07:42 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
Message-ID: <4123B712.6040502@v.loewis.de>

Kevin Smith wrote:
> def classmethod foo(x, y, z):
>     pass
> 
> That's it.

Indeed, that is form F (5.13) in

http://www.python.org/moin/PythonDecorators

So this proposal is not new, and has the advantages
and disadvantages listed in the Wiki. The main reason
to reject it (AFAICT) is the similarity to monstrosities
such as "public final synchronized void foo()".

Regards,
Martin
From mal at egenix.com  Wed Aug 18 22:10:32 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 18 22:10:43 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <4123A8A1.9060908@v.loewis.de>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121ABAB.9060605@interlink.com.au>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>	<41227F83.4060403@egenix.com>	<4122BBDA.4050401@prescod.net>
	<41231B30.2060307@egenix.com>	<41239572.8090808@v.loewis.de>
	<41239828.9000301@egenix.com>	<41239AAD.2080604@v.loewis.de>
	<4123A075.9010407@egenix.com> <4123A8A1.9060908@v.loewis.de>
Message-ID: <4123B7B8.2090003@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
> 
>> Indeed... and the same is true for almost all other methods
>> (except maybe .replace()).
>>
>> Sounds like a lot of code will break.
> 
> 
> We will see. The default string type will be Unicode, so code
> using .lower will continue to work in many cases.
> 
> The question is what open(path,"r").read() will return. It seems
> that Guido wants users to specify "rb" if they want that to be
> byte strings.

This would imply that we'd need to add an encoding parameter
that becomes a required parameter in case 'r' (without 'b')
is specified as mode.

Perhaps we should drop 'b' altogether and make encoding
a required parameter. We could have a 'binary' codec which
then passes through the data as is (as buffer object instead
of as Unicode object for most other codecs).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 18 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From walter at livinglogic.de  Wed Aug 18 22:11:48 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Aug 18 22:12:00 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4122E19E.9050906@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>
	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>
	<411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com>
	<4122E19E.9050906@v.loewis.de>
Message-ID: <4123B804.2000508@livinglogic.de>

Martin v. L?wis wrote:

> M.-A. Lemburg wrote:
> 
>> I've thought about this some more. Perhaps I'm still missing
>> something, but wouldn't it be possible to add a feeding
>> mode to the existing stream codecs by creating a new queue
>> data type (much like the queue you have in the test cases of
>> your patch) and using the stream codecs on these ?
> 
> Here is the problem. In UTF-8, how does the actual algorithm
> tell (the application) that the bytes it got on decoding provide
> for three fully decodable characters, and that 2 bytes are left
> undecoded, and that those bytes are not inherently ill-formed,
> but lack a third byte to complete the multi-byte sequence?
> 
> On top of that, you can implement whatever queuing or streaming
> APIs you want, but you *need* an efficient way to communicate
> incompleteness.

We already have an efficient way to communicate incompleteness:
the decode method returns the number of decoded bytes.

The questions remaining are

1) communicate to whom? IMHO the info should only be used
    internally by the StreamReader.

2) When is incompleteness OK? Incompleteness is of course
    not OK in the stateless API. For the stateful API,
    incompleteness has to be OK even when the input stream
    is (temporarily) exhausted, because otherwise a feed mode
    wouldn't work anyway. But then incompleteness is always OK,
    because the StreamReader can't distinguish a temporarily
    exhausted input stream from a permanently exhausted one.
    The only fix for this I can think of is the final argument.

Bye,
    Walter D?rwald

From Kevin.Smith at theMorgue.org  Wed Aug 18 22:13:35 2004
From: Kevin.Smith at theMorgue.org (Kevin Smith)
Date: Wed Aug 18 22:13:06 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>
Message-ID: <15E4012F-F153-11D8-BC29-000393829ED2@theMorgue.org>

On Aug 18, 2004, at 3:51 PM, Bob Ippolito wrote:
>
> Is @syntax really SO horrible that you propose that we make decorators 
> near useless to most but more "aesthetically pleasing" to some?

That's kind of a loaded question.  I don't think that this simpler 
syntax is "near useless."  It fulfills pretty much all of the problems 
that the original PEP aimed to fix.  And, I don't mean to be rude, but 
yes the @syntax is that horrible.

> Do you use a significant amount of decorators now?

I used them enough to write the PEP in the first place, and I would use 
them even more if there were a better syntax for them.

> Do you plan to use them much in the future?  It seems to me that most 
> people who argue against the sensible and flexible syntaxes are the 
> same people who will rarely if ever use decorators at all...

Kevin Smith
Kevin.Smith@sas.com

From walter at livinglogic.de  Wed Aug 18 22:12:56 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Aug 18 22:13:11 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412314F6.5060502@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>
Message-ID: <4123B848.2060105@livinglogic.de>

M.-A. Lemburg wrote:

> Martin v. L?wis wrote:
> 
>> M.-A. Lemburg wrote:
>>
>>> I've thought about this some more. Perhaps I'm still missing
>>> something, but wouldn't it be possible to add a feeding
>>> mode to the existing stream codecs by creating a new queue
>>> data type (much like the queue you have in the test cases of
>>> your patch) and using the stream codecs on these ?
>>
>> Here is the problem. In UTF-8, how does the actual algorithm
>> tell (the application) that the bytes it got on decoding provide
>> for three fully decodable characters, and that 2 bytes are left
>> undecoded, and that those bytes are not inherently ill-formed,
>> but lack a third byte to complete the multi-byte sequence?
> 
> This state can be stored in the stream codec instance,
> e.g. by using a special state object that is stored in
> the instance and passed to the encode/decode APIs of the
> codec or by implementing the stream codec itself in C.

That's exactly what my patch does. The state (the bytes
that have already been read from the input stream, but
couldn't be decoded and have to be used on the next
call to read()) are stored in the bytebuffer attribute
of the StreamReader. Most stateful decoder use this
type of state, the only one I can think of that uses more
than this is the UTF-7 decoder, where the decoder decodes
partial +xxxx- sequences, but then has to keep the current
shift state and the partially consumed bits and bytes.
This decoder could be changed, so that it works with only
a byte buffer too, but that would mean that the decoder
doesn't enter incomplete +xxxx- sequences, but retains them
in the byte buffer and only decodes them once the "-" is
encountered.

In fact a trivial implementation of any stateful decoder
could put *everything* it reads into the bytebuffer when
final==False and decode itin one go once read() is called
with final==True.

But IMHO each decoder should decode as much as possible.

> We do need to extend the API between the stream codec
> and the encode/decode functions, no doubt about that.
> However, this is an extension that is well hidden from
> the user of the codec and won't break code.

Exactly: this shouldn't be anything officially documented,
because what kind of data is passed around depends on the
codec. And when the stream reader is implemented in C there
isn't any API anyway.

>> On top of that, you can implement whatever queuing or streaming
>> APIs you want, but you *need* an efficient way to communicate
>> incompleteness.
> 
> Agreed.

Bye,
    Walter D?rwald


From Kevin.Smith at theMorgue.org  Wed Aug 18 22:24:13 2004
From: Kevin.Smith at theMorgue.org (Kevin Smith)
Date: Wed Aug 18 22:23:45 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <4123B712.6040502@v.loewis.de>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
	<4123B712.6040502@v.loewis.de>
Message-ID: <91FA4761-F154-11D8-BC29-000393829ED2@theMorgue.org>

On Aug 18, 2004, at 4:07 PM, Martin v. L?wis wrote:

> Kevin Smith wrote:
>> def classmethod foo(x, y, z):
>>     pass
>> That's it.
>
> Indeed, that is form F (5.13) in
>
> http://www.python.org/moin/PythonDecorators
>
> So this proposal is not new, and has the advantages
> and disadvantages listed in the Wiki. The main reason
> to reject it (AFAICT) is the similarity to monstrosities
> such as "public final synchronized void foo()".

Yes, I realize that this syntax isn't new (I believe that it was even 
used in an example in my original PEP).  I hadn't actually read Wiki 
before, but the only minus that I really see for this syntax is that 
you can't have arguments to the decorators.  I guess I just find the 
"public final synchronized void foo()" less monstrous than

@public
@final
@synchronized
@void
def foo()

Kevin Smith
Kevin.Smith@sas.com

From martin at v.loewis.de  Wed Aug 18 22:30:14 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 22:30:33 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412314F6.5060502@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>
Message-ID: <4123BC56.7030607@v.loewis.de>

> We do need to extend the API between the stream codec
> and the encode/decode functions, no doubt about that.
> However, this is an extension that is well hidden from
> the user of the codec and won't break code.

So you agree to the part of Walter's change that introduces
new C functions (PyUnicode_DecodeUTF7Stateful etc)?

I think most of the patch can be discarded: there is no
need for .encode and .decode to take an additional argument.
It is only necessary that the StreamReader and StreamWriter
are stateful, and that only for a selected subset of codecs.

Marc-Andre, if the original patch (diff.txt) was applied:
What *specific* change in that patch would break code?
What *specific* code (C or Python) would break under that
change?

I believe the original patch can be applied as-is, and
does not cause any breakage. It also introduces a change
between the codec and the encode/decode functions that is
well hidden from the user of the codec.

Regards,
Martin
From Kevin.Smith at sas.com  Wed Aug 18 21:38:24 2004
From: Kevin.Smith at sas.com (Kevin D.Smith)
Date: Wed Aug 18 22:37:51 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?  
Message-ID: <2B21BFF4-F14E-11D8-BC29-000393829ED2@sas.com>

For what it's worth, I wrote the original PEP 318.  I probably wasn't 
qualified, but I just wanted a nice simple way to declare class methods 
without having to repeat the function name.  After submitting it to 
BDFL for approval, more work was needed and the discussion of PEP 318 
on python-dev increased rapidly.  It was evident that I was in over my 
head, so I asked more someone more experienced to take over.

I guess others had bigger plans for my proposal that I had planned.  It 
has turned into the "solution" to many problems: type checking (both 
arguments and returned values), metaclasses, metadata, interfaces, 
function attributes, etc.).  Unfortunately, in the process, this simple 
request for syntactical sugar has turned into a monstrosity.  In my 
opinion, none of the proposed syntaxes really seem Pythonic.  This PEP 
just seems to be trying to solve too many problems.

Bear with me, but I'd like to propose one more syntax that is simple, 
easy for newbies to understand,  and nowhere near as powerful as the 
current PEP's syntax.  However, it doesn't add incoherent, arbitrary 
syntax either.

def classmethod foo(x, y, z):
     pass

That's it.  One "decorator" that is a callable object that takes a 
method as it's only argument.  No expressions, lists, tuples, etc.  
Just one callable object.  Ok, if you absolutely must have more than 
one.

def classmethod synchronized foo(x, y, z):
     pass

Once again, no expressions.  I know that this isn't going to solve 
everyone's type-checking, metadata, and function attribute problems, 
but let's face it, using this PEP for all of those things just creates 
ugly syntax.  There must be more Pythonic ways to do those things in 
their own PEPs.

Kevin Smith
Kevin.Smith@sas.com

From walter at livinglogic.de  Wed Aug 18 22:35:31 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Aug 18 22:37:55 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123B6F3.6060708@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de>
	<4123B6F3.6060708@egenix.com>
Message-ID: <4123BD93.2040107@livinglogic.de>

M.-A. Lemburg wrote:

> Walter D?rwald wrote:
> 
>>> I've thought about this some more. Perhaps I'm still missing
>>> something, but wouldn't it be possible to add a feeding
>>> mode to the existing stream codecs by creating a new queue
>>> data type (much like the queue you have in the test cases of
>>> your patch) and using the stream codecs on these ?
>>
>>
>> No, because when the decode method encounters an incomplete
>> chunk (and so return a size that is smaller then size of the
>> input) read() would have to push the remaining bytes back into
>> the queue. This would be code similar in functionality
>> to the feed() method from the patch, with the difference that
>> the buffer lives in the queue not the StreamReader. So
>> we won't gain any code simplification by going this route.
> 
> Maybe not code simplification, but the APIs will be well-
> separated.

They will not, because StreamReader.decode() already is a feed
style API (but with state amnesia).

Any stream decoder that I can think of can be (and most are)
implemented by overwriting decode().

> If we require the queue type for feeding mode operation
> we are free to define whatever APIs are needed to communicate
> between the codec and the queue type, e.g. we could define
> a method that pushes a few bytes back onto the queue end
> (much like ungetc() in C).

That would of course be a possibility.

>>> I think such a queue would be generally useful in other
>>> contexts as well, e.g. for implementing fast character based
>>> pipes between threads, non-Unicode feeding parsers, etc.
>>> Using such a type you could potentially add a feeding
>>> mode to stream or file-object API based algorithms very
>>> easily.
>>
>> Yes, so we could put this Queue class into a module with
>> string utilities. Maybe string.py?
> 
> Hmm, I think a separate module would be better since we
> could then recode the implementation in C at some point
> (and after the API has settled).
> We'd only need a new name for it, e.g. StreamQueue or
> something.

Sounds reasonable.

>>> We could then have a new class, e.g. FeedReader, which
>>> wraps the above in a nice API, much like StreamReaderWriter
>>> and StreamRecoder.
>>
>> But why should we, when decode() does most of what we need,
>> and the rest has to be implemented in both versions?
> 
> To hide the details from the user. It should be possible
> to instantiate one of these StreamQueueReaders (named
> after the queue type) and simply use it in feeding
> mode without having to bother about the details behind
> the implementation.
> 
> StreamReaderWriter and StreamRecoder exist for the same
> reason.

Let's compare example uses:

1) Having feed() as part of the StreamReader API:
---
s = u"???".encode("utf-8")
r = codecs.getreader("utf-8")()
for c in s:
    print r.feed(c)
---
2) Explicitely using a queue object:
---
from whatever import StreamQueue

s = u"???".encode("utf-8")
q = StreamQueue()
r = codecs.getreader("utf-8")(q)
for c in s:
    q.write(c)
    print r.read()
---
3) Using a special wrapper that implicitely creates a queue:
----
from whatever import StreamQueueWrapper
s = u"???".encode("utf-8")
r = StreamQueueWrapper(codecs.getreader("utf-8"))
for c in s:
    print r.feed(c)
----

I very much prefer option 1).

"If the implementation is hard to explain, it's a bad idea."

Bye,
    Walter D?rwald


From martin at v.loewis.de  Wed Aug 18 22:39:22 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 22:39:24 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123B804.2000508@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>
	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>
	<411D1EB4.10103@livinglogic.de> <412285D1.7010907@egenix.com>
	<4122E19E.9050906@v.loewis.de> <4123B804.2000508@livinglogic.de>
Message-ID: <4123BE7A.60305@v.loewis.de>

Walter D?rwald wrote:
>>   But then incompleteness is always OK,
>    because the StreamReader can't distinguish a temporarily
>    exhausted input stream from a permanently exhausted one.
>    The only fix for this I can think of is the final argument.

I don't think the final argument is needed. Methinks that the
.encode/.decode should not be used by an application. Instead,
applications should only use the file API on a reader/writer.

If so, stateful readers/writers can safely implement
encode/decode to take whatever state they have into account,
creating new state as they see fit.

Of course, stateful writers need to implement their own .close function,
which flushes the remaining bytes, and need to make sure that .close
is automatically invoked if the object goes away.

Regards,
Martin
From martin at v.loewis.de  Wed Aug 18 22:51:57 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 22:51:58 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123BD93.2040107@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de>
	<4123B6F3.6060708@egenix.com> <4123BD93.2040107@livinglogic.de>
Message-ID: <4123C16D.4060505@v.loewis.de>

Walter D?rwald wrote:
> They will not, because StreamReader.decode() already is a feed
> style API (but with state amnesia).
> 
> Any stream decoder that I can think of can be (and most are)
> implemented by overwriting decode().

I consider that an unfortunate implementation artefact. You
either use the stateless encode/decode that you get from
codecs.get(encoder/decoder) or you use the file API on
the streams. You never ever use encode/decode on streams.

I would have preferred if the default .write implementation
would have called self._internal_encode, and the Writer
would *contain* a Codec, rather than inheriting from Codec.
Alas, for (I guess) simplicity, a more direct (and more
confusing) approach was taken.

> 1) Having feed() as part of the StreamReader API:
> ---
> s = u"???".encode("utf-8")
> r = codecs.getreader("utf-8")()
> for c in s:
>    print r.feed(c)

Isn't that a totally unrelated issue? Aren't we talking about
short reads on sockets etc?

I would very much prefer to solve one problem at a time.

Regards,
Martin
From walter at livinglogic.de  Wed Aug 18 23:17:31 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Aug 18 23:17:39 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123BC56.7030607@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de>
Message-ID: <4123C76B.3030100@livinglogic.de>

Martin v. L?wis wrote:

>> We do need to extend the API between the stream codec
>> and the encode/decode functions, no doubt about that.
>> However, this is an extension that is well hidden from
>> the user of the codec and won't break code.
> 
> So you agree to the part of Walter's change that introduces
> new C functions (PyUnicode_DecodeUTF7Stateful etc)?
> 
> I think most of the patch can be discarded: there is no
> need for .encode and .decode to take an additional argument.

But then a file that contains the two bytes 0x61, 0xc3
will never generate an error when read via an UTF-8 reader.
The trailing 0xc3 will just be ignored.

Another option we have would be to add a final() method
to the StreamReader, that checks if all bytes have been
consumed. Maybe this should be done by StreamReader.close()?

> It is only necessary that the StreamReader and StreamWriter
> are stateful, and that only for a selected subset of codecs.
> 
> Marc-Andre, if the original patch (diff.txt) was applied:
> What *specific* change in that patch would break code?
> What *specific* code (C or Python) would break under that
> change?
> 
> I believe the original patch can be applied as-is, and
> does not cause any breakage.

The first version has a broken implementation of the
UTF-7 decoder. When decoding the byte sequence "+-"
in two calls to decode() (i.e. pass "+" in one call and
"-" in the next), no character got generated, because
inShift (as a flag) couldn't remember whether characters
where encountered between the "+" and the "-". Now
inShift counts the number of characters (and the shortcut
for a "+-" sequence appearing together has been removed.

> It also introduces a change
> between the codec and the encode/decode functions that is
> well hidden from the user of the codec.

Would a version of the patch without a final argument but
with a feed() method be accepted?

I'm imagining implementing an XML parser that uses Python's
unicode machinery and supports the
xml.sax.xmlreader.IncrementalParser interface.

With a feed() method in the stream reader this is rather simple:

init()
{
    PyObject *reader = PyCodec_StreamReader(encoding, Py_None, NULL);
    self.reader = PyObject_CallObject(reader, NULL);
}

int feed(char *bytes)
{
     parse(PyObject_CallMethod(self.reader, "feed", "s", bytes);
}

The feed method itself is rather simple (see the second
version of the patch).

Without the feed method(), we need the following:

1) A StreamQueue class that
    a) supports writing at one end and reading at the other end
    b) has a method for pushing back unused bytes to be returned
       in the next call to read()

2) A StreamQueueWrapper class that
    a) gets passed the StreamReader factory in the constructor,
       creates a StreamQueue instance, puts it into an attribute
       and passes it to the StreamReader factory (which must also
       be put into an attribute).
    b) has a feed() method that calls write() on the stream queue
       and read() on the stream reader and returns the result

Then the C implementation of the parser looks something like this:

init()
{
    PyObject *module = PyImport_ImportModule("whatever");
    PyObject *wclass = PyObject_GetAttr(module, "StreamQueueWrapper");
    PyObject *reader = PyCodec_StreamReader(encoding, Py_None, NULL);
    self.wrapper = PyObject_CallObject(wclass, reader);
}

int feed(char *bytes)
{
     parse(PyObject_CallMethod(self.wrapper, "feed", "s", bytes);
}

I find this neither easier to implement nor easier to explain.

Bye,
    Walter D?rwald


From python-kbutler at sabaydi.com  Wed Aug 18 23:33:14 2004
From: python-kbutler at sabaydi.com (Kevin J. Butler)
Date: Wed Aug 18 23:35:13 2004
Subject: [Python-Dev] PEP 318: Suggest we drop it
Message-ID: <4123CB1A.2060101@sabaydi.com>

- There is no universally liked syntax (or even a universally 
_tolerable_ syntax)
- decorators will incosistently retain or drop docstrings (this may even 
be the most appropriate thing to do)
- a "decorated" function may have an arbitrarily distant relationship 
with the function as implemented in the def statement (this is a killer)
- if you want to decorate a single function two separate ways, you're 
back to 2.3 syntax - say, one that logs, checks parameters, etc., and 
the "internal" optimal version. None of the decorator syntaxes 
facilitate this.

Because of all these issues, I think we should drop PEP 318.

The 2.3 form seems more reasonable all the time:

def __func( self ):
    pass
spanishInquisition = mydecorator( __func )
spanishInquisition.__doc__= """Something unexpected"""

parrot = otherdecorator( __func )
parrot.__doc__ = """And now for something completely different"""

Are any of the syntaxes enough better to justify the irritation or 
divisiveness of including it?

Syntactic sugar causes cancer of the semicolon, indeed...

kb
From martin at v.loewis.de  Wed Aug 18 23:57:22 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 18 23:57:28 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123C76B.3030100@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>
Message-ID: <4123D0C2.4080505@v.loewis.de>

Walter D?rwald wrote:
> But then a file that contains the two bytes 0x61, 0xc3
> will never generate an error when read via an UTF-8 reader.
> The trailing 0xc3 will just be ignored.
> 
> Another option we have would be to add a final() method
> to the StreamReader, that checks if all bytes have been
> consumed. 

Alternatively, we could add a .buffer() method that returns
any data that are still pending (either a Unicode string or
a byte string).

> Maybe this should be done by StreamReader.close()?

No. There is nothing wrong with only reading a part of a file.

> Now
> inShift counts the number of characters (and the shortcut
> for a "+-" sequence appearing together has been removed.

Ok. I didn't actually check the correctness of the individual
methods.

OTOH, I think time spent on UTF-7 is wasted, anyway.

> Would a version of the patch without a final argument but
> with a feed() method be accepted?

I don't see the need for a feed method. .read() should just
block until data are available, and that's it.

> I'm imagining implementing an XML parser that uses Python's
> unicode machinery and supports the
> xml.sax.xmlreader.IncrementalParser interface.

I think this is out of scope of this patch. The incremental
parser could implement a regular .read on a StringIO file
that also supports .feed.

> Without the feed method(), we need the following:
> 
> 1) A StreamQueue class that

Why is that? I thought we are talking about "Decoding
incomplete unicode"?

Regards,
Martin
From Jack.Jansen at cwi.nl  Thu Aug 19 00:16:33 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Thu Aug 19 00:16:36 2004
Subject: [Python-Dev] Byte string class hierarchy
Message-ID: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>

I may have missed a crucial bit of the discussion, having been away, so 
if this is completely besides the point let me know. But my feeling is 
that the crucial bit is the type inheritance graph of all the byte and 
string types. And I wonder whether the following graph would help us 
solve most problems (aside from introducing one new one, that may be a 
showstopper):

genericbytes
	mutablebytes
	bytes
		genericstring
			string
			unicode

The basic type for all bytes, buffers and strings is genericbytes. This 
abstract base type is neither mutable nor immutable, and has the 
interface that all of the types would share. Mutablebytes adds slice 
assignment and such. Bytes, on the other hand, adds hashing. 
genericstring is the magic stuff that's there already that makes 
unicode and string interoperable for hashing and dict keys and such.

Casting to a basetype is always free and doesn't copy anything, i.e. 
the bits stay the same. 'foo' in sourcecode is a string, and if you 
cast it to bytes you'll just get the bits, which is pretty much the 
same as what you get now. If you really want to make sure you get an 
8-bit ascii representation even if you run in an interpreter built with 
UCS4 as the default character set you must use 
bytes('foo'.encode('ascii')).

Casting to a subtype may cause a copy, but does not modify the bits. 
Casting sideways copies, and may modify the bits too, the current 
unicode encode/decode stuff. These 2 rules mean that unicode('foo') is 
something different from unicode(bytes('foo')), and probably illegal to 
boot, but I don't think that's too much of a problem: you shouldn't 
explicitly cast to bytes() unless you really want uninterpreted bits.

Operations like concatenation return the most specialised class. 
Mutablebytes is the only problem case here, we should probably forbid 
concatenating these with the others. The alternatives (return 
mutablebytes, return the other one, return the type of the first 
operand) all seem somewhat random.

Read() is guaranteed only to return genericbytes, but if you open a 
file in textmode they'll returns strings, and we should add the ability 
to open files for unicode and probably mutablebytes too. I'm not sure 
about socket.recv() and such, but something similar probably holds. 
Readline() really shouldn't be allowed on files  open in binary mode, 
but that may be a bit too much.

Write and friends accept genericbytes, and binary files will just dump 
the bits. Files open in text mode may convert unicode and string 
objects between representations.

The bad news (aside from any glaring holes I may have overseen in the 
above: shoot away!) is that I don't know what to do for hash on bytes 
objects. On the one hand I would like hash('foo') == 
hash(bytes('foo')). But that leads to also wanting hash(u'foo') == 
hash(bytes(u'foo')), and we can't really have that because hash('foo') 
== hash(u'foo') is needed to make string/unicode interoperability for 
dictionaries work. Note that for the value 'foo' this isn't a problem, 
but for 'f??' (thats F O-UMLAUT O-UMLAUT) it is. So it seems that 
making hash('foo') != hash(bytes('foo')) is the only reasonable 
solution (and probably also a good idea with the future in mind: 
explicit is better than implicit so just put a cast there if you want 
the binary bits to be interpreted as an ASCII or Unicode string!) it 
will probably break existing code.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

From bac at OCF.Berkeley.EDU  Thu Aug 19 00:26:12 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug 19 00:26:17 2004
Subject: [Python-Dev] PEP 318: Suggest we drop it
In-Reply-To: <4123CB1A.2060101@sabaydi.com>
References: <4123CB1A.2060101@sabaydi.com>
Message-ID: <4123D784.6060206@ocf.berkeley.edu>

Kevin J. Butler wrote:

> - There is no universally liked syntax (or even a universally 
> _tolerable_ syntax)

Yeah, but this happens every time a new syntax is proposed.

> - decorators will incosistently retain or drop docstrings (this may even 
> be the most appropriate thing to do)

No one ever said decorators were for newbies.  There are multiple things 
to take into consideration when designing them.

> - a "decorated" function may have an arbitrarily distant relationship 
> with the function as implemented in the def statement (this is a killer)

Why?  The decorator is listed before you even reach the def statement so 
you know how its semantics will change.

> - if you want to decorate a single function two separate ways, you're 
> back to 2.3 syntax - say, one that logs, checks parameters, etc., and 
> the "internal" optimal version. None of the decorator syntaxes 
> facilitate this.
> 

Yeah, but hardly any of the other suggestions do either.  And I don't 
think this 'one function, two separate decorators' will be common enough 
to warrant requiring it for the syntax.

> Because of all these issues, I think we should drop PEP 318.
> 

I disagree.  I honestly think a bunch of people jumped the gun on this. 
  Initially I wasn't crazy about the syntax either, but then I rewrote 
one of the test cases for test_decorators and I completely changed my 
mind.  While I am not saying even the majority of people would change 
their mind if they used the syntax I do think a decent chunk of people 
would.

And I think people need to realize that there is precedent for having 
this syntactically supported.  Both Java and C#, while not my favorite 
languages, both have something like this.  It would suggest that there 
is *something* to this if two separate language teams came up with 
supporting this all while we had our own non-syntactic support.

> The 2.3 form seems more reasonable all the time:
> 

Then use it and not the new syntax.  This is not meant to come off as 
snippy, but part of the cool thing about  all of this is you  can 
*still* use the 2.2 style (I am going all the way back to the first 
version that supported descriptors and thus introduced this whole idea 
of decorators).



After all of this, I think it is time to stop arguing over new syntax 
here or suggesting we drop it.  Not enough people don't want the feature 
to convince Guido to drop it.  Right now people should be hashing things 
out on c.l.py to get down to one alternative to propose to Guido (which 
I think is rather nice of him since he doesn't have to listen to anybody 
on any of this).  Everything else has just turned into white noise at 
this point.

And this is my final email on alternative decorators until that single 
alternative for Guido comes forth since that is the only acceptable next 
step.

-Brett
From nas at arctrix.com  Thu Aug 19 00:36:29 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu Aug 19 00:36:32 2004
Subject: [Python-Dev] Byte string class hierarchy
In-Reply-To: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>
References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>
Message-ID: <20040818223629.GA17550@mems-exchange.org>

On Thu, Aug 19, 2004 at 12:16:33AM +0200, Jack Jansen wrote:
> genericbytes
> 	mutablebytes
> 	bytes
> 		genericstring
> 			string
> 			unicode

I think characters (unicode or otherwise) should not be confused
with bytes.  Having 'unicode' as a subclass of 'bytes' is very
confusing to me.

  Neil
From martin at v.loewis.de  Thu Aug 19 00:38:31 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 19 00:38:28 2004
Subject: [Python-Dev] Byte string class hierarchy
In-Reply-To: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>
References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>
Message-ID: <4123DA67.7050902@v.loewis.de>

Jack Jansen wrote:
> genericbytes
>     mutablebytes
>     bytes
>         genericstring
>             string
>             unicode

I think this hiearchy is wrong. unicode is not a specialization of
genericybytes: a unicode strings is made out of characters, not out
of bytes.

> The basic type for all bytes, buffers and strings is genericbytes. This 
> abstract base type is neither mutable nor immutable, and has the 
> interface that all of the types would share. Mutablebytes adds slice 
> assignment and such. Bytes, on the other hand, adds hashing. 

There is a debate on whether such a type is really useful. Why do you
need hashing on bytes?

> genericstring is the magic stuff that's there already that makes unicode 
> and string interoperable for hashing and dict keys and such.

Interoperability, in Python, does not necessarily involve a common base
type.

> Casting to a basetype is always free and doesn't copy anything

And, of course, there is no casting at all in Python.

> Operations like concatenation return the most specialised class. 

Assuming the hieararchy on the top of your message, what does that mean?
Suppose I want to concatenate unicode and string: which of them is
more specialized?

> Read() is guaranteed only to return genericbytes, but if you open a file 
> in textmode they'll returns strings, and we should add the ability to 
> open files for unicode and probably mutablebytes too. 

I think Guido's proposal is that read(), in text mode, returns Unicode
strings, and (probably) that there is no string type in Python anymore.
read() on binary files would return a mutable byte array.

Regards,
Martin
From barry at python.org  Thu Aug 19 00:39:22 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Aug 19 00:39:00 2004
Subject: [Python-Dev] Byte string class hierarchy
In-Reply-To: <20040818223629.GA17550@mems-exchange.org>
References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>
	<20040818223629.GA17550@mems-exchange.org>
Message-ID: <1092868762.22400.97.camel@geddy.wooz.org>

On Wed, 2004-08-18 at 18:36, Neil Schemenauer wrote:
> On Thu, Aug 19, 2004 at 12:16:33AM +0200, Jack Jansen wrote:
> > genericbytes
> > 	mutablebytes
> > 	bytes
> > 		genericstring
> > 			string
> > 			unicode
> 
> I think characters (unicode or otherwise) should not be confused
> with bytes.  Having 'unicode' as a subclass of 'bytes' is very
> confusing to me.

Agreed!

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040818/cfad824b/attachment.pgp
From guido at python.org  Thu Aug 19 03:32:27 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 19 03:32:36 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/dist/src/Python compile.c, 2.319, 2.320
In-Reply-To: Your message of "Tue, 17 Aug 2004 22:22:09 PDT."
	<E1BxItl-00060J-1J@sc8-pr-cvs1.sourceforge.net> 
References: <E1BxItl-00060J-1J@sc8-pr-cvs1.sourceforge.net> 
Message-ID: <200408190132.i7J1WRO19056@guido.python.org>

> Move the bytecode optimizer upstream so that its results are saved
> in pyc files and not re-optimized upon import.  Saves a bit of
> startup time while still remaining decoupled from the rest of the
> compiler.

Hm, shouldn't the bytecode optimizer only be used when -O is used, and
hence pyo files are being written?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug 19 03:39:33 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 19 03:39:42 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: Your message of "Wed, 18 Aug 2004 10:49:52 +0200."
	<41231830.2090903@egenix.com> 
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com>
	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>
	<41227F83.4060403@egenix.com>
	<Pine.LNX.4.58.0408180821240.1555@rnd.onego.ru> 
	<41231830.2090903@egenix.com> 
Message-ID: <200408190139.i7J1dX819100@guido.python.org>

> Hmm, who ever said that we are going to drop the current 8-bit
> string implementation ?

I expect that in Python 3.0 aka Python 3000 we'll have only Unicode
strings and byte arrays, so yes, the current 8-bit string
implementation will eventually die.  Jython and IronPython are ahead
of us in this respect...

--Guido van Rossum (home page: http://www.python.org/~guido/)
From greg at cosc.canterbury.ac.nz  Thu Aug 19 03:42:36 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug 19 03:43:34 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <41239828.9000301@egenix.com>
Message-ID: <200408190142.i7J1gaZl025182@cosc353.cosc.canterbury.ac.nz>

> I'm still not convinced that we can simply drop the existing
> immutable 8-bit string type and replace it with a mutable bytes or
> buffer type, e.g. would buffer.lower() work on the buffer itself or
> return a lowered copy ?

Byte strings probably shouldn't have a lower() method at all, or any
other method that assumes the contents represent characters.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From python at rcn.com  Thu Aug 19 04:02:04 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Aug 19 04:02:26 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.319, 2.320
In-Reply-To: <200408190132.i7J1WRO19056@guido.python.org>
Message-ID: <000f01c48590$866705e0$e841fea9@oemcomputer>

> > Move the bytecode optimizer upstream so that its results are saved
> > in pyc files and not re-optimized upon import.  Saves a bit of
> > startup time while still remaining decoupled from the rest of the
> > compiler.
> 
> Hm, shouldn't the bytecode optimizer only be used when -O is used, and
> hence pyo files are being written?

Why?  That would throw away most of the benefits to most of the users
and gain nothing in return.  The peepholer was in place in for Py2.3 and
only benefits were seen.  I would save the -O option for something where
there is a trade-off (loss of docstrings, excessive compilation time,
possibly risky optimizations, or somesuch).  Here, the peepholer is
superfast and costs almost nothing.


Raymond

From bac at OCF.Berkeley.EDU  Thu Aug 19 05:40:09 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug 19 05:40:10 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Python	compile.c, 2.319, 2.320
In-Reply-To: <000f01c48590$866705e0$e841fea9@oemcomputer>
References: <000f01c48590$866705e0$e841fea9@oemcomputer>
Message-ID: <41242119.6060300@ocf.berkeley.edu>

Raymond Hettinger wrote:
>>>Move the bytecode optimizer upstream so that its results are saved
>>>in pyc files and not re-optimized upon import.  Saves a bit of
>>>startup time while still remaining decoupled from the rest of the
>>>compiler.
>>
>>Hm, shouldn't the bytecode optimizer only be used when -O is used, and
>>hence pyo files are being written?
> 
> 
> Why?  That would throw away most of the benefits to most of the users
> and gain nothing in return.  The peepholer was in place in for Py2.3 and
> only benefits were seen.  I would save the -O option for something where
> there is a trade-off (loss of docstrings, excessive compilation time,
> possibly risky optimizations, or somesuch).  Here, the peepholer is
> superfast and costs almost nothing.
> 

Seems we need a definition of philosophy for Python.  Is a compiler 
optimization *anything* that changes the opcode initially emitted by the 
compiler, or only opcodes that can have some adverse affect, such as 
larger files or longer compile times.

If you take a strict reading of the first philosophy the optimization of 
changing a constant having the unary negative op applied to it changed 
into a negative constant (that bit me in the ass hard for my thesis 
since it changes the CST directly; bet my next bug is something 
similar).  That does change what is going to be emitted very far 
upstream.  And if I remember from the last time I looked at the peephole 
optimizer it just changes some jump values to absolute and a few other 
minor things.

Personally, I go with the latter philosophy.  -O still has its place 
since it removes asserts.  But if an optimization doesn't change the 
semantics (such as removing asserts), doesn't take a lot of time (such 
as a bunch of walks along the opcodes), or increase space (unrolling 
loops), then I don't see any harm.  Just think of it as what the 
compiler would have emitted could it be inlined within its innards easily.

-Brett
From guido at python.org  Thu Aug 19 06:01:47 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 19 06:01:54 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: Your message of "Wed, 18 Aug 2004 22:07:46 +0200."
	<4123B712.6040502@v.loewis.de> 
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>  
	<4123B712.6040502@v.loewis.de> 
Message-ID: <200408190401.i7J41lt19485@guido.python.org>

Is anybody seriously trying to come up with a single alternative
decorator proposal that most folks "out there" can support, to be
presented to me (with implementation, please!) in time for 2.4b1?

The endless repetition of old proposals on the one hand, and the
outrageous overgeneralizations being proposed on the other hand, make
me wonder if @decorator is going to win by default -- because nobody
can agree on a better proposal.  Which is fine with me, but let it be
clear that if that happens, the community has itself to blame (or to
thank, depending on which syntax you favor :-).  The ipython and Leo
authors have already said that their tools will survive @decorators.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug 19 06:07:10 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 19 06:07:17 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.319, 2.320
In-Reply-To: Your message of "Wed, 18 Aug 2004 22:02:04 EDT."
	<000f01c48590$866705e0$e841fea9@oemcomputer> 
References: <000f01c48590$866705e0$e841fea9@oemcomputer> 
Message-ID: <200408190407.i7J47Ar19512@guido.python.org>

> > Hm, shouldn't the bytecode optimizer only be used when -O is used, and
> > hence pyo files are being written?
> 
> Why?  That would throw away most of the benefits to most of the
> users and gain nothing in return.  The peepholer was in place in for
> Py2.3 and only benefits were seen.  I would save the -O option for
> something where there is a trade-off (loss of docstrings, excessive
> compilation time, possibly risky optimizations, or somesuch).  Here,
> the peepholer is superfast and costs almost nothing.

Maybe I'm out of tune, but I thought that optimizations should be
turned off by default because most people don't need them and because
of the risk that the optimizer might break something.  Haven't there
been situations in Python where one optimization or another was found
to be unsafe after having been in use (in a release!) for a long time?
I'd rather be safe than sorry.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tim.peters at gmail.com  Thu Aug 19 06:26:50 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 19 06:27:00 2004
Subject: [Python-Dev] Threading in the Standard Library Tour Part II
In-Reply-To: <200408180458.i7I4wS2a023568@cosc353.cosc.canterbury.ac.nz>
References: <200408180458.i7I4wS2a023568@cosc353.cosc.canterbury.ac.nz>
Message-ID: <1f7befae04081821265795dd24@mail.gmail.com>

>> done-right-it-only-takes-0.97c-to-give-the-appearance-of-6-inches-ly
>>    y'rs  - tim

[Greg Ewing]
> Hmmm, so Tim is claiming to have an actual length of
>
> >>> 6 / sqrt(1 - (0.97 * 0.97))
> 24.680702093691806
>
> inches...

I *am* modest.  Expandable flaccidware for bots is just a matter of
purchasing and installing appropriate materials.  But I've often
noticed that insecure humans feel compelled to carry out the
calculation, as if pitiable Nature could compete with modern hydraulic
engineering.
From rnd at onego.ru  Thu Aug 19 06:27:16 2004
From: rnd at onego.ru (Roman Suzi)
Date: Thu Aug 19 06:27:16 2004
Subject: [Python-Dev] PEP 318: Suggest we drop it
In-Reply-To: <4123CB1A.2060101@sabaydi.com>
References: <4123CB1A.2060101@sabaydi.com>
Message-ID: <Pine.LNX.4.58.0408190817070.16021@rnd.onego.ru>

On Wed, 18 Aug 2004, Kevin J. Butler wrote:

>- decorators will incosistently retain or drop docstrings (this may even
>be the most appropriate thing to do)

Well, so does any function working on functions. Probably special
Decorator class could help make decorators friendly when needed.

>- a "decorated" function may have an arbitrarily distant relationship
>with the function as implemented in the def statement (this is a killer)

Is is the case in the DecoratorLibrary on the wiki? No.

>- if you want to decorate a single function two separate ways, you're

- any use cases for this?

>back to 2.3 syntax - say, one that logs, checks parameters, etc., and
>the "internal" optimal version. None of the decorator syntaxes
>facilitate this.
>
>Because of all these issues, I think we should drop PEP 318.

Still these issues do not outweight the usefulness of decorators
and their support for "write once" concept, while in old syntax one need
to write a function name 4 times to merely get it decorated and with __doc__
string.

>The 2.3 form seems more reasonable all the time:
>
>def __func( self ):
>    pass
>spanishInquisition = mydecorator( __func )
>spanishInquisition.__doc__= """Something unexpected"""
>
>parrot = otherdecorator( __func )
>parrot.__doc__ = """And now for something completely different"""
>
>Are any of the syntaxes enough better to justify the irritation or
>divisiveness of including it?

Theoretically it is an argument, but in practice I do not think
the above construct will appear frequently enough. If you have
the case, try class instead of function+decorator.


Sincerely yours, Roman Suzi
-- 
rnd@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
From tdelaney at avaya.com  Thu Aug 19 06:37:45 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Aug 19 06:37:54 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

> Is anybody seriously trying to come up with a single alternative
> decorator proposal that most folks "out there" can support, to be
> presented to me (with implementation, please!) in time for 2.4b1?

I'm afraid that absolutely zero consensus is emerging :( I've withdrawn
myself entirely from the discussions as they've proven completely
useless.

Last time I saw behaviour like this was PEP 308. Hmm - PEP 308 -
controversial syntax. PEP 318 - controversial syntax. PEP 328 - somewhat
controversial syntax (I still don't overly like it).

Anyone want to guess what PEP 338 will be like? ;)

Tim Delaney
From pje at telecommunity.com  Thu Aug 19 06:43:40 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug 19 06:43:31 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408190401.i7J41lt19485@guido.python.org>
References: <Your message of "Wed,
	18 Aug 2004 22:07:46 +0200." <4123B712.6040502@v.loewis.de>
	<CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
	<4123B712.6040502@v.loewis.de>
Message-ID: <5.1.1.6.0.20040819003254.02db88d0@mail.telecommunity.com>

At 09:01 PM 8/18/04 -0700, Guido van Rossum wrote:
>Is anybody seriously trying to come up with a single alternative
>decorator proposal that most folks "out there" can support, to be
>presented to me (with implementation, please!) in time for 2.4b1?

I've been trying to drum up support here for option J4, a compromise 
between list-after-def and @decorators:

     def p_statement_expr(p)
        @staticmethod
        @grammarrule('statement : expression')
        @version("Added in 2.4")
        @returns(None)
    as:
        """Docstring could be here, or in the decorator part above"""
        # body goes here

I think it combines the best of several proposals; @definitions are more 
visible than list-after-def, but are not part of the function body, and you 
get to see the signature first.  It doesn't define a new keyword, "as" is 
an unambiguous pseudo-keyword here, because decorators start with "@".

The principal downside is the presence of indentation without either a : or 
enclosing brackets.  The syntax I proposed was something like:

     funcdef: "def" NAME arglist [decorators] ":" suite
     decorators: INDENT ("@" decorator_spec NEWLINE)* [docstring NEWLINE] 
DEDENT "as"

There's been no response that I've noticed, perhaps because by the time I 
proposed it people were already sick of looking at options A through 
J3.  :)  Or maybe it just sucks for some reason invisible to me.

From greg at cosc.canterbury.ac.nz  Thu Aug 19 06:55:06 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Aug 19 06:55:14 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408190401.i7J41lt19485@guido.python.org>
Message-ID: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz>

Guido:

> Is anybody seriously trying to come up with a single alternative
> decorator proposal that most folks "out there" can support, to be
> presented to me (with implementation, please!) in time for 2.4b1?

>From the posts I've seen here, the only alternatives that have
considerable popular support are ones that you've already rejected. So
I suspect nobody really feels it's worth trying.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From tim.peters at gmail.com  Thu Aug 19 07:05:50 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 19 07:05:55 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.319, 2.320
In-Reply-To: <200408190407.i7J47Ar19512@guido.python.org>
References: <000f01c48590$866705e0$e841fea9@oemcomputer>
	<200408190407.i7J47Ar19512@guido.python.org>
Message-ID: <1f7befae0408182205231a0fe4@mail.gmail.com>

[Guido]
> Maybe I'm out of tune, but I thought that optimizations should be
> turned off by default because most people don't need them and because
> of the risk that the optimizer might break something.  Haven't there
> been situations in Python where one optimization or another was found
> to be unsafe after having been in use (in a release!) for a long time?
> I'd rather be safe than sorry.

The "optimize unary minus" optimization Brett mentioned was the cause
of at least 3 distinct bugs, from gross (threw away entire parse
subtrees in some cases) to achingly subtle (excruciating consequences
from losing the distinction between plus and minus *floating* zeroes
after going thru marshal after the optimization), and which took more
than a year to discover from first to last.

Here's a bug opened this month against the peephole optimizer in 2.3
(and still in 2.4a2):

    http://www.python.org/sf/1004088

IME, optimizing has a much higher subtle-bug-per-line-of-code
ratio,than any other kind of programming work excepting cowboy
undisciplined approaches to threading.  But since I made my living
writing optimizing compilers (for threaded programs -- heh) at the
time we first corresponded about Python, I may have influenced you
toward being too jittery.  Kinda like I've scared everyone to death
about ZODB's BTrees in recent times, despite that they've been solid
as a rock for at least a year <0.5 wink>.

Python's optimizations are still so endearingly unambitious that I
don't think they're any more dangerous than the rest of Python.  For
example, I've spent far more of my recent life tracking down horrid
consequences of seemingly minor changes in asyncore.py.  So while
optimizations have been a source of bugs, they've been a very minor
source.  I also really appreciate that -O *doesn't* add "new bugs".  I
also really appreciate that bytecode optimizations are still so
unambitious that the relationship between btye code and source code
remains obvious.
From kbk at shore.net  Thu Aug 19 07:29:35 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu Aug 19 07:29:41 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200408190529.i7J5TZF7010126@h006008a7bda6.ne.client2.attbi.com>


Patch / Bug Summary
___________________

Patches :  264 open (-20) /  2556 closed (+42) /  2820 total (+22)
Bugs    :  745 open ( +1) /  4384 closed (+23) /  5129 total (+24)
RFE     :  149 open ( +0) /   130 closed ( +0) /   279 total ( +0)

New / Reopened Patches
______________________

baseinteger: abstract superclass for int and long  (2004-08-11)
       http://python.org/sf/1007068  opened by  Dmitry Vasiliev

Return new string for single item joins (Bug #1001011)  (2004-08-11)
       http://python.org/sf/1007087  opened by  Nick Coghlan

Multi-line imports implementation  (2004-08-11)
       http://python.org/sf/1007189  opened by  Dima Dorfman

make inspect.getsource show @decorators  (2004-08-09)
       http://python.org/sf/1006219  reopened by  percivall

Fix for bug #992078  (2004-08-11)
CLOSED http://python.org/sf/1007539  opened by  Mitchell Surface

@decorators, including classes  (2004-08-12)
       http://python.org/sf/1007991  opened by  Jack Diederich

patch for 767150  (2004-08-12)
       http://python.org/sf/1008086  opened by  Mats Wichmann

Let test__locale save locale (bug #992078)  (2004-08-13)
CLOSED http://python.org/sf/1008588  opened by  Johannes Gijsbers

(bug 952953) execve empty 2nd arg fix  (2004-08-14)
       http://python.org/sf/1009075  opened by  Dave Watson

(bug 1005936) textunderscore python.sty fix  (2004-08-14)
       http://python.org/sf/1009373  opened by  Dave Watson

Docstring fix in Modules/_csv.c  (2004-08-15)
CLOSED http://python.org/sf/1009384  opened by  Cherniavsky Beni

Non-ascii in non-unicode __credits__ in Lib/pydoc.py  (2004-08-15)
       http://python.org/sf/1009389  opened by  Cherniavsky Beni

bottom-to-top decorator application order  (2004-08-14)
CLOSED http://python.org/sf/1009444  opened by  Brett Cannon

Fix @decorator evaluation order  (2004-08-15)
CLOSED http://python.org/sf/1009560  opened by  Mark Russell

Add missing types to __builtin__  (2004-08-16)
       http://python.org/sf/1009811  opened by  James Y Knight

PyErr_Clear() vs. AsynchronousException  (2004-08-16)
       http://python.org/sf/1009929  opened by  Armin Rigo

thread Module Breaks PyGILState_Ensure()  (2004-08-17)
       http://python.org/sf/1010677  opened by  Phil Thompson

Bad URL encoding in SimpleHTTPServer directory listing  (2004-08-18)
       http://python.org/sf/1011123  opened by  Niels Diepeveen

regex nit in encoding declarations  (2004-08-18)
CLOSED http://python.org/sf/1011144  opened by  George Yoshida

SystemError generated by struct.pack('P', 'notanumber')  (2004-08-18)
       http://python.org/sf/1011240  opened by  Dima Dorfman

Improve error reporting when Python opens source code  (2004-08-18)
       http://python.org/sf/1011822  opened by  Danny Yoo

fix inspect.getsource breaking with line-continuation & more  (2004-08-19)
       http://python.org/sf/1011890  opened by  Simon Percivall

Patches Closed
______________

Fix for bug #992078  (2004-08-11)
       http://python.org/sf/1007539  closed by  jlgijsbers

Match Solaris def'n of _XOPEN_SOURCE  (2004-08-10)
       http://python.org/sf/1006629  closed by  montanaro

Remove ODBC library references from Windows build  (2004-08-11)
       http://python.org/sf/1006916  closed by  loewis

support --with-tsc on PPC  (2004-08-09)
       http://python.org/sf/1005891  closed by  mwh

Disambiguate "min() or max()" exception string  (2004-08-08)
       http://python.org/sf/1005468  closed by  loewis

Make func_name writable  (2004-08-06)
       http://python.org/sf/1004703  closed by  mwh

Patch to allow building of paper-*/dist.pdf  (2004-08-09)
       http://python.org/sf/1005913  closed by  nnorwitz

Avoid seg fault if list object is modified during list.index  (2004-08-09)
       http://python.org/sf/1005778  closed by  nnorwitz

Let test__locale save locale (bug #992078)  (2004-08-13)
       http://python.org/sf/1008588  closed by  jlgijsbers

cgi.py and rfc822.py unquote fixes  (2002-06-24)
       http://python.org/sf/573197  closed by  jlgijsbers

Adds cookie support to urllib2.py  (2002-04-24)
       http://python.org/sf/548197  closed by  jlgijsbers

shutil.copyfile destroys hard links (Bug [ 851123 ])  (2003-12-05)
       http://python.org/sf/854853  closed by  jlgijsbers

realpath: resolve symlinks before normalizing (bug #990669)  (2004-07-23)
       http://python.org/sf/996627  closed by  jlgijsbers

normpath symlinks doc warning (bug #990669)  (2004-07-23)
       http://python.org/sf/996626  closed by  jlgijsbers

Docstring fix in Modules/_csv.c  (2004-08-15)
       http://python.org/sf/1009384  closed by  jlgijsbers

bottom-to-top decorator application order  (2004-08-14)
       http://python.org/sf/1009444  closed by  bcannon

Fixes for 'commands' module on win32  (2003-04-01)
       http://python.org/sf/713428  closed by  jlgijsbers

Make `commands' module work on Windows  (2004-02-03)
       http://python.org/sf/889949  closed by  jlgijsbers

make commands.getstatusoutput work on windows  (2002-12-31)
       http://python.org/sf/660505  closed by  jlgijsbers

Make commands.getstatusoutput work on Windows  (2003-07-16)
       http://python.org/sf/772029  closed by  jlgijsbers

help() with readline support  (2003-04-23)
       http://python.org/sf/726204  closed by  jlgijsbers

Fix @decorator evaluation order  (2004-08-15)
       http://python.org/sf/1009560  closed by  mwh

extending readline functionality  (2003-01-27)
       http://python.org/sf/675551  closed by  montanaro

fix inspect.getargs() crash on def foo((bar)) (bug #891637)  (2004-08-08)
       http://python.org/sf/1005466  closed by  doko

email/Message.py: del_param fails when specifying a header  (2004-01-08)
       http://python.org/sf/873418  closed by  bwarsaw

regex nit in encoding declarations  (2004-08-18)
       http://python.org/sf/1011144  closed by  loewis

Fix for tkFont.Font(name=...)  (2003-07-01)
       http://python.org/sf/764217  closed by  loewis

poor performance in xmlrpc response  (2002-02-14)
       http://python.org/sf/517256  closed by  loewis

warn on assignment to None, True, False  (2002-04-26)
       http://python.org/sf/549213  closed by  loewis

Oren Tirosh's fastnames patch  (2002-08-20)
       http://python.org/sf/597907  closed by  loewis

PEP 269 Implementation  (2002-08-23)
       http://python.org/sf/599331  closed by  loewis

factor out SMTPHandler.emit date handling  (2003-08-20)
       http://python.org/sf/791776  closed by  loewis

add HTTPResponse.getheaders()  (2003-09-04)
       http://python.org/sf/800236  closed by  loewis

Suggested change in how ntpath.expanduser works  (2003-09-08)
       http://python.org/sf/802159  closed by  loewis

Lib/email/Encoders.py iso-2022 is 7bit  (2003-09-12)
       http://python.org/sf/804885  closed by  loewis

socketmodule.c: fix for platforms w/o IPV6 (i.e.Solaris 5.7)  (2003-11-19)
       http://python.org/sf/845306  closed by  loewis

inspect.getsource show @decorators  (2004-08-09)
       http://python.org/sf/1006219  closed by  jlgijsbers

use just built python interp. to build the docs.  (2003-10-15)
       http://python.org/sf/823775  closed by  doko

Missing INCREF in PyType_Ready  (2004-06-26)
       http://python.org/sf/980082  closed by  loewis

Fix readline for utf-8 locales  (2004-03-11)
       http://python.org/sf/914291  closed by  loewis

tarfile.py fix for bug #949052  (2004-07-21)
       http://python.org/sf/995126  closed by  loewis

fix for title case bug: #995422  (2004-07-22)
       http://python.org/sf/995740  closed by  loewis

New / Reopened Bugs
___________________

os.startfile() doesn't accept Unicode filenames  (2004-08-11)
       http://python.org/sf/1007046  opened by  Matthias Huening

SGI (IRIX6.5.24) Problems building nismodule.c  (2004-08-11)
       http://python.org/sf/1007223  opened by  Maik Hertha

SGI (IRIX 6.5.24) Error building _codecs_iso2022.c  (2004-08-11)
       http://python.org/sf/1007249  opened by  Maik Hertha

os.getstatusoutput on win32  (2004-08-12)
CLOSED http://python.org/sf/1008275  opened by  Chmouel Boudjnah

logging.getLevelName returns a number  (2004-08-12)
       http://python.org/sf/1008295  opened by  Paulo Eduardo Neves

os.major() os.minor() example and description change  (2004-08-12)
       http://python.org/sf/1008310  opened by  Steven

Incorrect <link rel="index"> href in Tutorial.  (2004-08-13)
CLOSED http://python.org/sf/1008690  opened by  Johannes Gijsbers

vertical bar typeset horizontal in docs  (2004-08-13)
       http://python.org/sf/1008998  opened by  Alan

non-ascii readline input crashes python  (2004-08-14)
       http://python.org/sf/1009263  opened by  paul rubin

Cannot doctest a SyntaxError  (2004-08-15)
CLOSED http://python.org/sf/1009729  opened by  Raymond Hettinger

xmlrpclib, PEP291, "yield"  (2004-08-16)
       http://python.org/sf/1009803  opened by  Anthony Baxter

CPU usage shoots up with asyncore  (2004-08-16)
       http://python.org/sf/1010098  opened by  Thomas

smtpd.py docstring says wrong default class  (2004-08-16)
       http://python.org/sf/1010102  opened by  Chris Green

xml.dom documentation omits hasAttribute, hasAttributeNS  (2004-08-16)
       http://python.org/sf/1010196  opened by  Mike Brown

sys.ps1 not protected in EditorWindow.py  (2004-08-16)
       http://python.org/sf/1010370  opened by  Dave Florek

bsddb3 testsuite failure when running more than one time  (2004-08-17)
       http://python.org/sf/1010645  opened by  Matthias Klose

test_queue fails occasionally  (2004-08-17)
       http://python.org/sf/1010777  opened by  Michael Hudson

running test_codecmaps_* takes too much effort  (2004-08-17)
       http://python.org/sf/1010952  opened by  Ronald Oussoren

distutils install with -b / --build-base  (2004-08-18)
       http://python.org/sf/1011113  opened by  daishi harada

Make GIL acquire/release functions idempotent if PyEval_Init  (2004-08-18)
       http://python.org/sf/1011380  opened by  Gustavo J. A. M. Carneiro

2.4 asyncore breaks Zope  (2004-08-18)
       http://python.org/sf/1011606  opened by  Tim Peters

distutils install with -b / --build-base  (2004-08-18)
CLOSED http://python.org/sf/1011687  opened by  daishi harada

GC assertion failure  (2004-08-18)
CLOSED http://python.org/sf/1011790  opened by  Stephane Thiell

Cannot import PKZIP 2.50 created ZIP file  (2004-08-19)
       http://python.org/sf/1011893  opened by  Dennis Chuah

PyDoc does not support ZIP  (2004-08-19)
       http://python.org/sf/1011894  opened by  Dennis Chuah

Bugs Closed
___________

IDLE and encodings  (2003-09-18)
       http://python.org/sf/808719  closed by  loewis

Possible memory leak in output console  (2004-08-10)
       http://python.org/sf/1006740  closed by  akuchling

FutureWarning when running regression tests  (2004-08-09)
       http://python.org/sf/1006001  closed by  akuchling

_XOPEN_SOURCE issue on IRIX 5.3  (2004-08-08)
       http://python.org/sf/1005308  closed by  loewis

test_format fails 2.4a1  (2004-07-16)
       http://python.org/sf/992078  closed by  jlgijsbers

_SC_PAGE_SIZE unknown on IRIX 5.3  (2004-08-08)
       http://python.org/sf/1005568  closed by  loewis

HAVE_FDATASYNC incorrectly set for 2.3.4 under cygwin  (2004-07-04)
       http://python.org/sf/985154  closed by  nnorwitz

new.code() not cleanly checking its arguments  (2004-08-07)
       http://python.org/sf/1005248  closed by  mwh

os.getstatusoutput on win32  (2004-08-12)
       http://python.org/sf/1008275  closed by  jlgijsbers

Incorrect <link rel="index"> href in Tutorial.  (2004-08-13)
       http://python.org/sf/1008690  closed by  fdrake

pdb doesn't find some source files  (2003-05-15)
       http://python.org/sf/738154  closed by  jlgijsbers

inspect.getmodule symlink-related failur  (2002-06-17)
       http://python.org/sf/570300  closed by  bcannon

inconsistent acceptance of floats for range()  (2004-08-07)
       http://python.org/sf/1005325  closed by  bcannon

shutil.copy destroys hard links  (2003-11-29)
       http://python.org/sf/851123  closed by  jlgijsbers

os.path.realpath() does not handle symlinks correctly  (2004-07-14)
       http://python.org/sf/990669  closed by  jlgijsbers

misinforming help messages in pdb.py  (2004-07-12)
       http://python.org/sf/989672  closed by  jlgijsbers

Carbon.CF.CFMutableArray hangs interpreter  (2004-07-30)
       http://python.org/sf/1000914  closed by  ronaldoussoren

pydoc crashes on a class property  (2004-02-06)
       http://python.org/sf/891637  closed by  doko

Cannot doctest a SyntaxError  (2004-08-15)
       http://python.org/sf/1009729  closed by  tim_one

super instances don't support item assignment  (2003-09-12)
       http://python.org/sf/805304  closed by  rhettinger

New module: doc versus action  (2004-05-10)
       http://python.org/sf/951482  closed by  rhettinger

winzip cannot extrat tarfile created tars  (2004-05-06)
       http://python.org/sf/949052  closed by  loewis

distutils install with -b / --build-base  (2004-08-18)
       http://python.org/sf/1011687  closed by  daishiharada

GC assertion failure  (2004-08-18)
       http://python.org/sf/1011790  closed by  mbuna

New / Reopened RFE
__________________

translate Windows cr-lf when installing scripts on Linux  (2004-08-06)
       http://python.org/sf/1004696  reopened by  timcera

RFE Closed
__________

translate Windows cr-lf when installing scripts on Linux  (2004-08-06)
       http://python.org/sf/1004696  closed by  kbk

From guido at python.org  Thu Aug 19 07:58:25 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 19 07:58:33 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: Your message of "Thu, 19 Aug 2004 16:55:06 +1200."
	<200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> 
References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408190558.i7J5wPP19708@guido.python.org>

> From the posts I've seen here, the only alternatives that have
> considerable popular support are ones that you've already
> rejected. So I suspect nobody really feels it's worth trying.

Well, do people generally buy those rejections, or is their consensus
that I'm mistaken?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tdelaney at avaya.com  Thu Aug 19 08:37:24 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Aug 19 08:37:30 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01D5906D@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

>> From the posts I've seen here, the only alternatives that have
>> considerable popular support are ones that you've already
>> rejected. So I suspect nobody really feels it's worth trying.
> 
> Well, do people generally buy those rejections, or is their consensus
> that I'm mistaken?

I don't know about consensus, but there are a considerable number of
people who think you're mistaken (it's gotten pretty heated over on
c.l.py at times).

Personally, I think you're mistaken about decorators being hidden in
list-after-def, and problems with large, multi-line decorators - I think
such problems would be few and far between. I also think that decorators
are *not* the most important part of a function's signature - even
classmethod - but should be *part* of a function's signature.

Tim Delaney
From jcarlson at uci.edu  Thu Aug 19 08:53:16 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu Aug 19 08:59:45 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408190401.i7J41lt19485@guido.python.org>
References: <4123B712.6040502@v.loewis.de>
	<200408190401.i7J41lt19485@guido.python.org>
Message-ID: <20040818234705.3A9F.JCARLSON@uci.edu>


> Is anybody seriously trying to come up with a single alternative
> decorator proposal that most folks "out there" can support, to be
> presented to me (with implementation, please!) in time for 2.4b1?

There was a poll linked from the decorators wiki, that now says
(consider the poll now closed).  If it were a democracy, the standings
are:

A. @classmethod def foo(): (107)   16%
C1. def foo() [classmethod]: (269) 39%
E1. def foo(): @classmethod (311)  45%

Both C1 and E1, which are the clear favorites, have been previously
rejected by Guido for various reasons.

If I were to vote between C1 ad E1, I'd say E1, if only because its
structure encourages reuse.

 - Josiah

From fumanchu at amor.org  Thu Aug 19 09:21:23 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Thu Aug 19 09:26:43 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E0A@exchange.hqamor.amorhq.net>

Guido:
> Is anybody seriously trying to come up with a single alternative
> decorator proposal that most folks "out there" can support, to be
> presented to me (with implementation, please!) in time for 2.4b1?

Greg:
> From the posts I've seen here, the only alternatives that have
> considerable popular support are ones that you've already 
> rejected. So I suspect nobody really feels it's worth trying.

Andrew (replying on c.l.p. for some odd reason):
> To expand on that just a bit: the two alternatives that were most
> generally favoured (in Doug Holton's estimate) were the
> "list-after-def" (C1 in the wiki), and the "pie decorator at top of
> function body" (E1 in the wiki), both of which Guido rejected.
> In the poll that Doug ran, E1 had a slight margin over C1:
> http://mail.python.org/pipermail/python-list/2004-August/233479.html
> 
> However, C1 has an implementation, while E1 doesn't. (Aside: how
> difficult would it be to create an implementation of E1?)

Seems to me it's put up or shut up time, where "put up" means
"implement". Of those proposals which have an implementation:

A1: Currently in 2.4a.
B:  Guido prefers A1 over B. I haven't seen anyone advocating B,
actually.
C1: Guido rejected.
J2: Guido rejected because of new keyword, but then "softened" his
position.

So if you can immediately reject J2, Guido, I'd say there are no
candidates for an alternate. Otherwise, I'd be happy to drum up more
support for J2 and get a complete proposal to you (something more than a
pro/con list).


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From mal at egenix.com  Thu Aug 19 11:15:17 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 11:15:25 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <200408190139.i7J1dX819100@guido.python.org>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>	<4122304E.1070907@egenix.com>
	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>	<41227F83.4060403@egenix.com>
	<Pine.LNX.4.58.0408180821240.1555@rnd.onego.ru>	
	<41231830.2090903@egenix.com>
	<200408190139.i7J1dX819100@guido.python.org>
Message-ID: <41246FA5.1010804@egenix.com>

Guido van Rossum wrote:
>>Hmm, who ever said that we are going to drop the current 8-bit
>>string implementation ?
> 
> 
> I expect that in Python 3.0 aka Python 3000 we'll have only Unicode
> strings and byte arrays, so yes, the current 8-bit string
> implementation will eventually die.  Jython and IronPython are ahead
> of us in this respect...

Ok, so I suppose that we can learn from Jython and IronPython in
this respect...

How do they handle binary data and the interfacing between various
I/O facilities, e.g. files, sockets, pipes, user input, etc.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Thu Aug 19 11:21:38 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 11:21:41 2004
Subject: [Python-Dev] Byte string class hierarchy
In-Reply-To: <1092868762.22400.97.camel@geddy.wooz.org>
References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>	<20040818223629.GA17550@mems-exchange.org>
	<1092868762.22400.97.camel@geddy.wooz.org>
Message-ID: <41247122.7030000@egenix.com>

Barry Warsaw wrote:
> On Wed, 2004-08-18 at 18:36, Neil Schemenauer wrote:
> 
>>On Thu, Aug 19, 2004 at 12:16:33AM +0200, Jack Jansen wrote:
>>
>>>genericbytes
>>>	mutablebytes
>>>	bytes
>>>		genericstring
>>>			string
>>>			unicode
>>
>>I think characters (unicode or otherwise) should not be confused
>>with bytes.  Having 'unicode' as a subclass of 'bytes' is very
>>confusing to me.
> 
> 
> Agreed!

I assume the picture will look more like this:

basesequence
    mutable
       basebytes
	   bytes
	   array
	   cStringIO
	   mmap
    immutable
       unicode
       tuple
basenumber
    integer
    float
    decimal
    complex

etc.

I've dropped basestring here since we'll probably only have one
string type in Py3k. integer covers both int and long. Perhaps
we can merge float and decimal by that time as well ?!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From g.biggs at auckland.ac.nz  Thu Aug 19 11:23:12 2004
From: g.biggs at auckland.ac.nz (Geoffrey Biggs)
Date: Thu Aug 19 11:23:26 2004
Subject: [Python-Dev] Weird errors when adding a new built-in type
Message-ID: <41247180.5070304@auckland.ac.nz>

Evening all,
This is my first time posting to the dev list so, as the email 
instructed, I'll introduce myself. My name is Geoffrey Biggs and I'm a 
PhD student at the University of Auckland in New Zealand working on some 
specialised programming language concepts. I'm implementing my ideas in 
Python to prove they work (or don't work, as the case may be). Apologies 
in advance for posting something to the python-dev list that isn't 
directly about improving the standard Python, but I figured this is the 
place where the people who know what I'm asking about and might know the 
answer would best be found and it doesn't seem to fit in python-help's 
description.

I'm trying to add a new built-in number data type to Python with its own 
syntax, so I'm working directly with the interpreter rather than 
creating my own extension module (side note: I've appended something 
extra to the version thing in the Makefile - I doubt this is relevant to 
the problem but it's probably best you have all the info). The complex 
data type is similar to what I'm trying to do so I've been following 
that as an example. I've successfully extended the tokenizer and the 
parsenumber() function in compile.c to do what I want and written the 
data type in my two new files Objects/mynewobject.c and 
Include/mynewobject.h. I've included mynewobject.h in Python.h and added 
the two files to the Makefile.
However, when I tried to run the code by typing in the syntax to produce 
my data type, Python suffers a segmentation fault. I traced this to an 
attempt to get the hash of a null pointer when adding the new instance 
of my type after parsenumber() was called. For completeness, here's the 
backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x08080e9a in PyObject_Hash (v=0x81486c0) at Objects/object.c:1162
1162            if (tp->tp_hash != NULL)
(gdb) bt
#0  0x08080e9a in PyObject_Hash (v=0x81486c0) at Objects/object.c:1162
#1  0x0808e199 in tuplehash (v=0x40324574) at Objects/tupleobject.c:244
#2  0x08080eae in PyObject_Hash (v=0x40324574) at Objects/object.c:1163
#3  0x0807b46c in PyDict_GetItem (op=0x40319c14, key=0x40324574) at 
Objects/dictobject.c:492
#4  0x080c72c7 in com_add (c=0xbfffed60, list=0x40326114, 
dict=0x40319c14, v=0x40326290) at Python/compile.c:975
#5  0x080c74ac in com_addconst (c=0xbfffed60, v=0x40326290) at 
Python/compile.c:1001
#6  0x080c90e3 in com_atom (c=0xbfffed60, n=0x4028d500) at 
Python/compile.c:1689

The crash appears to be caused because while v exists, v's members are 
all 0 and so tp becomes 0 and you get a NULL pointer exception. As far 
as I can tell, this v is the second object in the tuple created in 
com_add() and is supposed to be the type of the object being added to 
the dictionary in a tuple. Not knowing why it was coming out as zero, I 
did some more poking around (been doing a lot of that over the past 5 
days...) and found that there is a file called bltinmodule.c with a 
bunch of lines, one of which mentions the complex data type that I am 
using as a guide:
SETBUILTIN("complex", &PyComplex_Type);
So I made one of these for mynewobject and added it. I figured from this 
bit of code that the reason I was getting NULL pointer exceptions was 
because my type hadn't been initialised in some way and that this would 
do it. But here's the problem: despite trying for longer than I've slept 
in the past week, I can't get it to compile. It always produces the 
following error (after the linker step that produces the python 
executable completes):

case $MAKEFLAGS in \
*-s*)  CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall 
-Wstrict-prototypes -D__SAMMY_EXT__' ./python -E ./setup.py -q build;; \
*)  CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-g -Wall 
-Wstrict-prototypes -D__SAMMY_EXT__' ./python -E ./setup.py build;; \
esac
make: *** [sharedmods] Error 139

Running the python executable doesn't work either, it starts and then 
segfaults immediatly with the backtrace indicating it dies trying to 
build the os module (which probably didn't get built). I haven't been 
able to figure out why this is happening. If I comment out the 
SETBUILTIN line I added it will compile fine but then I go back to the 
first problem. The only thing I found in google with this error was 
someone who was having trouble with a raw python source, and in that 
case it was a problem trying to build the ssl extension.
I have structured my object identically to the complex object, which 
works. So the question (finally) is, what's broken? What have I missed? 
My best guess is only that I've broken the interpreter somehow by 
missing something with my object implementation and this is affecting 
that step of the make process.
(This mess is all happening on Linux tillinshir 2.6.7-gentoo-r11-gb #1 
Wed Aug 4 11:13:14 NZST 2004 i686 mobile AMD Athlon(tm) XP 2000+ 
AuthenticAMD GNU/Linux, if that matters.)

Thanks in advance,
Geoff Biggs
From mal at egenix.com  Thu Aug 19 11:34:19 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 11:34:25 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123B804.2000508@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<4123B804.2000508@livinglogic.de>
Message-ID: <4124741B.3080605@egenix.com>

Walter D?rwald wrote:
> Martin v. L?wis wrote:
> 
>> M.-A. Lemburg wrote:
>>
>>> I've thought about this some more. Perhaps I'm still missing
>>> something, but wouldn't it be possible to add a feeding
>>> mode to the existing stream codecs by creating a new queue
>>> data type (much like the queue you have in the test cases of
>>> your patch) and using the stream codecs on these ?
>>
>>
>> Here is the problem. In UTF-8, how does the actual algorithm
>> tell (the application) that the bytes it got on decoding provide
>> for three fully decodable characters, and that 2 bytes are left
>> undecoded, and that those bytes are not inherently ill-formed,
>> but lack a third byte to complete the multi-byte sequence?
>>
>> On top of that, you can implement whatever queuing or streaming
>> APIs you want, but you *need* an efficient way to communicate
>> incompleteness.
> 
> 
> We already have an efficient way to communicate incompleteness:
> the decode method returns the number of decoded bytes.
> 
> The questions remaining are
> 
> 1) communicate to whom? IMHO the info should only be used
>    internally by the StreamReader.

Handling incompleteness should be something for the codec
to deal with. The queue doesn't have to know about it in an
way. However, the queue should have interfaces allowing the
codec to tell whether there are more bytes waiting to be
processed.

> 2) When is incompleteness OK? Incompleteness is of course
>    not OK in the stateless API. For the stateful API,
>    incompleteness has to be OK even when the input stream
>    is (temporarily) exhausted, because otherwise a feed mode
>    wouldn't work anyway. But then incompleteness is always OK,
>    because the StreamReader can't distinguish a temporarily
>    exhausted input stream from a permanently exhausted one.
>    The only fix for this I can think of is the final argument.

A final argument may be the way to go. But it should be an
argument for the .read() method (not only the .decode() method)
since that's the method reading the data from the queue.

I'd suggest that we extend the existing encode and decode
codec APIs to take an extra state argument that holds the
codec state in whatever format the codec needs (e.g. this
could be a tuple or a special object):

encode(data, errors='strict', state=None)
decode(data, errors='strict', state=None)

In the case of the .read() method, decode() would be
called. If the returned length_consumed does not match
the length of the data input, the remaining items would
have to be placed back onto the queue in non-final mode.
In final mode an exception would be raised to signal
the problem.

I think it's PEP time for this new extension. If time
permits I'll craft an initial version over the weekend.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Thu Aug 19 11:41:51 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 11:42:05 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123BC56.7030607@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>
	<4122E19E.9050906@v.loewis.de>	<412314F6.5060502@egenix.com>
	<4123BC56.7030607@v.loewis.de>
Message-ID: <412475DF.9010308@egenix.com>

Martin v. L?wis wrote:
>> We do need to extend the API between the stream codec
>> and the encode/decode functions, no doubt about that.
>> However, this is an extension that is well hidden from
>> the user of the codec and won't break code.
> 
> 
> So you agree to the part of Walter's change that introduces
> new C functions (PyUnicode_DecodeUTF7Stateful etc)?

We'll have to see... all this is still very much brainstorming.

> I think most of the patch can be discarded: there is no
> need for .encode and .decode to take an additional argument.
> It is only necessary that the StreamReader and StreamWriter
> are stateful, and that only for a selected subset of codecs.
> 
> Marc-Andre, if the original patch (diff.txt) was applied:
> What *specific* change in that patch would break code?
> What *specific* code (C or Python) would break under that
> change?
> 
> I believe the original patch can be applied as-is, and
> does not cause any breakage. It also introduces a change
> between the codec and the encode/decode functions that is
> well hidden from the user of the codec.

Let's flesh this out some more and get a better understanding
of what is needed and how the separation between the stream queue,
the stream codec and the underlying codec implementation can
be put to good use.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Thu Aug 19 12:15:49 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 12:15:51 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123BD93.2040107@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>
	<4123B202.5000200@livinglogic.de>	<4123B6F3.6060708@egenix.com>
	<4123BD93.2040107@livinglogic.de>
Message-ID: <41247DD5.8020100@egenix.com>

Walter D?rwald wrote:
> Let's compare example uses:
> 
> 1) Having feed() as part of the StreamReader API:
> ---
> s = u"???".encode("utf-8")
> r = codecs.getreader("utf-8")()
> for c in s:
>    print r.feed(c)
> ---

I consider adding a .feed() method to the stream codec
bad design. .feed() is something you do on a stream, not
a codec.

> 2) Explicitely using a queue object:
> ---
> from whatever import StreamQueue
> 
> s = u"???".encode("utf-8")
> q = StreamQueue()
> r = codecs.getreader("utf-8")(q)
> for c in s:
>    q.write(c)
>    print r.read()
> ---

This is probably how an advanced codec writer would use the APIs
to build new stream interfaces.

> 3) Using a special wrapper that implicitely creates a queue:
> ----
> from whatever import StreamQueueWrapper
> s = u"???".encode("utf-8")
> r = StreamQueueWrapper(codecs.getreader("utf-8"))
> for c in s:
>    print r.feed(c)
> ----

This could be turned into something more straight forward,
e.g.

from codecs import EncodedStream

# Load data
s = u"???".encode("utf-8")

# Write to encoded stream (one byte at a time) and print
# the read output
q = EncodedStream(input_encoding="utf-8", output_encoding="unicode")
for c in s:
    q.write(c)
    print q.read()

# Make sure we have processed all data:
if q.has_pending_data():
    raise ValueError, 'data truncated'

> I very much prefer option 1).

I prefer the above example because it's easy to read and
makes things explicit.

> "If the implementation is hard to explain, it's a bad idea."

The user usually doesn't care about the implementation, only it's
interfaces.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Thu Aug 19 12:29:12 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 12:29:21 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123C76B.3030100@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>
Message-ID: <412480F8.4060903@egenix.com>

Walter D?rwald wrote:
> Without the feed method(), we need the following:
> 
> 1) A StreamQueue class that
>    a) supports writing at one end and reading at the other end
>    b) has a method for pushing back unused bytes to be returned
>       in the next call to read()

Right.

It also needs a method giving the number of pending bytes in
the queue or just an API .has_pending_data() that returns
True/False.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From jim at zope.com  Thu Aug 19 12:29:19 2004
From: jim at zope.com (Jim Fulton)
Date: Thu Aug 19 12:30:01 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>
Message-ID: <412480FF.2030203@zope.com>

Bob Ippolito wrote:
> 
> On Aug 18, 2004, at 3:42 PM, Kevin Smith wrote:
> 
>> For what it's worth, I wrote the original PEP 318.  I probably wasn't 
>> qualified, but I just wanted a nice simple way to declare class 
>> methods without having to repeat the function name.  After submitting 
>> it to BDFL for approval, more work was needed and the discussion of 
>> PEP 318 on python-dev increased rapidly.  It was evident that I was in 
>> over my head, so I asked more someone more experienced to take over.
>>
>> I guess others had bigger plans for my proposal that I had planned.  
>> It has turned into the "solution" to many problems: type checking 
>> (both arguments and returned values), metaclasses, metadata, 
>> interfaces, function attributes, etc.).  Unfortunately, in the 
>> process, this simple request for syntactical sugar has turned into a 
>> monstrosity.  In my opinion, none of the proposed syntaxes really seem 
>> Pythonic.  This PEP just seems to be trying to solve too many problems.
>>
>> Bear with me, but I'd like to propose one more syntax that is simple, 
>> easy for newbies to understand,  and nowhere near as powerful as the 
>> current PEP's syntax.  However, it doesn't add incoherent, arbitrary 
>> syntax either.
>>
>> def classmethod foo(x, y, z):
>>     pass
>>
>> That's it.  One "decorator" that is a callable object that takes a 
>> method as it's only argument.  No expressions, lists, tuples, etc.  
>> Just one callable object.  Ok, if you absolutely must have more than one.
>>
>> def classmethod synchronized foo(x, y, z):
>>     pass
>>
>> Once again, no expressions.  I know that this isn't going to solve 
>> everyone's type-checking, metadata, and function attribute problems, 
>> but let's face it, using this PEP for all of those things just creates 
>> ugly syntax.  There must be more Pythonic ways to do those things in 
>> their own PEPs.
> 
> 
> The problem is that there is FAR FAR more use of type-checking, 
> metadata, and function attributes in the wild (from what I've seen and 
> written) than simple stuff like classmethod, staticmethod, and this 
> hypothetical synchronized. 

I've never seen anything but classmethod, staticmethod, property
and my own "contextmethod" and "cachedproperty", both of which are
better served by the simple syntax, in the "wild".   The simple
syntax serves these uses (yes, excluding  write properties) very well.
I certainly *will* use synchronized the the future (I've used something
like it in the past.)

 > I don't think proposals such as this do very
> much for anyone. 

Oh come on.  It does a lot for me.  IMO, it does a lot for future
users of the language by keeping it simpler.  Saying that a proposal
you don't agree with doesn't do much for anyone is unfair IMO.  I
think it's pretty inappropriate to devalue arguments just because
you don't agree with them.

 > It's too simple to be useful, and has too many
> side-effects on existing tools and people that try and read python 
> source :)

Please.  Any syntax will have effects on existing source and users.
IMO, this *obvious* syntax best serves the original purpose of the
PEP, which is to solve the *simple* problem if making the syntax
of class and static methods sane.

> Is @syntax really SO horrible that you propose that we make decorators 
> near useless to most but more "aesthetically pleasing" to some? 

IMO, yes, although I'd happily accept both syntaxes. I can live with
@foo to server the people who want it, but it bugs me that I have to
write:

   @clasmethod
   def foo ....

rather than the, to me, much more obvious and readable:

   def classmethod foo ...

 > Do you
> use a significant amount of decorators now? 

I use class methods and static methods. I use properties a lot,
including lots of read properties. I have my own kinds of decorators
as well, as mentioned above.

 > Do you plan to use them
> much in the future?  It seems to me that most people who argue against 
> the sensible and flexible syntaxes are the same people who will rarely 
> if ever use decorators at all...

Well, I use this sort of thing a lot. Most of my uses are served by the simpler
syntax.  For the uses that aren't served by the simpler syntax, the @foo syntax
either doesn't help, as in properties and variations on properties, or doesn't
add anything.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From jim at zope.com  Thu Aug 19 12:31:25 2004
From: jim at zope.com (Jim Fulton)
Date: Thu Aug 19 12:32:02 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <91FA4761-F154-11D8-BC29-000393829ED2@theMorgue.org>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>	<4123B712.6040502@v.loewis.de>
	<91FA4761-F154-11D8-BC29-000393829ED2@theMorgue.org>
Message-ID: <4124817D.8030600@zope.com>

Kevin Smith wrote:
> On Aug 18, 2004, at 4:07 PM, Martin v. L?wis wrote:
> 
...


>  I guess I just find the "public 
> final synchronized void foo()" less monstrous than
> 
> @public
> @final
> @synchronized
> @void
> def foo()

Me too!

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From Michael.Sparks at rd.bbc.co.uk  Thu Aug 19 12:32:57 2004
From: Michael.Sparks at rd.bbc.co.uk (Michael Sparks)
Date: Thu Aug 19 12:39:34 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408190401.i7J41lt19485@guido.python.org>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
	<4123B712.6040502@v.loewis.de>
	<200408190401.i7J41lt19485@guido.python.org>
Message-ID: <200408191132.57095.Michael.Sparks@rd.bbc.co.uk>

Guido,


On Thursday 19 Aug 2004 05:01, Guido van Rossum wrote:
> Is anybody seriously trying to come up with a single alternative
> decorator proposal that most folks "out there" can support, to be
> presented to me (with implementation, please!) in time for 2.4b1?

Yes. The following syntax discussed on comp.lang.python struck me as having
legs so I produced an implementation.  Hopefully this email summarises the 
resulting discussion so far adequately, and if you have a moment, an answer 
to the question I pose at the end of this email would be very welcome. 

To me, this syntax appears to retain the advantages of @decorator whilst
having some clear benefits for new users, or people who only occasionally
look at python for maintenance. A number of old hands have said they too
like this syntax, and couple of others who said they think it's too
heavyweight have so far rated it -0.

Example:

# This is syntax J2 from http://www.python.org/moin/PythonDecorators
decorate:
     staticmethod
     grammarrule('statement : expression')
     versioninfo("Added in 2.4")
     deprecated
     typeinfo(None)
def p_statement_expr(self, p):
     print p[1]

# Single line form, suggested by Paul Du Bois, Nick Coghlan
decorate: staticmethod 
def p_statement_expr(self, p):
   pass

This prompted me to look at the source for the first time and I've posted an 
initial implementation (for discussion purposes) here:
   * http://mail.python.org/pipermail/python-list/2004-August/233591.html

And also referenced/discussed it here:
   * http://mail.python.org/pipermail/python-dev/2004-August/047807.html

There is one issue outstanding with this implementation -- for some reason the 
change in indent level means that scoping errors occur unless you force the 
functions into scope thus:

=========================
class Foo:
   staticmethod # This is cruft to force staticmethod into scope

   decorate:
      staticmethod
   def hello(who):
      print "woo?", who

Foo.hello("HOO")
=========================

I have held back on finishing this off because I'd like to hear whether it is 
an acceptable alternative to people.

So far people who have expressed opinions on this syntax in since I posted the 
link to the implementation are:

Bob Ippolito
   * Voted -0, too heavyweight for simple case

Paul Du Bois,
   * Suggested single line for for simple case, and also stated "This is my
     favorite proposal so far."

Barry Warsaw,
   * Voted -0, same reasons as Bob, _softened_ on thought of everything on
     single line for simple case with reservations on keyword.

Nick Coghlan,
   * Liked the fact it simplifies folding for editors, also suggested single
     line for simple case

Robert Brewer,
   * Prompted me to forward the implementation/suggestion to this list, and
     has offered to help drum up support for this, and put forward a complete
     proposal!

(apologies if I've summarised things inaccurately...)

Single line form suggested for the simple case was:

decorate: staticmethod
def p_statement_expr(self, p):
   pass

The key reasons I prefer this slightly more verbose version are:
   1 In the complex case, it makes it clearer to see where the function begins

   2 The simple cases, such as staticmethod & classmethod, are very unusual
      definitions for general code. The fact that this is more heavyweight 
      makes the staticmethod/classmethod decorator jump out at you more
      when skimming code. (especially unfamiliar code) To me this increases
      readability. Naturally the sacrifice there is an extra burden at writing
      time.

   3 It gives someone something concrete to search for when looking for help:
      eg "python decorate" vs "python @"

One clear possible downside:
   * Like @decorator these can only be declarations, not statements. This in
     itself might be confusing. This ambiguity does not exist with @decorator
     in my opinion.

Regarding 1, compare the following two hypothetical pieces of code. The 
comment block before the function deliberately attaches to the function 
because some code documentation tools I've seen will take such blocks and 
include them in the docs as well as doc strings. (Meaning doc strings can be 
short/focussed)

#Syntax J2 from http://www.python.org/moin/PythonDecorators
#Some random comment
#More random comment
decorate:
     staticmethod
     grammarrule('statement : expression')
     versioninfo("Added in 2.4")
     deprecated
     typeinfo(None)
def p_statement_expr(self, p):
     print p[1]

Using current syntax:

#Current syntax in 2.4a2
#Some random comment
#More random comment
@staticmethod
@grammarrule('statement : expression')
@versioninfo("Added in 2.4")
@deprecated
@typeinfo(None)
def p_statement_expr(self, p):
     print p[1]

I think other arguments I'd put forward are probably more aesthetic than 
practical.

As noted above, I have held off resolving the (hopefully) minor scoping issues 
since I'd wanted to hear if this is an acceptable alternative.  The responses 
so far make me think that it is acceptable to people if the simple case has 
the single line form available, but there are reservations due to the keyword 
requirement.

If you have a moment, I would very much appreciate an indication as to
whether you would reject this at this stage or not?

Best Regards,


Michael.
-- 
Michael Sparks, Senior R&D Engineer, Digital Media Group
Michael.Sparks@rd.bbc.co.uk,
British Broadcasting Corporation, Research and Development
Kingswood Warren, Surrey KT20 6NP

This e-mail contains personal views which are not the views of the BBC.
From gmccaughan at synaptics-uk.com  Thu Aug 19 12:58:12 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Thu Aug 19 12:58:45 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408190558.i7J5wPP19708@guido.python.org>
References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz>
	<200408190558.i7J5wPP19708@guido.python.org>
Message-ID: <200408191158.12738.gmccaughan@synaptics-uk.com>

On Thursday 2004-08-19 06:58, Guido van Rossum wrote:
> > From the posts I've seen here, the only alternatives that have
> > considerable popular support are ones that you've already
> > rejected. So I suspect nobody really feels it's worth trying.
> 
> Well, do people generally buy those rejections, or is their consensus
> that I'm mistaken?

I'm one of the people who doesn't really feel it's worth trying.
*My* consensus :-) is that you're mistaken. I think that

  - the @-syntax is horrible and unpythonic as all-get-out,
    but usable;

  - your arguments against the []-before-colon form are utterly
    bogus;

  - I don't recall seeing a compelling argument against the
        decorate:
            classmethod
            memoize
            wibblify(3,"aardvark")
        def make_foo(cls, *args):
            do_something(args)
    proposal that was made recently in python-dev (but must surely
    have come up before). That syntax looks much better to me
    than the pie syntax. The "only" problem is that it's weird
    to have a suite of things that aren't statements to execute,
    but it doesn't seem any weirder to me than pie. Oh, one other
    problem: the order of application is non-obvious. Same as
    with pie. (I think it's obvious, by the way, that this syntax
    should include a without-line-break version for single decorators.)

I should expand on that second point, though I expect it's
all been said before.

Here's what you said about the list-before-colon syntax,
quoted from the Wiki page about decorators, with comments.

  | That's a total jumble of stuff ending with a smiley.  (True story: I
  | left out the colon when typing up this example and only noticed in
  | proofing.)

The jumbliness of your example is a consequence of the jumbly
set of decorators it uses, and has precious little to do with
the decorator syntax being used. It doesn't look much better,
to me at least, using pie syntax. And if you're seriously using
the fact that it ends with "]:" as an argument against it, then
something has gone badly wrong.

The fact that you made a minor error of the sort Python
catches automatically while typing in an example in a
syntax you have strong negative feelings about doesn't
seem to me like much of an argument against that syntax,
either.

  | Problems with this form:
  |
  | - it hides crucial information (e.g. that it is a static method)
  |   after the signature, where it is easily missed

I don't believe it's substantially more easily missed after the signature
than before.

  | - it's easy to miss the transition between a long argument list and a
  |   long decorator list

Not much easier than to miss the transition between a long argument
list and the function body. Anyway, what's so difficult about, say,

    def cumbersomely_named_method(arg1, arg2, arg3, arg4, arg5,
                                  arg91232, arg8724, arg8123721) \
        [ staticmethod,
          accepts(int, int, str, dict, int, None, None, int),
          memoize((1, 2, 3)) ]:
        """Here's the docstring."""
        do_something()
        do_something_else(arg1)
        # etc

This sort of thing is the worst case for the list-before-colon
syntax. It still doesn't seem much worse than pie. Whereas in
the simplest case I think pie is a lot worse:

    def make_foo(cls) [classmethod]:
        return cls("foo")

    @classmethod
    def make_foo(cls):
        return cls("foo")

  | - it's cumbersome to cut and paste a decorator list for reuse, because
  |   it starts and ends in the middle of a line

If a decorator list is long enough to be worth cutting and pasting
then (1) it's probably worth building a new decorator that encapsulates
what it does, and (2) you can always use a layout that *does* put it
on separate lines, like the cumbersomely_named_method example above.

  | Given that the whole point of adding decorator syntax is to move the
  | decorator from the end ("foo = staticmethod(foo)" after a 100-line
  | body) to the front, where it is more in-your-face, it should IMO be
  | moved all the way to the front.

That isn't the whole point, though it's certainly a lot of the point.
And list-before-colon seems pretty in-your-face to me, though
admittedly it doesn't have that "Oi, you, listen up, there's something
utterly unlike anything else in Python happening here" feel about it
that pie does.

                              *

Now, having said all that: let's suppose for the sake of argument
that your arguments against list-before-colon are completely without
merit. There's still the *fact* that you dislike it, and your
intuition is notoriously pretty good :-). So I suspect that maybe
there's something really bad about list-before-colon that I haven't
noticed, or that I've not seen the importance of. But, until such
time as that's pointed out to me in a way I'm not too stupid to
appreciate, I still like list-before-colon.

                              *

I'm not sure what you're thinking *should* happen here that would
convince you (or anyone else) to adopt a different syntax. All the
other Python developers choosing another syntax and agreeing to
go with it? That's never going to happen spontaneously when there's
more than one possible alternative.

I think that either list-before-colon or indented-suite (options
C1 and J2 on the Wiki page) are clearly preferable to pie, and for
what little it's worth I hereby declare my support for any
consensus or partial consensus that might emerge for either.

-- 
g

From hyeshik at gmail.com  Thu Aug 19 14:21:50 2004
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Thu Aug 19 14:21:53 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412480F8.4060903@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de> <412480F8.4060903@egenix.com>
Message-ID: <4f0b69dc040819052178b43de9@mail.gmail.com>

On Thu, 19 Aug 2004 12:29:12 +0200, M.-A. Lemburg <mal@egenix.com> wrote:
> Walter D?rwald wrote:
> > Without the feed method(), we need the following:
> >
> > 1) A StreamQueue class that
> >    a) supports writing at one end and reading at the other end
> >    b) has a method for pushing back unused bytes to be returned
> >       in the next call to read()
> 
> Right.
> 
> It also needs a method giving the number of pending bytes in
> the queue or just an API .has_pending_data() that returns
> True/False.
> 

+1 for adding .has_pending_data() stuff.  But it'll need a way to
flush pending data out for encodings where incomplete sequence not
always invalid. <wink> This is true for JIS X 0213 encodings.

>>> u'\u00e6'.encode('euc-jisx0213')
'\xa9\xdc'
>>> u'\u3000'.encode('euc-jisx0213')
'\xa1\xa1'
>>> u'\u00e6\u0300'.encode('euc-jisx0213')
'\xab\xc4'


Hye-Shik
From mal at egenix.com  Thu Aug 19 14:25:55 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 14:25:58 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4f0b69dc040819052178b43de9@mail.gmail.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>	<4123C76B.3030100@livinglogic.de>
	<412480F8.4060903@egenix.com>
	<4f0b69dc040819052178b43de9@mail.gmail.com>
Message-ID: <41249C53.8020902@egenix.com>

Hye-Shik Chang wrote:
> On Thu, 19 Aug 2004 12:29:12 +0200, M.-A. Lemburg <mal@egenix.com> wrote:
> 
>>Walter D?rwald wrote:
>>
>>>Without the feed method(), we need the following:
>>>
>>>1) A StreamQueue class that
>>>   a) supports writing at one end and reading at the other end
>>>   b) has a method for pushing back unused bytes to be returned
>>>      in the next call to read()
>>
>>Right.
>>
>>It also needs a method giving the number of pending bytes in
>>the queue or just an API .has_pending_data() that returns
>>True/False.
>>
> 
> 
> +1 for adding .has_pending_data() stuff.  But it'll need a way to
> flush pending data out for encodings where incomplete sequence not
> always invalid. <wink> This is true for JIS X 0213 encodings.
> 
> 
>>>>u'\u00e6'.encode('euc-jisx0213')
> 
> '\xa9\xdc'
> 
>>>>u'\u3000'.encode('euc-jisx0213')
> 
> '\xa1\xa1'
> 
>>>>u'\u00e6\u0300'.encode('euc-jisx0213')
> 
> '\xab\xc4'


I'm not sure I understand. The queue will also have an .unread()
method (or similiar) to write data back into the queue at the
reading head position. Are you suggesting that we add a .truncate()
method to truncate the read buffer at the current position ?

Since the queue will be in memory, we can also add .writeseek()
and .readseek() if that helps.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From gregory.lielens at fft.be  Thu Aug 19 15:27:03 2004
From: gregory.lielens at fft.be (Gregory Lielens)
Date: Thu Aug 19 15:27:07 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408191158.12738.gmccaughan@synaptics-uk.com>
References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz>
	<200408190558.i7J5wPP19708@guido.python.org>
	<200408191158.12738.gmccaughan@synaptics-uk.com>
Message-ID: <1092922023.6275.245.camel@gauss.fft>

On Thu, 2004-08-19 at 12:58, Gareth McCaughan wrote:
> On Thursday 2004-08-19 06:58, Guido van Rossum wrote:
> > > From the posts I've seen here, the only alternatives that have
> > > considerable popular support are ones that you've already
> > > rejected. So I suspect nobody really feels it's worth trying.
> > 
> > Well, do people generally buy those rejections, or is their consensus
> > that I'm mistaken?
> 
> I'm one of the people who doesn't really feel it's worth trying.

For what it's worth, me too ;-)
First, a warning: I have never used any decorator and I jumped in this
discussion (poor me ;-) ) purely from an aesthetic/syntax-coherence
point of view. It is just because @decorator feels so odd compared to
the rest of python that I though I had to react...

Why odd? Because:

@ is a new symbol, and is slightly annoying for ipython that I use
(while I have yet to encounter decorators :-) )

@ perform a "magic" operation, applying a callable object to something
that will be defined (possibly multiple line) after (and in a certain
order that we have to remember)...This is very unlike all other binary
and unary symbol operation we have currently in python, and not common
in programming languages imho

That being said, I am not anti-punctuation, nor anti-symbol, nor
anti-@?|!#%-cause-it-trigger-my-perl-allergy ;-) (I am behind a PEP for
elementwise operators, if anybody need proof ;-) ), but in the decorator
case I do not see it as a readability improvement

And mostly, what I find odd is the application of decorators to next
definition, this feels like an entirely new syntax concept to me, while
the list after def seems more like a slight extension, so is the
decorate: 
  dec1
  dec2
def f():
 ...
proposal (although I feel like the extension is more drastic in this
case)...

So my question/feeling is: are decorator so important that a whole-new
syntax (instead of an extension of existing syntax) is needed?

For now, I do not have seen any reason to believe it is, and worse, I do
not personally find the list after def or variants (especially my own
variant, def f(...)  "pseudo-keyword" or "%" iterable-object :, of
course ;-) ) any less clear or difficult to parse than the @ version,
even with very complex decorators.

I also think at this stage the only reason to change my feeling about
this would be to have other cases beside decorator where the pie-syntax
@stuff
@more_stuff
@and_some_extra_stuff
applied_to_me
would be used, so that this new syntax have more reason to exist...

However, even if my feeling against @ pie is strong, I slowly got ready
to bite the bullet (and to pray for seeing this @ as less as possible)
giving that there is a huge multiplication of possible syntax, while
comments on each possible syntax are seemingly decreasing...This looks
like end-of-story too me, hence my "too bad, but let's move" feeling...

One possible thing that may help make an alternative syntax emerge would
be for Guido to parse the wiki and definitely reject some of the
proposals, so that people can progressively gather behind the ones
left...a sort of sorry-you're dismissed kind of game ;-)

Else, I fear that only-one-alternative-syntax-or-it-will-be-@ is a
polite way to say it-will-be-@ ;-P


Regards,

Greg.

From skip at pobox.com  Thu Aug 19 16:11:53 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 19 16:12:16 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com>
Message-ID: <16676.46377.92891.139081@montanaro.dyndns.org>


    >> Is anybody seriously trying to come up with a single alternative
    >> decorator proposal that most folks "out there" can support, to be
    >> presented to me (with implementation, please!) in time for 2.4b1?

    Tim> I'm afraid that absolutely zero consensus is emerging :( I've
    Tim> withdrawn myself entirely from the discussions as they've proven
    Tim> completely useless.

Ditto.  I think the list-after-def proposal had some support early on but
people got very carried away proposing all sorts of other alternatives.

Skip
From bac at OCF.Berkeley.EDU  Thu Aug 19 09:02:07 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug 19 16:16:58 2004
Subject: [Python-Dev] python-dev Summary for 2004-08-01 through 2004-08-15
	[draft]
Message-ID: <4124506F.6040503@ocf.berkeley.edu>

Well, after reading 1,289 emails for this summary I finally finished it. 
  As you will notice in the Summary Announcements I have my yearly stats 
on the list since this summary marks the end of year two for me doing 
this.  As I did last year, the position is available to anyone who wants 
it and can do a comparable job.

Also notice that in the Summary Announcements I am asking for people to 
vote on one of two ways I am going to cut back on the amount of time the 
Summaries take up of my time.  If you actually read the summaries then 
do let me know since votes from regular python-dev'ers will count for 
more than from the general Python community.

I don't plan to send this summary out any sooner than Sunday night so 
you have at least until then to send me corrections.  I am burned out 
enough at the moment that I doubt I will hit that date, though.

Also, I doubt I covered anything new in my decorators summary, but if I 
did let me know and I will make sure to plug it into the PEP or wiki if 
needed.

---------

=====================
Summary Announcements
=====================
Well, I have now been doing the summaries for two years.  As has become 
a yearly tradition, I am offering to pass on the writing of the 
summaries to someone else.  My only requirement is you do a comparable 
job.  You do learn *tons* about Python's internals when you have to 
research a topic to be able to summarize it.  I know it greatly 
accelerated my understanding of both Python as a language and of its 
implementation.

With that said, it is also time for stats on the list to scare away 
anyone considering taking over this job  =) .  According to my probably 
buggy script that I wrote last year, I have read 10,190 emails over the 
past year (this month has already been the busiest month with 1,289 
emails and it is only half over; could set the record for busiest month 
ever).
The people with over 300 emails posted over the year are:

9. Michael Hudson (303)
8. Martin v. Lowis (307)
7. Barry Warsaw (341)
6. Phillip J. Eby (341) # not a typo; same number as Barry
5. Greg Ewing (354)
4. Raymond Hettinger (372)
3. Skip Montanaro (399)
2. Tim Peters (629)
1. Guido van Rossum (1031)

These 9 people make up over 40% of all the emails from the past year.

Longest threads were:

1. decorate-sort-undecorate (694 emails)
2. Call for defense of @decorators (195 emails)
3. PEP 318: Decorators last before colon (181 emails)
4. redefining is (162 emails)
5. Christmas Wishlist (162 emails)

These stats along with the insane amount of email has showed me 
something; the Summaries have detracted from my participation on 
python-dev this past year.  I have a bigger mouth and more opinions than 
the number of emails I sent to the list show.  This means something has 
to change, and it isn't going to be my mouth.  The Summaries need to 
change in one of two ways.

Option one is that I focus in on certain areas of interest and skip 
other ones.  I have labeled all of the summaries below with what their 
type is right under the titles.  Please email me your top ares of 
interest.  I realize that this month may not be typical but pretty much 
all areas are covered at least once so at least there is a good taste 
for the different areas.  So, choose from:

1. improvement stuff
       Pretty much anything that is not a major change to the 
language/stdlib.  Usually something you find out from the Misc/NEWS or 
the "What's New" doc.
2. how python-dev works
       Stuff like how python-dev handles things such as the PEP process, 
coding style, etc.
3. nuggets of info
       Cool tricks and ideas that come up on the list that the greater 
world probably does not know.
4. minor language change
       Stuff that deals with the language changing, but is not 
significant; such as discussions of existing PEPs.

Language evolution stuff (such as decorators) will always be covered so 
you don't need to vote on that.  If people like this first option then I 
will make sure I cover the area with most votes and everything else is 
just considered icing.

Option two out of all of this is people just say, "summarize what you 
want, Brett."  Then I just cover what I find interesting and just don't 
worry about covering a specific area.  I obviously prefer this option 
but if people really care about a specific area I want to make sure to 
cover it.  What will most likely happen is I will still cover almost 
everything but the thoroughness will be lower.  I will go more off of 
memory for example.

But something will change.  Being on the sidelines for the decorators 
discussion because I dreaded having to start reading all of those emails 
in terms of summarizing is not acceptable (and no, I can't just ignore 
some of them since that is just not how I work).  Having to read 1,289 
emails for just the first two weeks of this month finally caused me to 
lose my sanity.

========
Summary
========
--------------------------------------------------------------------------------
Multiple interpreters at a time in a C program kind of broken
--------------------------------------------------------------------------------
3. nuggets of info

Philip Eby thought he might have a fix for a known limitation of running 
multiple interpeters at once (using PyInterpreter_New() ) and having 
imports not being clearly separated between interpreters.  But Martin v. 
L?wis popped Philip's bubble somewhat by saying that he and some other 
people viewed multiple interpreter support as inherently broken.

Contributing threads:
   - `Safe to change a thread's interpreter? 
<http://mail.python.org/pipermail/python-dev/2004-August/046593.html>`__


--------------------------------------------
Another sucker becomes an official developer
--------------------------------------------
2. how python-dev works

Sean Reifschneider took the bait to become another one of Guido's minions.

Sean has been maintaining the RPM spec files for quite a while.

Contributing threads:
   - `New developer 
<http://mail.python.org/pipermail/python-dev/2004-August/046595.html>`__


----------------------------------------
Discovering leaks with a regrtest wrench
----------------------------------------
1. minor language change

Michael Hudson, who found and fixed a bunch of refcount leaks shortly 
after 2.3 was released, used his magical regrtest patch (which has 
subsequently been checked in) to hunt down leaks for 2.4 .  A bunch of 
tests were cleaned up to make them not raise false-positives along with 
fixing some true leaks.

Contributing threads:
   - `refleak hunting season 
<http://mail.python.org/pipermail/python-dev/2004-August/046611.html>`__


---------------------------------
Another Bug Day has come and gone
---------------------------------
2. how python-dev works

The Bug Day held on August 7 led to 19 bugs and 12 patches being closed. 
  You can see the results at 
http://www.python.org/cgi-bin/moinmoin/PythonBugDayStatus .

Feel guilty for not being able to make it?  Still want to help?  Go to 
that page to see bugs that could use some squashing that shouldn't be 
too difficult to deal with.

Contributing threads:
   - `Bug day coming up this Saturday 
<http://mail.python.org/pipermail/python-dev/2004-August/046645.html>`__


--------------------------------------------------
How to shut a compiler up about an unused variable
--------------------------------------------------
3. nuggets of info

Tim Peters had come up with some optimizations for list.pop() that 
avoided unneeded test+branches.  The problem is that it led to a 
variable possibly being unused.  C compilers tend to complain about that 
and so an initial solution was used.  Unfortunately gcc complained about 
it, and so yours truly came up with one.  But that solution was labeled 
as "perverse" (and it was; total hack solution), so another solution was 
found thanks to Sjoerd Mullender by just taking the variable and casting 
it to void.

Contributing threads:
   - `RE: [Python-checkins] python/dist/src/Objects listobject.c, ... 
<http://mail.python.org/pipermail/python-dev/2004-August/046678.html>`__


------------------------------------------------------
Variable sequence unpacking assignment shot down again
------------------------------------------------------
language evolution

David Cole suggested adding the ability to have sequence unpacking 
assignment just like *args for parameters; ``a, b, *c = (0, 1, 2, 3, 4, 
5)  # a == 0, b == 2, c == (3, 4, 5)``.  This idea got a -1 from Guido 
as not really needed.  That reaction makes Guido consistent; the idea 
was originally shot down back in Nov. 2002; 
http://www.python.org/dev/summary/2002-11-16_2002-11-30.html#half-baked-proposal-and-in-assignments 
.  If you really like that idea the previous summary contains a function 
that helps you do this in a way.

Contributing threads:
   - `Tuple/list assignment question 
<http://mail.python.org/pipermail/python-dev/2004-August/046684.html>`__


-------------------------------------------------------------------------------------
Changing the Big-O complexity for something in the language is now a 
language feature
-------------------------------------------------------------------------------------
language evolution

Armin Rigo came up with a way to have string concatenation in a loop 
(think ``for thing in iter_of_strings: concat_str += thing``) not be a 
quadratic algorithm thanks to some trickery for ``a = a + b`` and ``a += 
b`` conditions for strings.  The hope was to remove the (commonly 
considered) wart of having ``"".join(iter_of_strings)`` be the suggested 
way to concatenate a bunch of strings.

But Guido didn't like the patch.  His reasoning was that changing 
something that led to a change in the Big-O complexity of certain 
algorithms would inherently hurt other implementations of Python when 
people would start to code specifically for that performance gain.  For 
instance, having Jython be able to pull this trick off is, I believe, 
near impossible.  So, in order to make sure changes like this are 
considered before applying them, Guido instated a new rule that 
"implementation features that affect not just the running speed but the 
O() rate for certain algorithms should be considered language features, 
because any implementation will be required to implement them in order 
to ensure code portability" between implementations of Python.

In the end, though, this went in with a warning that the speed 
performance is not portable.  It is not to be used in the stdlib ever.

Contributing threads:
   - `Optimized string concatenation 
<http://mail.python.org/pipermail/python-dev/2004-August/046686.html>`__
   - `PEP 0008 confusion - here it is, but don't use it? 
<http://mail.python.org/pipermail/python-dev/2004-August/047194.html>`__


-------------------------------------------------------------------------------
Bet you didn't think about string interning and how that affects .pyc, 
did you?
-------------------------------------------------------------------------------
1. minor language change

Luckily Martin v. L?wis did.  A patch was originally applied to not 
intern strings representing filenames.  Problem is that every code 
object stores that string, so it increases .pyc files since it has to 
store multiple copies of that string instead of just one.

Contributing threads:
   - `Re: [Python-checkins] python/dist/src/Python compile.c ... 
<http://mail.python.org/pipermail/python-dev/2004-August/046703.html>`__


------------------------------------------------------------------------------
`PEP 324`_ (process - New POSIX process module) != process.py from 
ActiveState
------------------------------------------------------------------------------
1. minor language change

Guido asked if the APIs between the module proposed in `PEP 324`_ and 
the process.py module by Trent Mick of ActiveState were compatible. 
Turns out they are not.  Then the discussion went into select(), broken 
pipes, and other stuff not relevant to the OP.  =)

.. _PEP 324: http://www.python.org/peps/pep-0324.html

Contributing threads:
   - `PEP 324 (process module) 
<http://mail.python.org/pipermail/python-dev/2004-August/046707.html>`__


--------------------------------------------------------------------------------------------------------------------------------------------------
Getting it so string literals accept those nutty non-ASCII characters 
more easily (and getting files to be more upfront about their Unicode-ness)
--------------------------------------------------------------------------------------------------------------------------------------------------
language evolution

Fran?ois Pinard asked what people thought about two things related to 
Unicode.  First, he thought having a __coding__ attribute for files that 
contained the encoding of the text would be nice.  Martin v. L?wis said 
it would be possible.  MA Lemburg added his vote of support.  It has not 
been implemented to my knowledge yet, though.

The second idea was to have a directive of some sort on a per-module 
basis so that all string literals could be considered in something other 
than ASCII.  Once again Martin said it was doable with MA saying he 
liked the idea.  But it was pointed out that you might as well just use 
the 'u' in front of strings now.

This led to a slight discussion on good i18n practices.  Both Martin and 
MA seemed to suggest that if you are going to be doing most of your work 
in a single encoding then just use that and declare everything Unicode. 
  ANd if you are going to support a lot of different languages, use 
gettext and such.  Martin also stated it is best to get text into 
Unicode ASAP and then only convert to the final encoding at the last moment.

Contributing threads:
   - `Python in Unicode context 
<http://mail.python.org/pipermail/python-dev/2004-August/046727.html>`__


----------------------------------------------------------------------
An exception is an exception, unless it doesn't inherit from Exception
----------------------------------------------------------------------
language evolution

Coming up in a discussion on Unicode of all things, a discussion on 
moving exceptions over to new-style classes came up.  Guido pointed out 
that moving over to new-style classes would seem to suddenly require 
that anything passed to 'raise' be a new-style class and that goes 
against the current acceptance.

But in comes Paul Prescod with his argument that exceptions are 
inherently organized based on inheritance and thus requiring things 
being passed to 'raise' subclass Exception somewhere is not that big of 
a thing.  Guido really liked that point.  So if it goes this way and you 
don't like it blame Paul (who is a nice guy, so go easy on him  =).

And then along come Holger Krekel bringing up the point that using 
exceptions to do non-standard flow control is handy.  But the question 
was asked as to why he couldn't still just subclass Exception?  Was it 
that big of a deal?  He admitted it wasn't and said when the name 
Raisable was suggested that would make it easier.

Michael Hudson then came in and wrote up a hacked-up patch to turn 
exceptions into new-style classes.  Most stuff seemed to still work.

Python 3 was already going to switch to new-style classes for exceptions 
and string exceptions have already been deprecated.  Now add to the mix 
the possible requirement that anything passed to 'raise' require a 
common base class.

Contributing threads (note that the thread starts part way in a thread 
on Unicode and end before the end of the full thread):
   - `Python in Unicode context 
<http://mail.python.org/pipermail/python-dev/2004-August/046757.html>`__


-------------------------
Python 2.4a2 out the door
-------------------------
language evolution

Python 2.4a2 has been released.  As usual, please download it and test 
it with your code along with the regression test suite.

Contributing threads:
   - `trunk frozen for 2.4a2 
<http://mail.python.org/pipermail/python-dev/2004-August/046822.html>`__
   - `RELEASED Python 2.4, alpha 2 
<http://mail.python.org/pipermail/python-dev/2004-August/046864.html>`__


-------------------------------------------------------
Amendment to how to compile with the free .NET compiler
-------------------------------------------------------
3. nuggets of info

Nick Coghlan expanded upon my summary on how to compile with the free 
.NET compiler under Windows at 
http://www.python.org/dev/summary/2004-07-16_2004-07-31.html#how-to-get-python-to-compile-with-microsoft-s-free-compiler-that-should-just-come-with-the-os-standard 
.  See his email at 
http://mail.python.org/pipermail/python-dev/2004-August/047215.html on 
the extra steps.

Or just move to another OS.

Contributing threads:
   - `python-dev Summary for 2004-07-16 through 2004-07-31 [draft] 
<http://mail.python.org/pipermail/python-dev/2004-August/047215.html>`__


--------------------------------------
Darned Solaris, g++, and _XOPEN_SOURCE
--------------------------------------
1. improvement stuff

Skip Montanaro discovered that building under Solaris with g++ raised a 
warning about redefining _XOPEN_SOURCE.  pyconfig.h defines it, but 
apparently so does g++ in order for Solaris' toolchain to expose some 
code that is only available if it is defined.  Martin v. L?wis came up 
with a couple of suggestions on how to handle this.  Skip ended up going 
with the idea of setting _XOPEN_SOURCE to the same value as it is 
defined by g++.

Contributing threads:
   - `use of #pragma GCC system_header to suppress _XOPEN_SOURCE 
warnings 
<http://mail.python.org/pipermail/python-dev/2004-August/046955.html>`__


-------------------------------------------------------------------------------------
pre-PEP on a function for re that "captures matches for groups that 
match repeatedly"
-------------------------------------------------------------------------------------
1. minor language change

Mike Coleman presented a pre-PEP on adding a function to re that would 
create a tree (in the form of a list) containing all group matches in a 
string.  It got a little support, but the discussion quickly moved over 
to dreamings of a full-on scanner or parser package for the stdlib.

But if the idea of the PEP works for you then speak up on c.l.py .

Contributing threads:
   - `pre-PEP: Complete, Structured Regular Expression Group Matching 
<http://mail.python.org/pipermail/python-dev/2004-August/047042.html>`__


---------------------------------------
Making ourselves follow the PEP process
---------------------------------------
2. how python-dev works

It was noticed early on that the PEP process had broken down for `PEP 
318`__ (decorators).  What should happen is a PEP gets written (and the 
author becomes its champion), there is a public discussion, the PEP is 
updated, that's repeated until it is deemed done, gets get BDFL 
pronouncement, if Guido okays it the code goes in.  Unfortunately the 
part about  updating the PEP didn't really happen.

This led to two major things being stated and generally agreed upon. 
One is that PEPs should not be checked in if the PEP has not been 
updated.  While a minor nit is not a reason to hold back code, not 
updating after a major public debate is not acceptable.

This directly segued into the other point of a PEP needs to have a 
champion, period.  Since the developers on python-dev do not have the 
time to keep PEPs updated it is up to the PEP champion to make sure it 
is kept current.  If it isn't it is take to mean the champion no longer 
cares, which means python-dev no longer cares, which means the PEP gets 
rejected outright.  This will also make sure that there is a focus to 
the design of what the PEP wants and does not lead to a 
design-by-committee problem.

.. _PEP 318: http://www.python.org/peps/pep-0318.html

Contributing threads:
   - `PEP 318, and the PEP process 
<http://mail.python.org/pipermail/python-dev/2004-August/047020.html>`__


----------------------------------------------------------------
How to tell Windows from Linux without lifting up anyone's skirt
----------------------------------------------------------------
3. nuggets of info

The question came up on what the best way was to tell what platform you 
are running on.  The answer came down to why you were cared.  If it was 
whether or not you had a specific functionality (or lack thereof), then 
just test for the functionality.  If you had other needs, though, using 
sys.platform seemed to be the most reliable way (at least for Windows 
since all of them use 'win32').

Contributing threads:
   - `Asyncore .set_reuse_addr() on Windows 
<http://mail.python.org/pipermail/python-dev/2004-August/047269.html>`__


-----------------------------------------
func_name can now have an identity crisis
-----------------------------------------
1. improvement stuff

Thanks to the rampant decorator discussion, Skip Montanaro came up with 
the idea of wanting func_name to be writable.  This would allow 
decorators to wrap a function with another function and yet reset 
func_name to its original value, thus not having the wrapper function 
become the prevailing name.

Guido liked and Michael Hudson subsequently wrote up `patch #1004703`_.

.. _patch #1004703: http://www.python.org/sf/1004703

Contributing threads:
   - `PEP 318 - writing to func_name attribute 
<http://mail.python.org/pipermail/python-dev/2004-August/047097.html>`__


-----------------------------------------
statistics.py ain't goin' into the stdlib
-----------------------------------------
1. improvement stuff

Raymond Hettinger (at the last Python Bug Day) made the statement that 
he didn't think 'statistic's should be taken out of the sandbox in CVS 
and put into the stdlib.  He felt that most of the important algorithms 
were available elsewhere (such as nsmallest and nlargest in the heapq 
module) and the remaining functions that were not found anywhere were 
not that critical.

He did say, though, he would like to do a calculator module where those 
remaining functions could go.

Contributing threads:
   - `status of statistics.py? 
<http://mail.python.org/pipermail/python-dev/2004-August/047129.html>`__


------------------------------------
Making re.split care about emptiness
------------------------------------
1. improvement stuff

Mike Coleman wrote up a patch for re.split() that AM Kuchling presented 
to the list.  It adds an argument to the function to allow an empty 
string to be put into the resulting list when a match occurs, even if 
the match consumes no characters.

No decision on the exact semantics of the function, how to handle 
turning on the new functionality (some say an extra argument, some say 
adding another bit flag like traditional re arguments), or even if it 
will be accepted.

Contributing threads:
   - `re.split on empty patterns 
<http://mail.python.org/pipermail/python-dev/2004-August/047272.html>`__


----------------------------------------
Pickler's 'bin' argument on its last leg
----------------------------------------
1. improvement stuff

As per `PEP 307`_ (Extensions to the pickle protocol), the 'bin' 
argument is being upgraded from PendingDeprecationWarning to 
DeprecationWarning; so it's gone plaid and the only way to stop it is 
with some emergency change.

Contributing threads:
   - `Pickler()'s bin argument 
<http://mail.python.org/pipermail/python-dev/2004-August/047274.html>`__


---------------------------------
Some modules are getting the boot
---------------------------------
1. improvement stuff

TERMIOS, mpz, statcache, xreadlines and rotor are all being removed. 
mimify, MimeWriter, and whrandom will raise a DeprecationWarning. 
rfc822 and mimetools will not be raising DeprecationWarning as specified 
by `PEP 4`_ since some code in the stdlib still uses it.

.. _PEP 4: http://www.python.org/peps/pep-0004.html

Contributing threads:
   - `Removing stuff from 2.4 
<http://mail.python.org/pipermail/python-dev/2004-August/047308.html>`__


-------------------------------------------------
Should decimal.Context.copy() be deep or shallow?
-------------------------------------------------
1. improvement stuff

Raymond Hettinger brought up the question of whether 
decimal.Context.copy() should be a deep or shallow copy.  While 
tradition dictates it be shallow based on name, it seems like the 
functionality should be deep.  No one wants context to be shared between 
number naturally since numbers tend to be viewed as unique things.

Staring in 2.4a3 it will be deep unless people come up with reasons to 
switch it to shallow.

Contributing threads:
   - `decimal.Context.copy() shallow or deep? 
<http://mail.python.org/pipermail/python-dev/2004-August/047416.html>`__


----------------------------------------------------------
The dark corners of Python allow you to skip return values
----------------------------------------------------------
3. nuggets of info

Christian Tismer discovered that you can actually stop a return 
statement from returning if you encase it in a 'finally' block and tweak 
it slightly (see the OP to get what I mean by this).  Turns out this is 
a reliable feature of Python if you really want to use it.

Contributing threads:
   - `capturing RETURN_VALUE 
<http://mail.python.org/pipermail/python-dev/2004-August/047360.html>`__


--------------------------------------
Is an int/long combined type worth it?
--------------------------------------
language evolution

Dmitry Vasiliev pointed out that `PEP 237`_ (Unifying Long Integers and 
Integers) mentioned that a a new type named 'integer' might be 
introduced that subclassed both int and long.  The discussion waivered 
between whether it was at all needed, and if it was if it should be a 
true type or just a tuple containing both types for use with isinstance() .

No conclusion was reached in the end.

.. _PEP 237: http://www.python.org/peps/pep-0237.html

Contributing threads:
   - `Unifying Long Integers and Integers: baseint 
<http://mail.python.org/pipermail/python-dev/2004-August/047404.html>`__


--------------------------------------
Should byte arrays be a built-in type?
--------------------------------------
language evolution

Through the discussion bout having a common type combining int and long 
a discussion on whether a byte array type should be introduced.  The 
initial suggestion was for it to be practically synonymous with str 
since str itself stores everything as an array of 8-bit values.  The use 
cases would be for streams and such that just want a stream of bytes 
with no care for any encoding.

Syntactically, having a 'b' and 'B' cookie before literals was 
suggested.  The issue with this, though, is that byte arrays should be 
mutable, which would make literals that can be mutated, and that is 
frowned upon by Guido.

Semantically, aliasing bytes to the str type was suggested.  That was 
not loved since that could create confusion.  Returning an object from 
'array' was suggested and seemed okay.

In general it seemed this could be useful and could go in 2.5, but 
nothing for 2.4 .

Contributing threads:
   - `Unifying Long Integers and Integers: baseint 
<http://mail.python.org/pipermail/python-dev/2004-August/047665.html>`__


-----------------------------
Thar the Windows stack blows!
-----------------------------
1. minor language change

Running the new test_compiler test (if you run it with ``-uall`` for 
regrtest it will recompile the entire stdlib) was leading to an odd 
failure: the "process on Windows "just vanishes" without a trace, and 
without an error message of any kind, but with an exit code of 128". 
After a lot of work put in by a bunch of people (led by Tim Peters) the 
problem was tracked down to a blown C stack.

Turned out that the stack-checking code was not being called frequently 
enough to pick up the error.  The problem with this was that it was 
leading to odd errors that should have been MemoryError but were 
manifesting themselves as KeyError.  This was because PyDict_GetItem() 
was failing and return NULL which is the same as signaling the key 
doesn't exist in the dict.

Trick was trying to come up with a good way to deal with this.  Adding 
more calls would be very expensive (reliable way of catching it was 
sticking a check in pymalloc code) and so that was ruled out. 
PyDict_GetItem() couldn't change its return value since that would break 
more code than could be imagined.  So, in the end, the stack was 
increased to 2 MB on Windows.

Contributing threads:
   - `Another test_compiler mystery 
<http://mail.python.org/pipermail/python-dev/2004-August/047468.html>`__


------------------------------------------
Someone else falls for the dangling carrot
------------------------------------------
2. how python-dev works

Johannes Gijsbers now has checkin rights.  May he close many bugs.

Contributing threads:
   - `New Developer 
<http://mail.python.org/pipermail/python-dev/2004-August/047815.html>`__


----------------------------------------------------------
Lying about being in __builtin__ is not always a bad thing
----------------------------------------------------------
1. improvement stuff

James Knight noticed that some objects (mostly C types such as iterators 
for the built-in types and such) claim in their __module__ attribute 
that they come from __builtin__ which is not technically correct since 
you can't access them from there.  The desire to fix this came from 
allowing for proper introspection.  The leading idea is to put these 
types in __builtin__ properly so that they are no longer lying about 
where they come from.

Contributing threads:
   - `Classes that claim to be defined in __builtin__ but aren't 
<http://mail.python.org/pipermail/python-dev/2004-August/047477.html>`__


---------------------------------------------------------
Bringing order to the order of application for decorators
---------------------------------------------------------
1. minor language change

Turns out that the order of application for decorators was implemented 
in the reverse order of what it was supposed to be in 2.4a2 .  Luckily 
James Knight spotted this and let the list know.  It has been fixed, 
though, in CVS and now follows `PEP 318`_ to apply in bottom-up order::

   @first
   @second
   def blah(): pass

is equivalent to::

   blah = first(second(blah))

The arguments for this ordering beyond it just making more sense to 
Guido and others is that people will typically put decorators such as 
staticmethod and classmethod first.  Going with the original order would 
have led to errors in most situations if people were not expecting to be 
receiving a class or static method to be passed to the other decorators.

There was a short discussion on the order of instantiation for the 
decorators, but in the end the order chosen was the order listed; first 
would be first() instantiated followed by second().

Contributing threads:
   - `Decorator order implemented backwards? 
<http://mail.python.org/pipermail/python-dev/2004-August/047512.html>`__


-----------------------------------------------------------------------------------
PEP 292 (Simpler String Substitutions) getting gussied up for the Python 
2.4a3 ball
-----------------------------------------------------------------------------------
1. minor language change

`PEP 292`_'s implementation got fixed up.  The names of the class names 
were changed to Template and SafeTemplate, the whole process was made 
lazy, and just cleaned up in general (solution is small, simple, and 
rather cool; all subclassable and works with Unicode to boot).

But then questions over the merits of $-based string interpolation 
popped up.  People wondered if going from ``%(blah)s`` to ``${blah}`` 
was worth it.  The answer is "yes".  And that is just worst case; when 
you just want ``$blah`` you get an even bigger win.

The other question was over whether the string module should become a 
package.  The original idea was to stick the original string module in 
as a submodule of the package and the Template code in another module. 
This would allow easy removal of the 'string' module code that has been 
deprecated for eons.  Barry Warsaw (author of the PEP) asked Guido to 
make the call on this, but he hasn't had the time yet to get to this.

.. _PEP 292: http://www.python.org./peps/pep-0292.html

Contributing threads:
   - `PEP 292 
<http://mail.python.org/pipermail/python-dev/2004-August/047580.html>`__


--------------------------------
Multi-line imports get clarified
--------------------------------
1. minor language change

Multi-line imports will only accept parentheses around what is being 
explicitly imported and only if there is more than one item being 
imported.  Trailing commas are also to be accepted.

Contributing threads:
   - `Multi-line import implementation (first part) 
<http://mail.python.org/pipermail/python-dev/2004-August/047393.html>`__
   - `Multi-line import implementation (second part) 
<http://mail.python.org/pipermail/python-dev/2004-August/047614.html>`__
   - `Naming nit 
<http://mail.python.org/pipermail/python-dev/2004-August/047641.html>`__


------------------------------------------------
For those of you who need Python to run on Win64
------------------------------------------------
3. nuggets of info

Nick Bastin asked if anyone has gotten Python 2.3 to work under Win64. 
Martin v. L?wis said "yes" for Python 2.4, but not for 2.3 .  He 
suggested to Nick that he run vsextcomp_ to generate the targets on the 
2.4 VC 7 build files and then move that over to 2.3 .

.. _vsextcomp: http://www.sf.net/projects/vsextcomp

Contributing threads:
   - `2.3.4 on Win64? 
<http://mail.python.org/pipermail/python-dev/2004-August/047631.html>`__


----------------------------------------------
Sometimes concealing the truth is a good thing
----------------------------------------------
1. improvement stuff

Nick Coghlan discovered that some of the function in the 'commands' 
module did actually work under Windows and he wanted to make sure it was 
okay to fix another module to work under Windows and to document the 
fact.  But the whole idea was shot down by both Tim Peters and Guido in 
order to keep the module simple.  Keeping the whole thing UNIX-only is 
much easier than having an API that is only partly compatible with 
Windows (and with only certain versions of Windows at that).  Guido also 
said that the module would not even be accepted today since it doesn't 
add enough functionality.

Contributing threads:
   - `'commands' module on Win32 
<http://mail.python.org/pipermail/python-dev/2004-August/047774.html>`__


----------------------------------------------
atexit module, good; sys.exitfunc, not so good
----------------------------------------------
1. improvement stuff

Raymond Hettinger pointed out that the docs for the atexit module state 
that sys.exitfunc was to be deprecated.  Well, atexit has been in the 
stdlib since 2.0 so the deprecation is long overdue.  Looks like it will 
get its deprecation finally.

Contributing threads:
   - `Deprecate sys.exitfunc? 
<http://mail.python.org/pipermail/python-dev/2004-August/047842.html>`__


----------------------------------------------------------------------------------------------
My Personal Hell: decorators and the people who suggest new syntax for 
them...  next, on Oprah
----------------------------------------------------------------------------------------------
language evolution

What led to a record-setting flurry of emails to python-dev was set up 
when Guido gave the green light to checking in code to implement 
decorators using the '@' syntax (now known at the pie syntax thanks to 
its checkin coming very shortly after the OSCON Pie-thon competition and 
'@' sort of looking like a pie).  People didn't like it.  People were 
screaming that there had to be a better syntax than just settling for 
the least offensive one.  Others started to question whether decorators 
were really needed.  Others wanted to extend them even more and what 
their role truly was.  Either way this was all coming a little late.

But then Guido decided to make my life difficult by saying that if the 
community could come up with an agreed-upon alternative syntax to 
propose to him he would consider ripping out the '@' syntax; decorators 
have always been experimental and '@' was checked in so people had 
*something* to play with.  This meant everyone and their mother started 
to propose both new and old syntaxes for decorators.  This led to a 
record amount of email on python-dev (at least compared to what we have 
archives for; back to April 1999).

In order of my own personal sanity I am not going to cover any specific 
syntax.  There are just too many and I actually like the '@' syntax 
(just *try* using it; it grows on you quickly).  Instead this summary 
will cover any important design considerations for decorators (it should 
all be in the PEP but so much happened there is no way it all got in 
there) along with any other important considerations about them.  I will 
discuss anything specific to the '@' syntax since it is the currently 
winning syntax.

With that said, `PEP 318`_ and the PythonDecorators_ wiki page are the 
de-facto place for info on this whole situation.  I am probably going to 
be more biased than normal in this summary just out of time restraints 
to get this done by not over-analyzing people's suggestions and just 
assuming what I summarize here is not covered elsewhere; checking if it 
is would take too long.

First off, going with a syntax just so you could add backward-compatible 
support was not considered worth it.  It's a new feature so there is no 
need to hobble it at the get-go just so people who don't want to upgrade 
can still use it.  Plus the implementations to make this even possible 
were playing some tricks with the interpreter and not considered a good 
thing.  Besides you can, always will be able to, still use the way to do 
it in 2.2 and beyond.

Another design point that needed to be taken into consideration was 
ambiguity.  There could not be a question about what is a decorator and 
what isn't (list-before-def is an example of a syntax that can be 
ambiguous).

Syntactic support that allowed for possible future compiler 
optimizations was also considered important.  The on-going hope is to 
eventually get a JIT compiler for Python and if the decorators are not 
placed before the def somehow you make it much harder to optimize since 
you learn about the decorators after the fact of the function starting 
to be defined.

An argument against '@' was that people thought it got rather unwieldy 
quickly (and I am sure the new decision that each decorator must be on 
its own line is not making these objectors happy).  But one thing that 
must be remembered is that chances are most functions will not need more 
than one decorator if the even need one.  Looking through the stdlib 
sans the test suite you will notice there are not a lot of uses for any 
of the built-in decorators.  Yes, usage will probably grow with 
syntactic support for decorators, but not exponentially in terms of 
applying multiple decorators to a single function.

Allowing decorators to take arguments is a requirement.  Not being able 
to pass arguments to a decorator at instantiation time would severely 
cripple their usefulness.

Being prefix instead of postfix was considered important.  Knowing about 
what is coming up was something Guido thought is better than finding out 
about some special tweak to the use of a function based on a decorator 
listed after the definition line.  And in the method body had been 
completely ruled out.

In terms of implementation details, decorators can only be functions 
directly (and not lambda).  This means no ``@a or b`` tricks and such. 
It also means ``@foo().bar()`` is not allowed.  This is all gut feeling 
from Guido for why the restriction should be there.  It could change in 
the future since it is easier to loosen restrictions that make something 
more strict.

A problem with '@' in terms of portability is that Leo_ and IPython_ 
both use '@' for special syntax.  The authors of both tools seem willing 
to change their tools (not necessarily overly joyous, though  =) .

Guido wanted a syntax "that ... [is] easy to remember once explained". 
It did not need to be miraculously obvious what it did just by looking 
at the first time.

Some people didn't like '@' because they keep wanting to say "at" when 
they saw it.  To that, I give you Tim Peters' fix for this: "Pick the 
keyword you'd like.  Then whenever you see "@", pronounce
that word instead <wink>".

A single decorator that applied to multiple code blocks is completely 
out.  Guido gave a couple of reasons why, but one that he had mentioned 
multiple times is indenting the code block again just for the decorator 
is out.

Some people suggested that things be kept simple by just throwing out 
the entire idea of syntactic support.  Could happen if everyone agreed 
to that, but I wouldn't count on it.

A supposed knock against '@' was that it wasn't Pythonic.  Well, it is 
Pythonic if Guido says it is so not really a valid argument.


OK, tirade time.  If you don't want to listen to me get up on my 
soapbox, then just skip the rest of this...

"In the old days, Guido would Pronounce, and we'd all bite our tongues 
(although not necessarily each his own).  The less time Guido can make 
for Python, the more important becomes graceful capitulation."  Tim said 
this and it makes me wish for the old days.  People had *months* to 
comment on decorators and no one spoke up until something went into the 
language.  Procrastination is not a virtue when it comes to major 
language evolution discussions.  What was worse was when the emails 
started repeating themselves (which was pretty much from the get-go when 
this exploded).  Seemed like people decided to start talking without 
doing some research.  Granted the PEP was outdated and the wiki page was 
not up yet, but this stuff was covered in the Summaries before and you 
could have just Googled for the previous threads.

Personally, if I was Guido, I would have said that the community had 
their chance to speak up and they just didn't take it.  But Guido is a 
nicer guy than that so people are getting a second chance with this. 
Personally this came off a great case of the tyranny of the majority in 
my eyes.  There is a reason why Python is a dictatorship.

At this point people should be hashing out which syntax alternative they 
want to present to Guido on comp.lang.python_.  No more talking on 
python-dev, no more syntax proposals.  The community should set a goal 
date (Sept. 1 seems good) and just choose a bloody alternative.  Then 
when Guido makes his choice people accept it or just go to another 
language.  No one had better voice their disappoint once Guido chooses 
his syntax or I will personally come beat you over with a stick for 
being a whiner.

OK, that's out of my system.  I feel better now.

.. _PEP 318: http://www.python.org./peps/pep-0318.html
.. _PythonDecorators: 
http://www.python.org/cgi-bin/moinmoin/PythonDecorators
.. _Leo: http://leo.sourceforge.net/
.. _IPython: http://ipython.scipy.org/

Contributing threads (sans emails that were warnocked):
   - `2.4a2, and @decorators 
<http://mail.python.org/pipermail/python-dev/2004-August/046599.html>`__
   - `Plea for simpler decorator syntax,	in addition to pie-shaped 
syntax 
<http://mail.python.org/pipermail/python-dev/2004-August/046798.html>`__
   - `Call for defense of @decorators 
<http://mail.python.org/pipermail/python-dev/2004-August/046878.html>`__
   - `@decorators, the PEP and the "options" out there? 
<http://mail.python.org/pipermail/python-dev/2004-August/046905.html>`__
   - `About pep 318--Intro 
<http://mail.python.org/pipermail/python-dev/2004-August/046918.html>`__
   - `Questions about '@' in pep 318 
<http://mail.python.org/pipermail/python-dev/2004-August/046934.html>`__
   - `A usability argument for list-after-def 
<http://mail.python.org/pipermail/python-dev/2004-August/046935.html>`__
   - `The impact of '@' on Leo 
<http://mail.python.org/pipermail/python-dev/2004-August/046937.html>`__
   - `Similar syntax 
<http://mail.python.org/pipermail/python-dev/2004-August/046969.html>`__
   - `def fn (args) [dec,dec]: 
<http://mail.python.org/pipermail/python-dev/2004-August/047112.html>`__
   - `pep 318, Decorators for Functions, Methods and Classes 
<http://mail.python.org/pipermail/python-dev/2004-August/047132.html>`__
   - `Density of pie-decorator syntax 
<http://mail.python.org/pipermail/python-dev/2004-August/047154.html>`__
   - `elements of decorator syntax suggestions 
<http://mail.python.org/pipermail/python-dev/2004-August/047185.html>`__
   - `318: Annotation background 
<http://mail.python.org/pipermail/python-dev/2004-August/047218.html>`__
   - `IPython, @, and option E from the wiki 
<http://mail.python.org/pipermail/python-dev/2004-August/047223.html>`__
   - `Decorators: vertical bar syntax 
<http://mail.python.org/pipermail/python-dev/2004-August/047231.html>`__
   - `Suggesting '.' decorators (PEP318) 
<http://mail.python.org/pipermail/python-dev/2004-August/047234.html>`__
   - `Attribute strings? 
<http://mail.python.org/pipermail/python-dev/2004-August/047362.html>`__
   - `request: add keywords for static and class methods only 
<http://mail.python.org/pipermail/python-dev/2004-August/047498.html>`__
   - `@decorator syntax is sugar, but for what exactly? 
<http://mail.python.org/pipermail/python-dev/2004-August/047500.html>`__
   - `Semantics of decorators? 
<http://mail.python.org/pipermail/python-dev/2004-August/047623.html>`__
   - `A decorator syntax not yet mentioned (I think!) 
<http://mail.python.org/pipermail/python-dev/2004-August/047625.html>`__
   - `Another approach to decorators. 
<http://mail.python.org/pipermail/python-dev/2004-August/047647.html>`__
   - `Decorators after 'def' should be reconsidered 
<http://mail.python.org/pipermail/python-dev/2004-August/047637.html>`__
   - `def ... decorate 
<http://mail.python.org/pipermail/python-dev/2004-August/047781.html>`__
   - `Decorator syntax J2 (decorate..def) 
<http://mail.python.org/pipermail/python-dev/2004-August/047807.html>`__
   - `Decorators and docstrings 
<http://mail.python.org/pipermail/python-dev/2004-August/047827.html>`__
   - `block-based decorators and other block issues 
<http://mail.python.org/pipermail/python-dev/2004-August/047839.html>`__
   - `More concerns about decorators 
<http://mail.python.org/pipermail/python-dev/2004-August/047863.html>`__
From theller at python.net  Thu Aug 19 16:18:54 2004
From: theller at python.net (Thomas Heller)
Date: Thu Aug 19 16:18:59 2004
Subject: [Python-Dev] refcounts.dat
Message-ID: <y8kbp5wx.fsf@python.net>

Is the Doc/api/refcounts.dat file no longer maintained?  The last commit
was on 2002/10/04.  How does the C api manual the refcount info?

Thanks,

Thomas

From jim at zope.com  Thu Aug 19 16:22:13 2004
From: jim at zope.com (Jim Fulton)
Date: Thu Aug 19 16:22:16 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408190558.i7J5wPP19708@guido.python.org>
References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz>
	<200408190558.i7J5wPP19708@guido.python.org>
Message-ID: <4124B795.8000809@zope.com>

Guido van Rossum wrote:
>>From the posts I've seen here, the only alternatives that have
>>considerable popular support are ones that you've already
>>rejected. So I suspect nobody really feels it's worth trying.
> 
> 
> Well, do people generally buy those rejections, or is their consensus
> that I'm mistaken?

As I've noted before (sorry for repeating myself), I think the controversy
around this issue is evidence of a problem.  If you know that the decorator
idea is the right thing to do and you are comfortable with a syntax, then I
think you should go forward, and we'll all (speaking for myself :) follow
along.  OTOH, if you are just tired of talking about it and want to move
on, and don't know for sure that Python needs this, then I encourage you
to move on without decorators for now.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From skip at pobox.com  Thu Aug 19 16:35:59 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 19 16:36:23 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <200408190529.i7J5TZF7010126@h006008a7bda6.ne.client2.attbi.com>
References: <200408190529.i7J5TZF7010126@h006008a7bda6.ne.client2.attbi.com>
Message-ID: <16676.47823.382278.737098@montanaro.dyndns.org>


    Kurt> Patch / Bug Summary
    Kurt> ___________________

    Kurt> Patches :  264 open (-20) /  2556 closed (+42) /  2820 total (+22)
    Kurt> Bugs    :  745 open ( +1) /  4384 closed (+23) /  5129 total (+24)
    Kurt> RFE     :  149 open ( +0) /   130 closed ( +0) /   279 total ( +0)

    Kurt> New / Reopened Patches
    Kurt> ______________________

    ...

Kurt (et al),

Any thoughts about posting this to c.l.py as well as python-dev?  Maybe it
would encourage some more participation...

Skip
From nas at arctrix.com  Thu Aug 19 16:40:19 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu Aug 19 16:40:23 2004
Subject: [Python-Dev] Byte string class hierarchy
In-Reply-To: <41247122.7030000@egenix.com>
References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>
	<20040818223629.GA17550@mems-exchange.org>
	<1092868762.22400.97.camel@geddy.wooz.org>
	<41247122.7030000@egenix.com>
Message-ID: <20040819144019.GA19851@mems-exchange.org>

On Thu, Aug 19, 2004 at 11:21:38AM +0200, M.-A. Lemburg wrote:
> I assume the picture will look more like this:
> 
> basesequence
>    mutable
>       basebytes
> 	   bytes
> 	   array
> 	   cStringIO
> 	   mmap
>    immutable
>       unicode
>       tuple
> basenumber
>    integer
>    float
>    decimal
>    complex

Inheritance is overrated. :-)  For example, I think mutability may
be an orthogonal property.  Also, I don't think cStringIO is
sequence-like, it's more like a file.

  Neil
From mal at egenix.com  Thu Aug 19 16:49:27 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 16:49:31 2004
Subject: [Python-Dev] Byte string class hierarchy
In-Reply-To: <20040819144019.GA19851@mems-exchange.org>
References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>	<20040818223629.GA17550@mems-exchange.org>	<1092868762.22400.97.camel@geddy.wooz.org>	<41247122.7030000@egenix.com>
	<20040819144019.GA19851@mems-exchange.org>
Message-ID: <4124BDF7.8050508@egenix.com>

Neil Schemenauer wrote:
> On Thu, Aug 19, 2004 at 11:21:38AM +0200, M.-A. Lemburg wrote:
> 
>>I assume the picture will look more like this:
>>
>>basesequence
>>   mutable
>>      basebytes
>>	   bytes
>>	   array
>>	   cStringIO
>>	   mmap
>>   immutable
>>      unicode
>>      tuple
>>basenumber
>>   integer
>>   float
>>   decimal
>>   complex
> 
> 
> Inheritance is overrated. :-)  For example, I think mutability may
> be an orthogonal property.  Also, I don't think cStringIO is
> sequence-like, it's more like a file.

This was just a rough sketch.

I agree that inheritance is overrated, but in absence of well-defined
interfaces, it often helps :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mwh at python.net  Thu Aug 19 16:52:31 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug 19 16:52:33 2004
Subject: [Python-Dev] Byte string class hierarchy
In-Reply-To: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl> (Jack Jansen's
	message of "Thu, 19 Aug 2004 00:16:33 +0200")
References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>
Message-ID: <2macwr9o40.fsf@starship.python.net>

Jack Jansen <Jack.Jansen@cwi.nl> writes:

> I may have missed a crucial bit of the discussion, having been away,
> so if this is completely besides the point let me know. But my feeling
> is that the crucial bit is the type inheritance graph of all the byte
> and string types. And I wonder whether the following graph would help
> us solve most problems (aside from introducing one new one, that may
> be a showstopper):
>
> genericbytes
> 	mutablebytes
> 	bytes
> 		genericstring
> 			string
> 			unicode

Um, I was thinking the hierarchy would look like this:

object
    unicode
    bytes

Cheers,
mwh

-- 
  I saw `cout' being shifted "Hello world" times to the left and
  stopped right there.                                -- Steve Gonedes
From mwh at python.net  Thu Aug 19 17:14:35 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug 19 17:14:37 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com>
	(Timothy
	C. Delaney's message of "Thu, 19 Aug 2004 14:37:45 +1000")
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com>
Message-ID: <2mwtzv88is.fsf@starship.python.net>

"Delaney, Timothy C (Timothy)" <tdelaney@avaya.com> writes:

> Guido van Rossum wrote:
>
>> Is anybody seriously trying to come up with a single alternative
>> decorator proposal that most folks "out there" can support, to be
>> presented to me (with implementation, please!) in time for 2.4b1?
>
> I'm afraid that absolutely zero consensus is emerging :( I've withdrawn
> myself entirely from the discussions as they've proven completely
> useless.

Me too.

Cheers,
mwh

-- 
  The PROPER way to handle HTML postings is to cancel the article,
  then hire a hitman to kill the poster, his wife and kids, and fuck
  his dog and smash his computer into little bits. Anything more is
  just extremism.                                 -- Paul Tomblin, asr
From barry at python.org  Thu Aug 19 17:15:47 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Aug 19 17:15:48 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <412480FF.2030203@zope.com>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>
	<412480FF.2030203@zope.com>
Message-ID: <1092928547.8719.70.camel@geddy.wooz.org>

On Thu, 2004-08-19 at 06:29, Jim Fulton wrote:

> IMO, yes, although I'd happily accept both syntaxes. I can live with
> @foo to server the people who want it, but it bugs me that I have to
> write:
> 
>    @clasmethod
>    def foo ....
> 
> rather than the, to me, much more obvious and readable:
> 
>    def classmethod foo ...

This one doesn't really bother me, in fact, I've actually come to like
the decorator in the former position, rather than the latter.  I've
rewritten my decorator code to use almost all of the proposed syntaxes,
just to see how they look or what the annoyance factor is.

I'm pretty comfortable with pie-decorators before def syntax.  I think
they're a good compromise, and are quite readable /and/ writable.  I
don't like pie-decorator-in-first-line syntax as much for the same
reason I don't like the former choice above: both make the decorator
less prominent in the definition, which isn't something I think you
want.  The latter choice above is worse because it also hides the
function name.

I think it's important to understand that we're now down to aesthetic
choices about elegance, readability, ease of use, etc.  For those
reasons, we'll never agree because everyone has different tastes. 
Python represents Guido's aesthetic in language design, and we all agree
that he gets most such decisions right, so I trust him on this one too
-- and I was /not/ a pie-decorator fan originally.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/53c8d5e4/attachment.pgp
From barry at python.org  Thu Aug 19 17:23:22 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Aug 19 17:22:56 2004
Subject: [Python-Dev] Byte string class hierarchy
In-Reply-To: <2macwr9o40.fsf@starship.python.net>
References: <433429D2-F164-11D8-9E0D-000D934FF6B4@cwi.nl>
	<2macwr9o40.fsf@starship.python.net>
Message-ID: <1092929002.8719.72.camel@geddy.wooz.org>

On Thu, 2004-08-19 at 10:52, Michael Hudson wrote:

> Um, I was thinking the hierarchy would look like this:
> 
> object
>     unicode
>     bytes

Flat is better than nested.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/fb1c32eb/attachment.pgp
From amk at amk.ca  Thu Aug 19 17:38:12 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Thu Aug 19 17:39:06 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <2mwtzv88is.fsf@starship.python.net>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com>
	<2mwtzv88is.fsf@starship.python.net>
Message-ID: <20040819153812.GA21213@rogue.amk.ca>

On Thu, Aug 19, 2004 at 04:14:35PM +0100, Michael Hudson wrote:
> "Delaney, Timothy C (Timothy)" <tdelaney@avaya.com> writes:
> > I'm afraid that absolutely zero consensus is emerging :( I've withdrawn
> > myself entirely from the discussions as they've proven completely
> > useless.
> Me too.

Me three.

(I don't think I'm going to become a heavy user of decorators --
little of my current code could use them -- but for what it's worth
I'm happy with either @decorator or list-after-def forms.  I don't
hate the '@' character, but neither do I find list-after-def very
confusing, especially if you format the code clearly.)

--amk

From aahz at pythoncraft.com  Thu Aug 19 17:41:32 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Aug 19 17:41:36 2004
Subject: [Python-Dev] Weird errors when adding a new built-in type
In-Reply-To: <41247180.5070304@auckland.ac.nz>
References: <41247180.5070304@auckland.ac.nz>
Message-ID: <20040819154132.GC21234@panix.com>

On Thu, Aug 19, 2004, Geoffrey Biggs wrote:
>
> I'm trying to add a new built-in number data type to Python with its own 
> syntax, so I'm working directly with the interpreter rather than 
> creating my own extension module (side note: I've appended something 
> extra to the version thing in the Makefile - I doubt this is relevant to 
> the problem but it's probably best you have all the info). The complex 
> data type is similar to what I'm trying to do so I've been following 
> that as an example. I've successfully extended the tokenizer and the 
> parsenumber() function in compile.c to do what I want and written the 
> data type in my two new files Objects/mynewobject.c and 
> Include/mynewobject.h. I've included mynewobject.h in Python.h and added 
> the two files to the Makefile.
> However, when I tried to run the code by typing in the syntax to produce 
> my data type, Python suffers a segmentation fault. 

Unless you're planning to attempt pushing this into the core at some
point in the future, you're best off asking on comp.lang.python;
although some of the core developers no longer have time for c.l.py,
they're also not likely to answer your questions here, either.  ;-)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From walter at livinglogic.de  Thu Aug 19 17:45:26 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 19 17:45:34 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123C16D.4060505@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de>
	<4123B6F3.6060708@egenix.com> <4123BD93.2040107@livinglogic.de>
	<4123C16D.4060505@v.loewis.de>
Message-ID: <4124CB16.90705@livinglogic.de>

Martin v. L?wis wrote:

> Walter D?rwald wrote:
> 
>> They will not, because StreamReader.decode() already is a feed
>> style API (but with state amnesia).
>>
>> Any stream decoder that I can think of can be (and most are)
>> implemented by overwriting decode().
> 
> I consider that an unfortunate implementation artefact. You
> either use the stateless encode/decode that you get from
> codecs.get(encoder/decoder) or you use the file API on
> the streams. You never ever use encode/decode on streams.

That is exactly the problem with the current API.
StreamReader mixes two concepts:

1) The stateful API, which allows decoding a byte input
    in chunk and the state of the decoder is kept between
    calls.
2) A file API where the chunks to be decoded are read
    from a byte stream.

> I would have preferred if the default .write implementation
> would have called self._internal_encode, and the Writer
> would *contain* a Codec, rather than inheriting from Codec.

This would separate the two concepts from above.

> Alas, for (I guess) simplicity, a more direct (and more
> confusing) approach was taken.
> 
>> 1) Having feed() as part of the StreamReader API:
>> ---
>> s = u"???".encode("utf-8")
>> r = codecs.getreader("utf-8")()
>> for c in s:
>>    print r.feed(c)
> 
> 
> Isn't that a totally unrelated issue? Aren't we talking about
> short reads on sockets etc?

We're talking about two problems:

1) The current implementation does not really support the
    stateful API, because trailing incomplete byte sequences
    lead to errors.
2) The current file API is not really convenient for decoding
    when the input is not read for a stream.

> I would very much prefer to solve one problem at a time.

Bye,
    Walter D?rwald


From anthony at interlink.com.au  Thu Aug 19 17:50:49 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Aug 19 17:51:01 2004
Subject: [Python-Dev] 2.4a3 release is September 2, SF tracker keywords
Message-ID: <4124CC59.80007@interlink.com.au>

After consultations with the relevant folks, I want to
get 2.4a3 out the door on September 2nd. I'll be following
up to this email with a list of things that are still
pending. This _should_ be the final alpha release. After
that, the first beta.

I'm wondering if there's a way I can mark bugs
and patches that should be in before b1. It appears
from SF's tracker that the only way I'm going to be
able to do that is by putting keywords in the summary.
I'm considering doing this. Rather than rip off the
entirety of the mozilla's insane list of keywords,
here's the ones I care about:

   - should be in before b1 (changes functionality)
   - should be in before final
   - regression since 2.3

Does anyone else have any others that they think are
useful? I'd probably go with something like [beta],
[final], [regr] in the summary. The other alternative
is to make a bunch of new groups - this doesn't excite
me at all, because a) I can't do it, and b) we can't
get rid of them once they're in.


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From aahz at pythoncraft.com  Thu Aug 19 17:55:07 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Aug 19 17:55:16 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408191158.12738.gmccaughan@synaptics-uk.com>
References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz>
	<200408190558.i7J5wPP19708@guido.python.org>
	<200408191158.12738.gmccaughan@synaptics-uk.com>
Message-ID: <20040819155507.GE21234@panix.com>

On Thu, Aug 19, 2004, Gareth McCaughan wrote:
> On Thursday 2004-08-19 06:58, Guido van Rossum wrote:
>>
>>> From the posts I've seen here, the only alternatives that have
>>> considerable popular support are ones that you've already
>>> rejected. So I suspect nobody really feels it's worth trying.
>> 
>> Well, do people generally buy those rejections, or is their consensus
>> that I'm mistaken?
> 
> I'm one of the people who doesn't really feel it's worth trying.
> *My* consensus :-) is that you're mistaken. I think that
> 
>   - the @-syntax is horrible and unpythonic as all-get-out,
>     but usable;
> 
>   - your arguments against the []-before-colon form are utterly
>     bogus;

Aside from changing "utterly bogus" to "excessive", I'm entirely +1 on
everything Gareth wrote in his post.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From mal at egenix.com  Thu Aug 19 18:06:46 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 18:06:49 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4124CB16.90705@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>
	<4123B202.5000200@livinglogic.de>	<4123B6F3.6060708@egenix.com>
	<4123BD93.2040107@livinglogic.de>	<4123C16D.4060505@v.loewis.de>
	<4124CB16.90705@livinglogic.de>
Message-ID: <4124D016.1050600@egenix.com>

Walter D?rwald wrote:
> Martin v. L?wis wrote:
> 
>> Walter D?rwald wrote:
>>
>>> They will not, because StreamReader.decode() already is a feed
>>> style API (but with state amnesia).
>>>
>>> Any stream decoder that I can think of can be (and most are)
>>> implemented by overwriting decode().
>>
>>
>> I consider that an unfortunate implementation artefact. You
>> either use the stateless encode/decode that you get from
>> codecs.get(encoder/decoder) or you use the file API on
>> the streams. You never ever use encode/decode on streams.
> 
> 
> That is exactly the problem with the current API.
> StreamReader mixes two concepts:
> 
> 1) The stateful API, which allows decoding a byte input
>    in chunk and the state of the decoder is kept between
>    calls.
> 2) A file API where the chunks to be decoded are read
>    from a byte stream.
> 
>> I would have preferred if the default .write implementation
>> would have called self._internal_encode, and the Writer
>> would *contain* a Codec, rather than inheriting from Codec.
> 
> This would separate the two concepts from above.

Note that StreamCodec only inherits from Codec for
convenience reasons (you can define a StreamCodec
using the stateless .encode() and .decode() methods
you get from Codec) and for logical reasons: a StreamCodec
happens to be a  Codec as well, so isinstance(obj, Codec) should
be true for a StreamCodec as well.

There's nothing preventing you from overriding .encode()
and .decode() in a StreamReader or adding new methods that
implement a different approach to encode and decode.

Users should always use the file API of StreamReader
et al., not the .encode() and .decode() methods.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mzarate at uoguelph.ca  Thu Aug 19 18:13:11 2004
From: mzarate at uoguelph.ca (Martin Zarate)
Date: Thu Aug 19 18:13:16 2004
Subject: [Python-Dev] Considering decorator syntax on an individual feature
	basis
Message-ID: <1092931991.4124d1976b8ba@webmail.uoguelph.ca>

Issues with decorator syntax.

I think we can all agree that the decorator argument is now wholly a matter of
aesthetic taste, with the brief exception of namespace-based stuff that got
argued when I suggested to use a block instead.

The problem is that we keep jumping from the beginning of the argument to the
end - we keep saying "what syntax do we want" rather than "which syntactic
features do we want".

The features in question, which should be decided individually rather than as a
whole:

- decorator before/within/after def.
Guido is obviously a fan of the "before" approach, and I agree.  The "after" to
me is just ugly, and the "within" (on the same line, I mean) is impractical for
some of the lengthier uses people describe for decorators, although I think that
it is the most "pythonic" in general appearance.

- special character/keyword/no new symbol.
Guido prefers the @ character (I dislike it - it looks meaningless, and it
breaks existing tools - I find it unpythonic).  The "no new symbol" approach
(such as the "list before def", or decorator-after-def-before-name) is often
difficult to parse and, while elegant looking, can make it visually difficult
to distinguish the decorators (and hard to grep).  As such, I prefer the new
keyword statement approach - it is more verbose, but I like it.

- decorator on one/many line(s)
The approach described by the paired-block syntax, and the @ syntax both assume
that users want to include only one decorator per line.  Personally, I am
inclined to follow something like the new import statment wherein the user can
have a comma-delimited list of decorators and can span the list across lines if
they so choose.

- one symbol per decorator vs. one symbol for all decorators
The @ approach has one symbol for each decorator.  The block approach has one
symbol that encompasses all decorators.  If a single symbol is used, then one
must consider whether that symbol occurs on the same line as the decorators or
not.  If not, then the symbol must be brief to avoid letting its repetition be
clumsy.  Personally, I prefer a single use of the symbol, but its really
personal taste.

- indent def/not indent def
If the decorator is before the def, this must be considered.  Indenting the def
is very Pythonic, as it shows the heirarchial relationship visually... but it
also seems an unnecessary indendation in this case, as there is no other case
where an indentation occurs for the benefit of a single statement.

I think if each of these issues could be resolved and agreed to individually,
then a consensis would be far more apparent as to the ideal solution, or at
least some of the myriad solutions could be filtered out.

I think the Python community has mostly settled on a "before def" approach, and
I think that most agree that the @ character is very ugly, but some are coming
around (mostly out of exhaustion though, it sounds).  So if we can agree on the
latter considerations.  Personally, my favourite would be a single-line but
continuable statement (like the "import" statement) which causes the assignment
that occurs on the following line be wrapped by its mentioned objects.  Use a
nice keyword like deco or remake (or even just "be").  Possibly indent the def
one level to show the relationship of the import statement and to allow for def
assignments to look good when used with the statement.

The fact is that there is no Pythonic precedence here - we're talking about a
statement that holds one other statement and has N parameters.  Whenever Python
has had to break into new ground with statements, its been a bit of a mess (take
lambda for example).

Oh, and in defense of "be" as a statement - how's this?

be staticmethod, returning(int), taking(int, bool, bool)
def foo:
    pass

or

be (
    staticmethod,
    synchronised):
foo = lambda: none

but that's just my tastes.  Oh well, I'm violating my own rule, jumping ahead
with a full syntax when we can't even find consensus on the parts of it.
From goodger at python.org  Thu Aug 19 18:44:20 2004
From: goodger at python.org (David Goodger)
Date: Thu Aug 19 18:44:25 2004
Subject: [Python-Dev] Re: 2.4a3 release is September 2, SF tracker keywords
In-Reply-To: <4124CC59.80007@interlink.com.au>
References: <4124CC59.80007@interlink.com.au>
Message-ID: <4124D8E4.8030705@python.org>

[Anthony Baxter]
>   - regression since 2.3

What does that one mean?

-- 
David Goodger <http://python.net/~goodger>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/965fcb4b/signature.pgp
From neal at metaslash.com  Thu Aug 19 18:47:34 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Thu Aug 19 18:47:52 2004
Subject: [Python-Dev] 2.4a3 release is September 2, SF tracker keywords
In-Reply-To: <4124CC59.80007@interlink.com.au>
References: <4124CC59.80007@interlink.com.au>
Message-ID: <20040819164734.GY19366@epoch.metaslash.com>

On Fri, Aug 20, 2004 at 01:50:49AM +1000, Anthony Baxter wrote:
> 
> I'm wondering if there's a way I can mark bugs
> and patches that should be in before b1. 
> 
> Does anyone else have any others that they think are
> useful? I'd probably go with something like [beta],
> [final], [regr] in the summary. The other alternative
> is to make a bunch of new groups - this doesn't excite
> me at all, because a) I can't do it, and b) we can't
> get rid of them once they're in.

There are two more possible hacks: 1) use priorities.
This probably isn't good enough.  2) create SF users
that aren't real people, but used for assigning levels.

E.g., py-beta, py-final, py-regr.

You would still have to get an admin to add these, but
you could create the accounts yourself and they aren't
necessarily permanent.  I don't know how well this would work.
It's definitely a hack. :-)

Neal
From tim.peters at gmail.com  Thu Aug 19 18:48:08 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 19 18:48:11 2004
Subject: [Python-Dev] 2.4a3 release is September 2, SF tracker keywords
In-Reply-To: <4124CC59.80007@interlink.com.au>
References: <4124CC59.80007@interlink.com.au>
Message-ID: <1f7befae040819094825bfacf0@mail.gmail.com>

[Anthony Baxter]
> After consultations with the relevant folks, I want to
> get 2.4a3 out the door on September 2nd.

+1 on both parts.  In particular, changes since a2 have been too rapid
and broad to be comfortable with calling the next one a beta.

> ...
> I'm wondering if there's a way I can mark bugs
> and patches that should be in before b1. It appears
> from SF's tracker that the only way I'm going to be
> able to do that is by putting keywords in the summary.
> I'm considering doing this.

Changing priorities has worked well for this in the past.  SF even
colors them differently

I'm going to add one for you:

     - must be resolved for a3

Priority 9.  It gets addressed or a3 doesn't ship.

>   - should be in before b1 (changes functionality)

Priority 8.

>   - should be in before final

Priority 7.

>   - regression since 2.3

Depends on whether the specific regression must be resolved for a3,
b1, or final -- or is going to be declared "a feature" and 2.3's
behavior "a bug".

> Does anyone else have any others that they think are
> useful? I'd probably go with something like [beta],
> [final], [regr] in the summary.

Over time, those tags become misleading, because they're so easy to
ignore.  People won't allow high priorities to remain high, though. 
It irritates them.  That's good <wink>.

> The other alternative is to make a bunch of new groups - this
> doesn't excite me at all, because a) I can't do it, and b) we can't
> get rid of them once they're in.

Yup, new groups suck.
From walter at livinglogic.de  Thu Aug 19 18:49:44 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 19 18:49:50 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4123D0C2.4080505@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de> <4123D0C2.4080505@v.loewis.de>
Message-ID: <4124DA28.50106@livinglogic.de>

Martin v. L?wis wrote:

> Walter D?rwald wrote:
> 
>> But then a file that contains the two bytes 0x61, 0xc3
>> will never generate an error when read via an UTF-8 reader.
>> The trailing 0xc3 will just be ignored.
>>
>> Another option we have would be to add a final() method
>> to the StreamReader, that checks if all bytes have been
>> consumed. 
> 
> Alternatively, we could add a .buffer() method that returns
> any data that are still pending (either a Unicode string or
> a byte string).

Both approaches have one problem: Error handling won't
work for them. If the error handling is "replace", the decoder
should return U+FFFD for the final trailing incomplete sequence
in read(). This won't happen when read() never reads those
bytes from the input stream.

>> Maybe this should be done by StreamReader.close()?
> 
> No. There is nothing wrong with only reading a part of a file.

Yes, but if read() is called without arguments then everything
from the input stream should be read and used.

>> Now
>> inShift counts the number of characters (and the shortcut
>> for a "+-" sequence appearing together has been removed.
> 
> Ok. I didn't actually check the correctness of the individual
> methods.
> 
> OTOH, I think time spent on UTF-7 is wasted, anyway.

;) But it's a good example of how complicated state
management can get.

>> Would a version of the patch without a final argument but
>> with a feed() method be accepted?
> 
> I don't see the need for a feed method. .read() should just
> block until data are available, and that's it.

There are situations where this can never work: Take a look
at xml.sax.xmlreader.IncrementalParser. This interface
has a feed() method which the user can call multiple times
to pass byte string chunks to the XML parser. These chunks
have to be decoded by the parser. Now if the parser wants
to use Python's StreamReader it has to wrap the bytes passed
to the feed() method into a stream interface. This looks
something like the Queue class from the patch:

class Queue(object):
     def __init__(self):
         self._buffer = ""

     def write(self, chars):
         self._buffer += chars

     def read(self, size=-1):
         if size<0:
             s = self._buffer
             self._buffer = ""
             return s
         else:
             s = self._buffer[:size]
             self._buffer = self._buffer[size:]
             return s

The parser creates such an object and passes it to the
StreamReader constructor. Now when feed() is called for
the XML parser the parser calls queue.write(bytes) to
put the bytes into the queue. Now the parser can call
read() on the StreamReader (which in turn will read
from the queue (on the other end)) to get decoded data.

But this will fail when StreamReader.read() block
until more data is available. This will never happen,
because the data will be put in the queue explicitely
by calls to the feed() method of the XML parser.

Or take a look at sio.DecodingInputFilter. This should
be an alternative implementation of reading a stream
an decoding bytes to unicode. But the current implementation
is broken because it uses the stateless API. But once
we switch to the stateful API DecodingInputFilter becomes
useless: DecodingInputFilter.read() just looks like this:
def read():
    return self.stream.read()
(with stream being the stateful stream reader from
codecs.getreader()), because DecodingInputFilter is
forced to use the stream API of StreamReader.

>> I'm imagining implementing an XML parser that uses Python's
>> unicode machinery and supports the
>> xml.sax.xmlreader.IncrementalParser interface.
> 
> I think this is out of scope of this patch. The incremental
> parser could implement a regular .read on a StringIO file
> that also supports .feed.

This adds to much infrastructure, when the alternative
implementation is trivial. Take a look at the first
version of the patch. Implementing a feed() method just
mean factoring the lines:

data = self.bytebuffer + newdata
object, decodedbytes = self.decode(data, self.errors)
self.bytebuffer = data[decodedbytes:]

into a separate method named feed():

def feed(newdata):
    data = self.bytebuffer + newdata
    object, decodedbytes = self.decode(data, self.errors)
    self.bytebuffer = data[decodedbytes:]
    return object

So the feed functionality does already exist. It's just
not in a usable form.

A using StringIO wouldn't work because we need both
a read and a write position.

>> Without the feed method(), we need the following:
>>
>> 1) A StreamQueue class that
>
> Why is that? I thought we are talking about "Decoding
> incomplete unicode"?

Well, I had to choose a subject. ;)

Bye,
    Walter D?rwald


From mwh at python.net  Thu Aug 19 18:50:22 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Aug 19 18:50:39 2004
Subject: [Python-Dev] Re: 2.4a3 release is September 2, SF tracker keywords
In-Reply-To: <4124D8E4.8030705@python.org> (David Goodger's message of "Thu,
	19 Aug 2004 12:44:20 -0400")
References: <4124CC59.80007@interlink.com.au> <4124D8E4.8030705@python.org>
Message-ID: <2mk6vv8435.fsf@starship.python.net>

David Goodger <goodger@python.org> writes:

> [Anthony Baxter]
>>   - regression since 2.3
>
> What does that one mean?

I guess thing's like Tim's recent asyncore pain.

Cheers,
mwh

-- 
  The source passes lint without any complaint (if invoked with
  >/dev/null).    -- Daniel Fischer, http://www.ioccc.org/1998/df.hint
From pm_mon at yahoo.com  Thu Aug 19 13:45:09 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Thu Aug 19 19:00:55 2004
Subject: [Python-Dev] Re: PEP 318: Can't we all just get along?
In-Reply-To: <412480FF.2030203@zope.com>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>
	<412480FF.2030203@zope.com>
Message-ID: <cg23s3$vgd$2@sea.gmane.org>

Jim Fulton wrote:

> 
> IMO, yes, although I'd happily accept both syntaxes. I can live with
> @foo to server the people who want it, but it bugs me that I have to
> write:
> 
>   @clasmethod
>   def foo ....
> 
> rather than the, to me, much more obvious and readable:
> 
>   def classmethod foo ...
> 

Probably the best-loved features of Python are the shortcuts.  The fact 
that you don't have to explicitly declare very much and that the 
language supports clear, shorthand syntax for frequently used operations...

	x = [1,2,3]

	y = r[2:9]

	def __foo():

That's the good stuff of Python.

I believe that identifying staticmethods and classmethods is an 
operation that is (or will be) frequently used as compared to the other 
decorators.  So it would be great if there was an obvious, shorthand way 
for doing that as well.

Even better (IMO) would be a solution that didn't really add any new 
keywords to the language, but instead simply formalized widely-used 
coding conventions, as was done to denote code block boundaries.

The use of indentation to identify code blocks is a wonderful example of 
Python shorthand.  It works (some even call it 'genius') because it 
leverages a coding convention shared by a majority of programmers.

A week or so ago, Stefan Eischet on comp.lang.python suggested that the 
type (static|class|instance) of a method could be inferred through 
examination of its first parameter.  I agree with his assessment.

The vast majority of instance methods I've seen all use 'self' as the 
first parameter.  Likewise, most class methods use 'cls' or 'klass' as 
their first parameter.  If we exploit these conventions, we end up with 
a simple, clear, obvious mechanism for denoting (this aspect of) a 
method's type.

     class Foo(Object):
         def m1(self, a, b):    # this is an instance method of Foo
             pass

         def m2(cls, a, b):     # this is a class method of Foo
             pass

         def m3(a, b):          # this is a static method of Foo
             pass

A special Object (capital 'O') class could work this magic so that old 
code didn't break.

I know that this is odd.  But then so are most of the great things about 
Python.

Paul


From marktrussell at btopenworld.com  Thu Aug 19 19:15:26 2004
From: marktrussell at btopenworld.com (Mark Russell)
Date: Thu Aug 19 19:15:29 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <1092928547.8719.70.camel@geddy.wooz.org>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>
	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>
	<412480FF.2030203@zope.com> <1092928547.8719.70.camel@geddy.wooz.org>
Message-ID: <1092935726.1542.23.camel@localhost>

On Thu, 2004-08-19 at 16:15, Barry Warsaw wrote:
> Python represents Guido's aesthetic in language design, and we all agree
> that he gets most such decisions right, so I trust him on this one too
> -- and I was /not/ a pie-decorator fan originally.

I think it's a significant point in favour of pie syntax that it grows
on people quite quickly if they use it for a bit.  Has anyone has tried
it out for a few days and still hated it just as much at the end?

Mark

From dave at boost-consulting.com  Thu Aug 19 19:31:47 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Aug 19 19:32:53 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/dist/src/Python compile.c, 2.319, 2.320
References: <000f01c48590$866705e0$e841fea9@oemcomputer>
	<200408190407.i7J47Ar19512@guido.python.org>
Message-ID: <uu0uzowzg.fsf@boost-consulting.com>

Guido van Rossum <guido@python.org> writes:

>> > Hm, shouldn't the bytecode optimizer only be used when -O is used, and
>> > hence pyo files are being written?
>> 
>> Why?  That would throw away most of the benefits to most of the
>> users and gain nothing in return.  The peepholer was in place in for
>> Py2.3 and only benefits were seen.  I would save the -O option for
>> something where there is a trade-off (loss of docstrings, excessive
>> compilation time, possibly risky optimizations, or somesuch).  Here,
>> the peepholer is superfast and costs almost nothing.
>
> Maybe I'm out of tune, but I thought that optimizations should be
> turned off by default because most people don't need them and because
> of the risk that the optimizer might break something.  Haven't there
> been situations in Python where one optimization or another was found
> to be unsafe after having been in use (in a release!) for a long
> time?

Isn't that a good argument for having them turned on all the time?
The easiest way to ship code that suffers from an optimizer bug is to
do all your development and most of your testing with unoptimized
code.

In C/C++, there's a good reason to develop code unoptimized: it's much
easier to debug.  I'm not sure that applies to Python.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From walter at livinglogic.de  Thu Aug 19 19:34:33 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 19 19:34:38 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4124741B.3080605@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<4123B804.2000508@livinglogic.de> <4124741B.3080605@egenix.com>
Message-ID: <4124E4A9.7070204@livinglogic.de>

M.-A. Lemburg wrote:

> Walter D?rwald wrote:
> 
>> Martin v. L?wis wrote:
>>
>> [...]
>> We already have an efficient way to communicate incompleteness:
>> the decode method returns the number of decoded bytes.
>>
>> The questions remaining are
>>
>> 1) communicate to whom? IMHO the info should only be used
>>    internally by the StreamReader.
> 
> Handling incompleteness should be something for the codec
> to deal with.

Absolutely. This means that decode() should not be called
by the user. (But the implementation of read() (and feed(),
if we have it) calls it.)

> The queue doesn't have to know about it in an
> way. However, the queue should have interfaces allowing the
> codec to tell whether there are more bytes waiting to be
> processed.

This won't work when the byte stream wrapped by the
StreamReader is not a queue. (Or do you want the wrap the
byte stream in a queue? This would be three wrapping layers.)

And the information is not really useful, because it might
change (e.g. when the user puts additional data into the
queue/stream.)

>> 2) When is incompleteness OK? Incompleteness is of course
>>    not OK in the stateless API. For the stateful API,
>>    incompleteness has to be OK even when the input stream
>>    is (temporarily) exhausted, because otherwise a feed mode
>>    wouldn't work anyway. But then incompleteness is always OK,
>>    because the StreamReader can't distinguish a temporarily
>>    exhausted input stream from a permanently exhausted one.
>>    The only fix for this I can think of is the final argument.
> 
> A final argument may be the way to go. But it should be an
> argument for the .read() method (not only the .decode() method)
> since that's the method reading the data from the queue.

Yes. E.g. the low level charmap decode function doesn't
need the final argument, because there is zero state to
be kept between calls.

> I'd suggest that we extend the existing encode and decode
> codec APIs to take an extra state argument that holds the
> codec state in whatever format the codec needs (e.g. this
> could be a tuple or a special object):
> 
> encode(data, errors='strict', state=None)
> decode(data, errors='strict', state=None)

We don't need a specification for that. The stateless
API doesn't need an explicit state (the state is just
a bunch of variables at the C level) and for the
stateful API the state can be put into StreamReader
attributes. How this state looks is totally up to
the StreamReader itself (see the UTF-7 reader in the
patch for an example). If the stream reader passes
on this state to a low level decoding function
implemented in C, how this state info looks is again
totally up to the codec.

So I think we don't have to specify anything in this
area.

> In the case of the .read() method, decode() would be
> called. If the returned length_consumed does not match
> the length of the data input, the remaining items would
> have to be placed back onto the queue in non-final mode.
> In final mode an exception would be raised to signal
> the problem.

Yes, in non-final mode the bytes would have to be retained
and in final mode an exception is raised (except when
the error handling callback does something else). But I don't
think we should put a queue between the byte stream and
the StreamReader (at least not in the sense of a queue as
another file like object). The remaining items can be kept
in an attribute of the StreamReader instance, that's what
---
data = self.bytebuffer + newdata
object, decodedbytes = self.decode(data, self.errors)
self.bytebuffer = data[decodedbytes:]
---
does in the patch.

The first line combines the items retained from the
last call with those read from the stream (or passed
to the feed method).

The second line does semi-stateful decoding of those
bytes.

The third line puts the new remaining items back into
the buffer.

The decoding is "semi-stateful", because the info about
the remaining bytes is not stored by decode itself, but
by the caller of decode. feed() is the method that does
fully stateful decoding of byte chunks.

> I think it's PEP time for this new extension. If time
> permits I'll craft an initial version over the weekend.

I'm looking forward to the results.

Bye,
    Walter D?rwald


From bac at OCF.Berkeley.EDU  Thu Aug 19 19:33:00 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug 19 19:40:27 2004
Subject: [Python-Dev] 2.4a3 release is September 2, SF tracker keywords
In-Reply-To: <1f7befae040819094825bfacf0@mail.gmail.com>
References: <4124CC59.80007@interlink.com.au>
	<1f7befae040819094825bfacf0@mail.gmail.com>
Message-ID: <4124E44C.8010805@ocf.berkeley.edu>

Tim Peters wrote:

> [Anthony Baxter]
>>...
>>I'm wondering if there's a way I can mark bugs
>>and patches that should be in before b1. It appears
>>from SF's tracker that the only way I'm going to be
>>able to do that is by putting keywords in the summary.
>>I'm considering doing this.
> 
> 
> Changing priorities has worked well for this in the past.  SF even
> colors them differently
> 
> I'm going to add one for you:
> 
>      - must be resolved for a3
> 
> Priority 9.  It gets addressed or a3 doesn't ship.
> 
> 
>>  - should be in before b1 (changes functionality)
> 
> 
> Priority 8.
> 
> 
>>  - should be in before final
> 
> 
> Priority 7.
> 
> 
>>  - regression since 2.3
> 
> 
> Depends on whether the specific regression must be resolved for a3,
> b1, or final -- or is going to be declared "a feature" and 2.3's
> behavior "a bug".
> 

Priority approach seems good.  Can also easily sort by priority in the 
tracker.  While you can sort by the summary as well would have to make 
sure that the names are in a proper alphabetical order to make that useful.

> 
>>Does anyone else have any others that they think are
>>useful? I'd probably go with something like [beta],
>>[final], [regr] in the summary.
> 
> 
> Over time, those tags become misleading, because they're so easy to
> ignore.  People won't allow high priorities to remain high, though. 
> It irritates them.  That's good <wink>.
> 

The other thing is that if something doesn't get done then the tag will 
become outdated.  While it is true that this shouldn't happen if it does 
it would be a pain to go back through and remove those tags.

-Brett
From skip at pobox.com  Thu Aug 19 19:40:49 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 19 19:41:04 2004
Subject: [Python-Dev] refcounts.dat
In-Reply-To: <y8kbp5wx.fsf@python.net>
References: <y8kbp5wx.fsf@python.net>
Message-ID: <16676.58913.458682.732050@montanaro.dyndns.org>


    Thomas> Is the Doc/api/refcounts.dat file no longer maintained?  The
    Thomas> last commit was on 2002/10/04.  How does the C api manual the
    Thomas> refcount info?

I suppose like any other piece of documentation refcounts.dat gets
out-of-date without some periodic attention.  If there are functions missing
from that file they can be added (or a bug report filed).

Take a look at Doc/tools/anno-api.py for details on how the file is used.

Skip
From jcarlson at uci.edu  Thu Aug 19 19:59:13 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu Aug 19 20:06:04 2004
Subject: [Python-Dev] Considering decorator syntax on an individual
	feature basis
In-Reply-To: <1092931991.4124d1976b8ba@webmail.uoguelph.ca>
References: <1092931991.4124d1976b8ba@webmail.uoguelph.ca>
Message-ID: <20040819095717.3444.JCARLSON@uci.edu>


> The features in question, which should be decided individually rather than as a
> whole:

Sounds good.  I'll do something different here and remove your and my
opinions from this post, and stick to "facts", or at least statements
that can be reasonably derived from "facts..


> - decorator before/within/after def.

While I understand that Python is not a democracy, voting does allow us
to discover certain preferences within the community.

According to the informal poll linked from the Python decorators wiki,
that I posted about earlier, which seemed to have gotten no response,
Guido's before approach is in the minority according to the community;
"list within def" and "@ after def" favorites by a clear margin.

A. @classmethod def foo(): (107)   16%
C1. def foo() [classmethod]: (269) 39%
E1. def foo(): @classmethod (311)  45%

The fact that the two favored syntax families have been rejected by
Guido suggests that the community has strong opinions either against
@before, or in favor of [] within or @after.

Based on people's statements against the @ symbol in the @before syntax,
the voting suggests that location plays an important role, as @ before
has slightly more than 34% as many votes as @after (or 40% as many as
list within).

It seems that those who voted for C1 disagree with the formatting
argument presented against C1, and those who voted for E1 think that
within the block is the proper location for decorators.


> - special character/keyword/no new symbol.

All of which are options in all three before, within and after def
syntax families.  If a symbol is necessary for the location that is
chosen, perhaps an informal poll to choose the favorite within the
community is in order.


> - decorator on one/many line(s)
> - one symbol per decorator vs. one symbol for all decorators

Not really relevant.  If we allow:
    @decorator(arg1, arg2)

Then one/many lines and one/many symbols is a simple user-supplied
decorator away, regardless of the underlying syntax.


> - indent def/not indent def

Mostly J1 vs every other option.  Due to the lack of a reasonable number
of people posting with "indentation is the right way" on python-dev
during that flurry of hundreds of emails, I don't believe indenting the
def is remotely in the running.


> I think if each of these issues could be resolved and agreed to individually,
> then a consensis would be far more apparent as to the ideal solution, or at
> least some of the myriad solutions could be filtered out.

Agreed.


> I think the Python community has mostly settled on a "before def" approach, and

The informal poll numbers say otherwise.


 - Josiah

From walter at livinglogic.de  Thu Aug 19 20:09:17 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 19 20:09:22 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <41247DD5.8020100@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>
	<4123B202.5000200@livinglogic.de>	<4123B6F3.6060708@egenix.com>
	<4123BD93.2040107@livinglogic.de> <41247DD5.8020100@egenix.com>
Message-ID: <4124ECCD.9080209@livinglogic.de>

M.-A. Lemburg wrote:

> Walter D?rwald wrote:
> 
>> Let's compare example uses:
>>
>> 1) Having feed() as part of the StreamReader API:
>> ---
>> s = u"???".encode("utf-8")
>> r = codecs.getreader("utf-8")()
>> for c in s:
>>    print r.feed(c)
>> ---
> 
> I consider adding a .feed() method to the stream codec
> bad design. .feed() is something you do on a stream, not
> a codec.

I don't care about the name, we can call it
stateful_decode_byte_chunk() or whatever. (In fact I'd
prefer to call it decode(), but that name is already
taken by another method. Of course we could always
rename decode() to _internal_decode() like Martin
suggested.)

>> 2) Explicitely using a queue object:
>> ---
>> from whatever import StreamQueue
>>
>> s = u"???".encode("utf-8")
>> q = StreamQueue()
>> r = codecs.getreader("utf-8")(q)
>> for c in s:
>>    q.write(c)
>>    print r.read()
>> ---
> 
> This is probably how an advanced codec writer would use the APIs
> to build new stream interfaces.
 >
>> 3) Using a special wrapper that implicitely creates a queue:
>> ----
>> from whatever import StreamQueueWrapper
>> s = u"???".encode("utf-8")
>> r = StreamQueueWrapper(codecs.getreader("utf-8"))
>> for c in s:
>>    print r.feed(c)
>> ----
> 
> 
> This could be turned into something more straight forward,
> e.g.
> 
> from codecs import EncodedStream
> 
> # Load data
> s = u"???".encode("utf-8")
> 
> # Write to encoded stream (one byte at a time) and print
> # the read output
> q = EncodedStream(input_encoding="utf-8", output_encoding="unicode")

This is confusing, because there is no encoding named "unicode".
This should probably read:

q = EncodedQueue(encoding="utf-8", errors="strict")

> for c in s:
>    q.write(c)
>    print q.read()
> 
> # Make sure we have processed all data:
> if q.has_pending_data():
>    raise ValueError, 'data truncated'

This should be the job of the error callback, the last part should
probably be:

for c in s:
    q.write(c)
    print q.read()
print q.read(final=True)

>> I very much prefer option 1).
> 
> I prefer the above example because it's easy to read and
> makes things explicit.
> 
>> "If the implementation is hard to explain, it's a bad idea."
> 
> The user usually doesn't care about the implementation, only it's
> interfaces.

Bye,
    Walter D?rwald


From walter at livinglogic.de  Thu Aug 19 20:12:31 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 19 20:12:35 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412480F8.4060903@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de> <412480F8.4060903@egenix.com>
Message-ID: <4124ED8F.1020105@livinglogic.de>

M.-A. Lemburg wrote:

> Walter D?rwald wrote:
> 
>> Without the feed method(), we need the following:
>>
>> 1) A StreamQueue class that
>>    a) supports writing at one end and reading at the other end
>>    b) has a method for pushing back unused bytes to be returned
>>       in the next call to read()
> 
> Right.

As already stated in a previous post, I don't think StreamQueue
needs a pushback() method. trailing bytes should all be stored
in the StreamReader.

> It also needs a method giving the number of pending bytes in
> the queue or just an API .has_pending_data() that returns
> True/False.

What would this method be used for?

Bye,
    Walter D?rwald


From tim.peters at gmail.com  Thu Aug 19 20:19:35 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 19 20:19:43 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.319, 2.320
In-Reply-To: <uu0uzowzg.fsf@boost-consulting.com>
References: <000f01c48590$866705e0$e841fea9@oemcomputer>
	<200408190407.i7J47Ar19512@guido.python.org>
	<uu0uzowzg.fsf@boost-consulting.com>
Message-ID: <1f7befae04081911192336c3b8@mail.gmail.com>

[Guido]
>> Maybe I'm out of tune, but I thought that optimizations should be
>> turned off by default because most people don't need them and because
>> of the risk that the optimizer might break something.  Haven't there
>> been situations in Python where one optimization or another was found
>> to be unsafe after having been in use (in a release!) for a long
>> time?
 
[David Abrahams]
> Isn't that a good argument for having them turned on all the time?
> The easiest way to ship code that suffers from an optimizer bug is to
> do all your development and most of your testing with unoptimized
> code.

That's my belief in this case (as I tried to say earlier, perhaps
unsuccessfully).  The current peephole optimizer has been
overwhelmingly successful (the bug report I referenced was about
modules with more than 64KB of bytecode, and Python has always had
troubles of one kind or another with those -- so it grew another one
in that area, BFD), and I'm glad everyone runs it all the time.

> In C/C++, there's a good reason to develop code unoptimized: it's much
> easier to debug.  I'm not sure that applies to Python.

It *used* to apply, and for a similar reason, when -O also controlled
whether SET_LINENO opcodes were generated, and the Python debugger was
blind without them.  That's no longer the case.  The only reason to
avoid -O now is to retain

    if __debug__:

blocks and (same thing) assert statements.  That can make a big speed
difference in Zope and ZEO, for example.  Life would definitely be
worse if -O could introduce new bugs (other than programmer errors of
putting essential code inside __debug__ blocks).
From anthony at interlink.com.au  Thu Aug 19 20:20:51 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Aug 19 20:21:02 2004
Subject: [Python-Dev] Considering decorator syntax on an individual	feature
	basis
In-Reply-To: <20040819095717.3444.JCARLSON@uci.edu>
References: <1092931991.4124d1976b8ba@webmail.uoguelph.ca>
	<20040819095717.3444.JCARLSON@uci.edu>
Message-ID: <4124EF83.40609@interlink.com.au>

Please, please, please - could the discussion about decorators
take place on comp.lang.python, unless there's something new to
add to the thread? python-dev is rapidly becoming unusable.

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From guido at python.org  Thu Aug 19 20:38:48 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 19 20:38:55 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: Your message of "Thu, 19 Aug 2004 11:15:17 +0200."
	<41246FA5.1010804@egenix.com> 
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net> <4122304E.1070907@egenix.com>
	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>
	<41227F83.4060403@egenix.com>
	<Pine.LNX.4.58.0408180821240.1555@rnd.onego.ru>
	<41231830.2090903@egenix.com>
	<200408190139.i7J1dX819100@guido.python.org> 
	<41246FA5.1010804@egenix.com> 
Message-ID: <200408191838.i7JIcmm21008@guido.python.org>

> Ok, so I suppose that we can learn from Jython and IronPython in
> this respect...
> 
> How do they handle binary data and the interfacing between various
> I/O facilities, e.g. files, sockets, pipes, user input, etc.

I'm not sure, but I expect that in most cases they use Unicode strings
in order to be compatibly with Python's standard library.  That's not
the outcome I'd like to see though.  I believe Jython at least also
has a bytes-like type (probably a thin wrapper around Java's byte
array) that's used for interfacing to java classes.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug 19 20:52:38 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 19 20:52:45 2004
Subject: [Python-Dev] 2.4a3 release is September 2, SF tracker keywords
In-Reply-To: Your message of "Thu, 19 Aug 2004 12:48:08 EDT."
	<1f7befae040819094825bfacf0@mail.gmail.com> 
References: <4124CC59.80007@interlink.com.au>  
	<1f7befae040819094825bfacf0@mail.gmail.com> 
Message-ID: <200408191852.i7JIqcR21129@guido.python.org>

+1 on Tim's suggestion of using priorities.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Thu Aug 19 20:53:02 2004
From: guido at python.org (Guido van Rossum)
Date: Thu Aug 19 20:53:09 2004
Subject: [Python-Dev] Re: 2.4a3 release is September 2, SF tracker keywords
In-Reply-To: Your message of "Thu, 19 Aug 2004 17:50:22 BST."
	<2mk6vv8435.fsf@starship.python.net> 
References: <4124CC59.80007@interlink.com.au> <4124D8E4.8030705@python.org> 
	<2mk6vv8435.fsf@starship.python.net> 
Message-ID: <200408191853.i7JIr2t21142@guido.python.org>

> >>   - regression since 2.3
> >
> > What does that one mean?
> 
> I guess thing's like Tim's recent asyncore pain.

And time.strftime()?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From martin at v.loewis.de  Thu Aug 19 20:54:38 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 19 20:54:36 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412475DF.9010308@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>
	<4122E19E.9050906@v.loewis.de>	<412314F6.5060502@egenix.com>
	<4123BC56.7030607@v.loewis.de> <412475DF.9010308@egenix.com>
Message-ID: <4124F76E.9040901@v.loewis.de>

M.-A. Lemburg wrote:
>> Marc-Andre, if the original patch (diff.txt) was applied:
>> What *specific* change in that patch would break code?
>> What *specific* code (C or Python) would break under that
>> change?
[...]
> Let's flesh this out some more and get a better understanding
> of what is needed and how the separation between the stream queue,
> the stream codec and the underlying codec implementation can
> be put to good use.

That really didn't answer the question: What would be technically
wrong with accepting Walter's patches?

I smell over-designing: there is a specific problem at hand
(incomplete reads resulting in partial decoding); any solution
should attempt to *only* solve this problem, not any other
problem.

Regards,
Martin


From martin at v.loewis.de  Thu Aug 19 20:58:07 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 19 20:58:04 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4124CB16.90705@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4123B202.5000200@livinglogic.de>
	<4123B6F3.6060708@egenix.com> <4123BD93.2040107@livinglogic.de>
	<4123C16D.4060505@v.loewis.de> <4124CB16.90705@livinglogic.de>
Message-ID: <4124F83F.1080707@v.loewis.de>

Walter D?rwald wrote:
> 1) The current implementation does not really support the
>    stateful API, because trailing incomplete byte sequences
>    lead to errors.

Correct.

> 2) The current file API is not really convenient for decoding
>    when the input is not read for a stream.

I don't see this problem. It is straight-forward to come up
with a file-like object that converts the data from whatever
source into a stream. If it wasn't a byte stream/string eventually,
there would be no way to meaningfully decode it.

Regards,
Martin
From foom at fuhm.net  Thu Aug 19 20:59:03 2004
From: foom at fuhm.net (James Y Knight)
Date: Thu Aug 19 20:59:09 2004
Subject: [Python-Dev] Re: PEP 318: Can't we all just get along?
In-Reply-To: <cg23s3$vgd$2@sea.gmane.org>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>
	<412480FF.2030203@zope.com> <cg23s3$vgd$2@sea.gmane.org>
Message-ID: <D692A7B8-F211-11D8-9909-000A95A50FB2@fuhm.net>

On Aug 19, 2004, at 7:45 AM, Paul Morrow wrote:
> The vast majority of instance methods I've seen all use 'self' as the 
> first parameter.  Likewise, most class methods use 'cls' or 'klass' as 
> their first parameter.  If we exploit these conventions, we end up 
> with a simple, clear, obvious mechanism for denoting (this aspect of) 
> a method's type.
>
>     class Foo(Object):
>         def m1(self, a, b):    # this is an instance method of Foo
>             pass
>
>         def m2(cls, a, b):     # this is a class method of Foo
>             pass
>
>         def m3(a, b):          # this is a static method of Foo
>             pass
>
> A special Object (capital 'O') class could work this magic so that old 
> code didn't break.
>
> I know that this is odd.  But then so are most of the great things 
> about Python.

You can do that today. See also 
http://www.python.org/pycon/dc2004/papers/48/conveniencytypes.py

However, note that IMO it is quite rude to use a metaclass (or your 
capital O object -- same thing) to do this, as it will break any 
objects inheriting from your class that don't expect the strange 
automatic behavior. This auto-class/staticmethod-ification should be 
local to your code, and thus is really a candidate for a class 
decorator.

@automethods
class Foo(object):
   ...

James

From martin at v.loewis.de  Thu Aug 19 21:05:43 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 19 21:05:38 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4124DA28.50106@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>
	<4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de>
Message-ID: <4124FA07.1070701@v.loewis.de>

Walter D?rwald wrote:
> Both approaches have one problem: Error handling won't
> work for them. If the error handling is "replace", the decoder
> should return U+FFFD for the final trailing incomplete sequence
> in read(). This won't happen when read() never reads those
> bytes from the input stream.

Ok. So it really looks like a final flag on read is necessary.

> Well, I had to choose a subject. ;)

I still would prefer if the two issues were discussed separately.

Regards,
Martin
From martin at v.loewis.de  Thu Aug 19 21:20:18 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 19 21:20:16 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <16676.47823.382278.737098@montanaro.dyndns.org>
References: <200408190529.i7J5TZF7010126@h006008a7bda6.ne.client2.attbi.com>
	<16676.47823.382278.737098@montanaro.dyndns.org>
Message-ID: <4124FD72.4090901@v.loewis.de>

Skip Montanaro wrote:
> Any thoughts about posting this to c.l.py as well as python-dev?  Maybe it
> would encourage some more participation...

+1.

Martin
From martin at v.loewis.de  Thu Aug 19 21:21:59 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 19 21:21:57 2004
Subject: [Python-Dev] Re: 2.4a3 release is September 2, SF tracker keywords
In-Reply-To: <4124D8E4.8030705@python.org>
References: <4124CC59.80007@interlink.com.au> <4124D8E4.8030705@python.org>
Message-ID: <4124FDD7.6030305@v.loewis.de>

David Goodger wrote:
> [Anthony Baxter]
> 
>>   - regression since 2.3
> 
> 
> What does that one mean?

Are you asking what a regression is? In that context:
A feature/functionality that used to work in 2.3, but stopped
working in CVS.

Regards,
Martin
From pm_mon at yahoo.com  Thu Aug 19 21:30:38 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Thu Aug 19 21:30:43 2004
Subject: [Python-Dev] Re: PEP 318: Can't we all just get along?
In-Reply-To: <D692A7B8-F211-11D8-9909-000A95A50FB2@fuhm.net>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>	<412480FF.2030203@zope.com>
	<cg23s3$vgd$2@sea.gmane.org>
	<D692A7B8-F211-11D8-9909-000A95A50FB2@fuhm.net>
Message-ID: <cg2v4s$l89$1@sea.gmane.org>

James Y Knight wrote:

> On Aug 19, 2004, at 7:45 AM, Paul Morrow wrote:
> 
>> The vast majority of instance methods I've seen all use 'self' as the 
>> first parameter.  Likewise, most class methods use 'cls' or 'klass' as 
>> their first parameter.  If we exploit these conventions, we end up 
>> with a simple, clear, obvious mechanism for denoting (this aspect of) 
>> a method's type.
>>
>>     class Foo(Object):
>>         def m1(self, a, b):    # this is an instance method of Foo
>>             pass
>>
>>         def m2(cls, a, b):     # this is a class method of Foo
>>             pass
>>
>>         def m3(a, b):          # this is a static method of Foo
>>             pass
>>
>> A special Object (capital 'O') class could work this magic so that old 
>> code didn't break.
>>
>> I know that this is odd.  But then so are most of the great things 
>> about Python.
> 
> 
> You can do that today. See also 
> http://www.python.org/pycon/dc2004/papers/48/conveniencytypes.py
> 

Aha!  Thanks!

> However, note that IMO it is quite rude to use a metaclass (or your 
> capital O object -- same thing) to do this, as it will break any objects 
> inheriting from your class that don't expect the strange automatic 
> behavior. This auto-class/staticmethod-ification should be local to your 
> code, and thus is really a candidate for a class decorator.
> 
> @automethods
> class Foo(object):
>   ...
> 

But then wouldn't subclasses of Foo have the same rude behavior?  In 
fact, isn't the use of any metaclass rude, by your definition, as it 
will change the default class behavior in some way?




From bob at redivi.com  Thu Aug 19 21:38:24 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Aug 19 21:39:01 2004
Subject: [Python-Dev] Re: PEP 318: Can't we all just get along?
In-Reply-To: <cg2v4s$l89$1@sea.gmane.org>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>	<412480FF.2030203@zope.com>
	<cg23s3$vgd$2@sea.gmane.org>
	<D692A7B8-F211-11D8-9909-000A95A50FB2@fuhm.net>
	<cg2v4s$l89$1@sea.gmane.org>
Message-ID: <56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com>


On Aug 19, 2004, at 3:30 PM, Paul Morrow wrote:

> James Y Knight wrote:
>
>> On Aug 19, 2004, at 7:45 AM, Paul Morrow wrote:
>>> The vast majority of instance methods I've seen all use 'self' as 
>>> the first parameter.  Likewise, most class methods use 'cls' or 
>>> 'klass' as their first parameter.  If we exploit these conventions, 
>>> we end up with a simple, clear, obvious mechanism for denoting (this 
>>> aspect of) a method's type.
>>>
>>>     class Foo(Object):
>>>         def m1(self, a, b):    # this is an instance method of Foo
>>>             pass
>>>
>>>         def m2(cls, a, b):     # this is a class method of Foo
>>>             pass
>>>
>>>         def m3(a, b):          # this is a static method of Foo
>>>             pass
>>>
>>> A special Object (capital 'O') class could work this magic so that 
>>> old code didn't break.
>>>
>>> I know that this is odd.  But then so are most of the great things 
>>> about Python.
>> You can do that today. See also 
>> http://www.python.org/pycon/dc2004/papers/48/conveniencytypes.py
>
> Aha!  Thanks!

Whatever happened to explicit is better than implicit?  ;)

>> However, note that IMO it is quite rude to use a metaclass (or your 
>> capital O object -- same thing) to do this, as it will break any 
>> objects inheriting from your class that don't expect the strange 
>> automatic behavior. This auto-class/staticmethod-ification should be 
>> local to your code, and thus is really a candidate for a class 
>> decorator.
>> @automethods
>> class Foo(object):
>>   ...
>
> But then wouldn't subclasses of Foo have the same rude behavior?  In 
> fact, isn't the use of any metaclass rude, by your definition, as it 
> will change the default class behavior in some way?

No, a decorator happens once.  A metaclass can/does have permanent 
effects and becomes an integral and/or annoying part of the inheritance 
graph.

class Foo(object):
     __metaclass__ = Bar
     ...

is *much* different than

class Foo(object):
     ...
Foo = barEquivalentFunction(Foo)

which is the same as

@barEquivalentFunction
class Foo(object):
     ...

-bob
From mal at egenix.com  Thu Aug 19 21:59:06 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 21:59:12 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4124F76E.9040901@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<412475DF.9010308@egenix.com> <4124F76E.9040901@v.loewis.de>
Message-ID: <4125068A.2010509@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
> 
>>> Marc-Andre, if the original patch (diff.txt) was applied:
>>> What *specific* change in that patch would break code?
>>> What *specific* code (C or Python) would break under that
>>> change?
> 
> [...]
> 
>> Let's flesh this out some more and get a better understanding
>> of what is needed and how the separation between the stream queue,
>> the stream codec and the underlying codec implementation can
>> be put to good use.
> 
> 
> That really didn't answer the question: What would be technically
> wrong with accepting Walter's patches?
> 
> I smell over-designing: there is a specific problem at hand
> (incomplete reads resulting in partial decoding); any solution
> should attempt to *only* solve this problem, not any other
> problem.

The specific problem is that of providing a codec that can
run in feeding mode where you can feed in data and read
it in a way that allows incomplete input data to fail over
nicely.

Since this requires two head positions (one for the writer,
one for the reader), a queue implementation is the right
thing to use.

We are having this discussion to find a suitable design
to provide this functionality in a nice and clean way.
I don't see anything wrong with this.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Thu Aug 19 22:03:41 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 22:03:45 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4124ED8F.1020105@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>	<4123C76B.3030100@livinglogic.de>
	<412480F8.4060903@egenix.com> <4124ED8F.1020105@livinglogic.de>
Message-ID: <4125079D.4000609@egenix.com>

Walter D?rwald wrote:
> M.-A. Lemburg wrote:
> 
>> Walter D?rwald wrote:
>>
>>> Without the feed method(), we need the following:
>>>
>>> 1) A StreamQueue class that
>>>    a) supports writing at one end and reading at the other end
>>>    b) has a method for pushing back unused bytes to be returned
>>>       in the next call to read()
>>
>>
>> Right.
> 
> 
> As already stated in a previous post, I don't think StreamQueue
> needs a pushback() method. trailing bytes should all be stored
> in the StreamReader.

I'd leave that to the StreamReader implementor. I would always
push the data back onto the queue, simply because it's unprocessed
data.

>> It also needs a method giving the number of pending bytes in
>> the queue or just an API .has_pending_data() that returns
>> True/False.
> 
> 
> What would this method be used for?

If you push the data back onto the queue, you will probably want
to check whether there's pending data left. That's what this
method is intended to tell you.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Thu Aug 19 22:09:09 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 19 22:09:12 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4124ECCD.9080209@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>	<4123B202.5000200@livinglogic.de>	<4123B6F3.6060708@egenix.com>	<4123BD93.2040107@livinglogic.de>
	<41247DD5.8020100@egenix.com> <4124ECCD.9080209@livinglogic.de>
Message-ID: <412508E5.7010307@egenix.com>

Walter D?rwald wrote:
> M.-A. Lemburg wrote:
> 
>> Walter D?rwald wrote:
>>
>>> Let's compare example uses:
>>>
>>> 1) Having feed() as part of the StreamReader API:
>>> ---
>>> s = u"???".encode("utf-8")
>>> r = codecs.getreader("utf-8")()
>>> for c in s:
>>>    print r.feed(c)
>>> ---
>>
>>
>> I consider adding a .feed() method to the stream codec
>> bad design. .feed() is something you do on a stream, not
>> a codec.
> 
> 
> I don't care about the name, we can call it
> stateful_decode_byte_chunk() or whatever. (In fact I'd
> prefer to call it decode(), but that name is already
> taken by another method. Of course we could always
> rename decode() to _internal_decode() like Martin
> suggested.)

It's not that name that doesn't fit, it's the fact
that you are mixing a stream action into a codec which
I'd rather see well separated.

>>> 2) Explicitely using a queue object:
>>> ---
>>> from whatever import StreamQueue
>>>
>>> s = u"???".encode("utf-8")
>>> q = StreamQueue()
>>> r = codecs.getreader("utf-8")(q)
>>> for c in s:
>>>    q.write(c)
>>>    print r.read()
>>> ---
>>
>>
>> This is probably how an advanced codec writer would use the APIs
>> to build new stream interfaces.
> 
>  >
> 
>>> 3) Using a special wrapper that implicitely creates a queue:
>>> ----
>>> from whatever import StreamQueueWrapper
>>> s = u"???".encode("utf-8")
>>> r = StreamQueueWrapper(codecs.getreader("utf-8"))
>>> for c in s:
>>>    print r.feed(c)
>>> ----
>>
>>
>>
>> This could be turned into something more straight forward,
>> e.g.
>>
>> from codecs import EncodedStream
>>
>> # Load data
>> s = u"???".encode("utf-8")
>>
>> # Write to encoded stream (one byte at a time) and print
>> # the read output
>> q = EncodedStream(input_encoding="utf-8", output_encoding="unicode")
> 
> 
> This is confusing, because there is no encoding named "unicode".
> This should probably read:
> 
> q = EncodedQueue(encoding="utf-8", errors="strict")

Fine.

I was thinking of something similar to EncodedFile()
which also has two separate encodings, one for the file side
of things and one for the Python side.

>> for c in s:
>>    q.write(c)
>>    print q.read()
>>
>> # Make sure we have processed all data:
>> if q.has_pending_data():
>>    raise ValueError, 'data truncated'
> 
> 
> This should be the job of the error callback, the last part should
> probably be:
> 
> for c in s:
>    q.write(c)
>    print q.read()
> print q.read(final=True)

Ok; both methods have their use cases. (You seem to be obsessed
with this final argument ;-)

>>> I very much prefer option 1).
>>
>>
>> I prefer the above example because it's easy to read and
>> makes things explicit.
>>
>>> "If the implementation is hard to explain, it's a bad idea."
>>
>>
>> The user usually doesn't care about the implementation, only it's
>> interfaces.
> 
> 
> Bye,
>    Walter D?rwald
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/mal%40egenix.com

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From exarkun at divmod.com  Thu Aug 19 22:09:21 2004
From: exarkun at divmod.com (Jp Calderone)
Date: Thu Aug 19 22:09:28 2004
Subject: [Python-Dev] Re: PEP 318: Can't we all just get along?
In-Reply-To: <56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>	<412480FF.2030203@zope.com>	<cg23s3$vgd$2@sea.gmane.org>	<D692A7B8-F211-11D8-9909-000A95A50FB2@fuhm.net>	<cg2v4s$l89$1@sea.gmane.org>
	<56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com>
Message-ID: <412508F1.6010500@divmod.com>

Bob Ippolito wrote:
> 
> class Foo(object):
>     __metaclass__ = Bar
>     ...
> 
> is *much* different than
> 
> class Foo(object):
>     ...
> Foo = barEquivalentFunction(Foo)
> 
> which is the same as
> 
> @barEquivalentFunction
> class Foo(object):
>     ...
> 

   Only if you choose to write it such that it is different.  Either can 
result in effects that persist down the inheritence tree, and either can 
result in effects that do not.

   Jp
From skip at pobox.com  Thu Aug 19 22:40:24 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu Aug 19 22:40:44 2004
Subject: [Python-Dev] decimal module portability to 2.3?
Message-ID: <16677.4152.404288.108320@montanaro.dyndns.org>

The docstring for the decimal module says in part:

  This is a Py2.3 implementation of decimal floating point arithmetic based on
  the General Decimal Arithmetic Specification:

Is it intended to remain 2.3-compatible for the forseeable future?  It's not
mentioned in PEP 291, nor is there any discussion of portability to 2.3 in
PEP 327.

I ask because we're at 2.3.4 at work, but could use it today.  I'd hate to
have to track it to backport fixes though.  It would be ideal if I could
simply copy the 2.4 version as it matures.

Thx,

Skip
From bac at OCF.Berkeley.EDU  Thu Aug 19 22:42:06 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug 19 22:42:26 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <16676.46377.92891.139081@montanaro.dyndns.org>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE01D58FA4@au3010avexu1.global.avaya.com>
	<16676.46377.92891.139081@montanaro.dyndns.org>
Message-ID: <4125109E.6040706@ocf.berkeley.edu>

Skip Montanaro wrote:

>     >> Is anybody seriously trying to come up with a single alternative
>     >> decorator proposal that most folks "out there" can support, to be
>     >> presented to me (with implementation, please!) in time for 2.4b1?
> 
>     Tim> I'm afraid that absolutely zero consensus is emerging :( I've
>     Tim> withdrawn myself entirely from the discussions as they've proven
>     Tim> completely useless.
> 
> Ditto.  I think the list-after-def proposal had some support early on but
> people got very carried away proposing all sorts of other alternatives.
> 

Having just read the bulk of the decorator emails in two nights, I can 
confidently say that three proposals had steam.

One was definitely list-after-def.

Another was using a pipe (E2, although there is none showing it above 
the function definition).  People seemed to like that except for the 
font argument against it.

The last one was using '!' instead of either '@' or '|'.

If people want to get an alternative in I say vote on these four  (the 
'!' has to choose where it goes relative to the 'def') on c.l.py and 
present it to Guido 5 days before a3 (so by the 27th; that's a week from 
tomorrow) to get it in since people will need time to play with it 
before we hit b1.  Ignore if Guido rejected it at some point and just 
see if everyone can just agree on *something*.

Then Guido decides and this thing can be over with and people can just 
move on to learning how to use decorators properly.

-Brett
From martin at v.loewis.de  Thu Aug 19 22:42:48 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 19 22:42:43 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4125068A.2010509@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<412475DF.9010308@egenix.com> <4124F76E.9040901@v.loewis.de>
	<4125068A.2010509@egenix.com>
Message-ID: <412510C8.5060508@v.loewis.de>

M.-A. Lemburg wrote:

> We are having this discussion to find a suitable design
> to provide this functionality in a nice and clean way.
> I don't see anything wrong with this.

The problem is that this has nothing to do with decoding
incomplete unicode.

Regards,
Martin

From bob at redivi.com  Thu Aug 19 22:55:04 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Aug 19 22:55:42 2004
Subject: [Python-Dev] Re: PEP 318: Can't we all just get along?
In-Reply-To: <412508F1.6010500@divmod.com>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>	<412480FF.2030203@zope.com>	<cg23s3$vgd$2@sea.gmane.org>	<D692A7B8-F211-11D8-9909-000A95A50FB2@fuhm.net>	<cg2v4s$l89$1@sea.gmane.org>
	<56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com>
	<412508F1.6010500@divmod.com>
Message-ID: <0BD15540-F222-11D8-BC06-000A95686CD8@redivi.com>


On Aug 19, 2004, at 4:09 PM, Jp Calderone wrote:

> Bob Ippolito wrote:
>> class Foo(object):
>>     __metaclass__ = Bar
>>     ...
>> is *much* different than
>> class Foo(object):
>>     ...
>> Foo = barEquivalentFunction(Foo)
>> which is the same as
>> @barEquivalentFunction
>> class Foo(object):
>>     ...
>
>   Only if you choose to write it such that it is different.  Either 
> can result in effects that persist down the inheritence tree, and 
> either can result in effects that do not.

Of course, but my point is that it's default behavior for the "Bar" 
metaclass to become part of the inheritance graph, where default 
behavior for "barEquivalentFunction" wouldn't ever do such a thing.  
Actually, aside from returning a different class object altogether, the 
only thing I can imagine "barEquivalentFunction" doing that would 
affect the inheritance graph is to mutate __bases__, which is just 
crazy.

-bob
From pm_mon at yahoo.com  Thu Aug 19 23:06:23 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Thu Aug 19 23:06:24 2004
Subject: [Python-Dev] Re: PEP 318: Can't we all just get along?
In-Reply-To: <56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com>
References: <CC9FC7A0-F14E-11D8-BC29-000393829ED2@theMorgue.org>	<F627CCA1-F14F-11D8-86FF-000A95686CD8@redivi.com>	<412480FF.2030203@zope.com>	<cg23s3$vgd$2@sea.gmane.org>	<D692A7B8-F211-11D8-9909-000A95A50FB2@fuhm.net>	<cg2v4s$l89$1@sea.gmane.org>
	<56129CCA-F217-11D8-BC06-000A95686CD8@redivi.com>
Message-ID: <cg34oc$5ad$1@sea.gmane.org>

Bob Ippolito wrote:
> 
> On Aug 19, 2004, at 3:30 PM, Paul Morrow wrote:
> 
>> James Y Knight wrote:
>>
>>> On Aug 19, 2004, at 7:45 AM, Paul Morrow wrote:
>>>
>>>> The vast majority of instance methods I've seen all use 'self' as 
>>>> the first parameter.  Likewise, most class methods use 'cls' or 
>>>> 'klass' as their first parameter.  If we exploit these conventions, 
>>>> we end up with a simple, clear, obvious mechanism for denoting (this 
>>>> aspect of) a method's type.
>>>>
>>>>     class Foo(Object):
>>>>         def m1(self, a, b):    # this is an instance method of Foo
>>>>             pass
>>>>
>>>>         def m2(cls, a, b):     # this is a class method of Foo
>>>>             pass
>>>>
>>>>         def m3(a, b):          # this is a static method of Foo
>>>>             pass
>>>>
>>>> A special Object (capital 'O') class could work this magic so that 
>>>> old code didn't break.
>>>>
>>>> I know that this is odd.  But then so are most of the great things 
>>>> about Python.
>>>
>>> You can do that today. See also 
>>> http://www.python.org/pycon/dc2004/papers/48/conveniencytypes.py
>>
>>
>> Aha!  Thanks!
> 
> 
> Whatever happened to explicit is better than implicit?  ;)
> 

I've often wondered about that principle.  For example, wouldn't static 
typing (actual declarations, as in Java) be more explicit than dynamic 
typing?

Either way, it seems that if dynamic method typing 
(static|class|instance) was enabled for a class, then

     def m2(cls, a, b): pass

would be every bit as explicit (a means of specifying that m2 is a class 
method) as

     x = []

(is of specifying that x is (well, contains) a list).

From FBatista at uniFON.com.ar  Thu Aug 19 23:06:15 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Thu Aug 19 23:14:15 2004
Subject: [Python-Dev] decimal module portability to 2.3?
Message-ID: <A128D751272CD411BC9200508BC2194D053C7815@escpl.tcp.com.ar>

[Skip Montanaro]

#- Is it intended to remain 2.3-compatible for the forseeable 
#- future?  It's not
#- mentioned in PEP 291, nor is there any discussion of 
#- portability to 2.3 in
#- PEP 327.

I don't have plans to do some effort to keep it "2.3-runnable" when Py2.4
hits the street. Don't know if Raymond or somebody will preocupate about
this.

Could decimal be backported to 2.3 and be included in some 2.3.5? Don't know
if this behaviour is common to Python.


#- I ask because we're at 2.3.4 at work, but could use it 
#- today.  I'd hate to
#- have to track it to backport fixes though.  It would be 
#- ideal if I could
#- simply copy the 2.4 version as it matures.

I have a similar issue. 

I copied the decimal.py code and included it on the SiGeFi package
(http://sourceforge.net/projects/sigefi), so when somebody downloads the 0.2
version could use it with Py2.3 (it's the version required).

My plan (right now) is to wait Py2.4, then remove decimal.py from the
package and require Py2.4 (this is justified because SiGeFi will reach "user
usage" next year).

.	Facundo
From bac at OCF.Berkeley.EDU  Thu Aug 19 23:35:58 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug 19 23:36:01 2004
Subject: [Python-Dev] decimal module portability to 2.3?
In-Reply-To: <A128D751272CD411BC9200508BC2194D053C7815@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D053C7815@escpl.tcp.com.ar>
Message-ID: <41251D3E.1090305@ocf.berkeley.edu>

Batista, Facundo wrote:

> Could decimal be backported to 2.3 and be included in some 2.3.5? Don't know
> if this behaviour is common to Python.
> 

That can't be done.  That would introduce new features in a bug release 
which is a big no-no.

-Brett
From python at rcn.com  Thu Aug 19 23:41:16 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Aug 19 23:41:46 2004
Subject: [Python-Dev] decimal module portability to 2.3?
In-Reply-To: <A128D751272CD411BC9200508BC2194D053C7815@escpl.tcp.com.ar>
Message-ID: <000801c48635$46986e40$e841fea9@oemcomputer>

> [Skip Montanaro]
> 
> #- Is it intended to remain 2.3-compatible for the forseeable
> #- future?  It's not
> #- mentioned in PEP 291, nor is there any discussion of
> #- portability to 2.3 in
> #- PEP 327.
> 
> I don't have plans to do some effort to keep it "2.3-runnable" when
Py2.4
> hits the street. Don't know if Raymond or somebody will preocupate
about
> this.

Yes, I've been keeping it Py2.3 compatible and periodically test it on
2.3 just to make sure.

Am hesitant to introduce this as a specific, on-going restriction to the
code, but I don't want to throw-away 2.3 compatibility unless there is a
darned good offsetting gain.


Raymond

From bac at OCF.Berkeley.EDU  Thu Aug 19 23:42:00 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug 19 23:42:05 2004
Subject: [Python-Dev] Unifying Long Integers and Integers: baseint
In-Reply-To: <1092659867.4120aa9ba1239@mcherm.com>
References: <1092659867.4120aa9ba1239@mcherm.com>
Message-ID: <41251EA8.8050008@ocf.berkeley.edu>

Michael Chermside wrote:

>>>>>if the only reason for it is to use isinstance?
>>>>
>>>>So that an extension author *could* write an int-like type deriving
>>>>from it?
>>>
>>>But didn't you just say that people shouldn't be
>>>deriving their own int-like types from baseinteger?
>>
>>Indeed, in general they shouldn't.  But for specialized purposes it
>>might be needed (that's why I emphasized *could*).
> 
> 
> I call YAGNI. We're talking about creating the class baseinteger
> which might be useful ONLY for people creating new kinds of integers
> in Python which will NOT extend int or long but WILL need to be
> treated just like integers. Who is really likely to do that? And if
> in the process we introduce a new class which won't be needed in
> the long run (ie Python 3000 has just one type, called "int" and has
> no need for baseinteger). So I maintain that it's not needed (and
> is, in fact, confusing to users) unless someone has a real use case.
> 

I'm with Michael on this.  We have gone this long without having a need 
for a baseinteger type (when was long introduced?) so I don't see a need 
to add it now.  Let's just live with the dichotomy until Python 3000 
(moving over to 3000 as Guido suggested in the "PEP 3000" thread) comes out.

-Brett
From python at rcn.com  Thu Aug 19 23:47:26 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Aug 19 23:47:48 2004
Subject: [Python-Dev] decimal module portability to 2.3?
In-Reply-To: <41251D3E.1090305@ocf.berkeley.edu>
Message-ID: <000901c48636$1eb2b240$e841fea9@oemcomputer>

> Batista, Facundo wrote:
> 
> > Could decimal be backported to 2.3 and be included in some 2.3.5?
Don't
> know
> > if this behaviour is common to Python.

[Brett] 
> That can't be done.  That would introduce new features in a bug
release
> which is a big no-no.

Right.  Though keeping it 2.3 compatible means people can download it as
a standalone module just like any other third-party module.  This worked
out well for sets.py which I've made 2.2 compatible.

By keeping decimal 2.3 compatible, folks can make periodic downloads to
take advantage of bug fixes and efforts to keep it in sync with the
on-going evolution of the spec.

Also, I'm expecting that at some point in the future, there will be a
concerted effort to improve its performance -- it runs two orders of
magnitude slower than we would like (see the telco benchmark in the
sandbox).


Raymond


From tjreedy at udel.edu  Thu Aug 19 23:49:25 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu Aug 19 23:49:30 2004
Subject: [Python-Dev] Re: PEP 318: Suggest we drop it
References: <4123CB1A.2060101@sabaydi.com>
	<Pine.LNX.4.58.0408190817070.16021@rnd.onego.ru>
Message-ID: <cg3795$8mq$1@sea.gmane.org>


"Roman Suzi" <rnd@onego.ru> wrote in message
news:Pine.LNX.4.58.0408190817070.16021@rnd.onego.ru...
> and their support for "write once" concept, while in old syntax one need
> to write a function name 4 times to merely get it decorated and with
__doc__
> string.

I believed this once also, but then realized that one only need write a
long function name once (other than in the doc string)  -- in the final
assignment.  Perhaps something like

d = '''actual_long_name(ague) == return syncroed flooblejack
  ague == seq of doobies
'''
def f(ague):
    pass
f.__doc__ = d
f.atto = 'yes'
actual_long_name = syncro(staticmethod(deco(f)))

Even without a doc string or comment at the top, finding the actual name at
the bottem of a block might be as easy as finding it in the middle of a
block with several @ lines.

So I no longer count this as so strong an argument against the postfix form
as I one did.

Terry J. Reedy



From bac at OCF.Berkeley.EDU  Fri Aug 20 00:18:43 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug 20 00:18:48 2004
Subject: [Python-Dev] decimal module portability to 2.3?
In-Reply-To: <000901c48636$1eb2b240$e841fea9@oemcomputer>
References: <000901c48636$1eb2b240$e841fea9@oemcomputer>
Message-ID: <41252743.7000301@ocf.berkeley.edu>

Raymond Hettinger wrote:

>>Batista, Facundo wrote:
>>
>>
>>>Could decimal be backported to 2.3 and be included in some 2.3.5?
> 
> Don't
> 
>>know
>>
>>>if this behaviour is common to Python.
> 
> 
> [Brett] 
> 
>>That can't be done.  That would introduce new features in a bug
> 
> release
> 
>>which is a big no-no.
> 
> 
> Right.  Though keeping it 2.3 compatible means people can download it as
> a standalone module just like any other third-party module.  This worked
> out well for sets.py which I've made 2.2 compatible.
> 
> By keeping decimal 2.3 compatible, folks can make periodic downloads to
> take advantage of bug fixes and efforts to keep it in sync with the
> on-going evolution of the spec.
> 

Right, but you stated in another email you don't want to make it 
explicit by declaring it in PEP 291 so that is only true at the moment 
and could go away.  At which point people would need an easy way to get 
a hold of an older version before compatibility is broken unless the 
sandbox version is kept around and bugfixed on occasion like datetime.

-Brett
From barry at python.org  Fri Aug 20 00:32:52 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 00:32:28 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1f7befae040811221916a955fd@mail.gmail.com>
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost>
	<1f7befae040811221916a955fd@mail.gmail.com>
Message-ID: <1092954772.8712.147.camel@geddy.wooz.org>

On Thu, 2004-08-12 at 01:19, Tim Peters wrote:

> I do object to this part:
> 
>      If the $ character appears at the end of the line, or is followed
>      by any other character than those described above, it is treated
>      as if it had been escaped, appearing in the resulting string
>      unchanged.
> 
> There's already a facility to escape $, and it's dead easy to use.  $
> isn't a frequently needed character either in most apps.  So escaping
> $ "by magic" too is, I think, more likely to make typing errors harder
> to find than to do anyone real good.

What would you do about $'s at the end of the string?  I think the
implementation would be much more complex if you didn't have this rule,
e.g. you'd have to match \$$ and the $-placeholder regexps would
basically have to match everything.  Then the .sub call would have to be
more complex too, because it would have to check for the existence of
those match groups and then raise an exception.  Or something like that.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/8d26f4bc/attachment.pgp
From barry at python.org  Fri Aug 20 00:36:42 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 00:36:22 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <59e9fd3a0408112305fa1b667@mail.gmail.com>
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost>
	<1f7befae040811221916a955fd@mail.gmail.com>
	<200408120539.i7C5dqL29226@guido.python.org>
	<59e9fd3a0408112305fa1b667@mail.gmail.com>
Message-ID: <1092955002.8714.149.camel@geddy.wooz.org>

On Thu, 2004-08-12 at 02:05, Andrew Durdin wrote:
> One thing the PEP doesn't make clear: do these templates only
> interpolate string variables, or will they handle anything with a
> __str__ method?
> 
> e.g., will the following raise an exception:
> 
> print Template("I am ${height} centimetres tall.") % {height: 183}

In the current implementation, this will raise an exception.  I'm
uncomfortable with automatic stringification (hey, create your own dict
subclass for the mapping if you want ;) -- but I've included this as an
open issue in the next rev of the PEP.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/5ac2e1f7/attachment.pgp
From adurdin at gmail.com  Fri Aug 20 00:41:47 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Fri Aug 20 00:41:57 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408191158.12738.gmccaughan@synaptics-uk.com>
References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz>
	<200408190558.i7J5wPP19708@guido.python.org>
	<200408191158.12738.gmccaughan@synaptics-uk.com>
Message-ID: <59e9fd3a040819154158b60833@mail.gmail.com>

On Thu, 19 Aug 2004 11:58:12 +0100, Gareth McCaughan
<gmccaughan@synaptics-uk.com> wrote:
> On Thursday 2004-08-19 06:58, Guido van Rossum wrote:
> >
> > Well, do people generally buy those rejections, or is their consensus
> > that I'm mistaken?
> 
> I'm one of the people who doesn't really feel it's worth trying.
> *My* consensus :-) is that you're mistaken. I think that
> 

I agree with Gareth on both of his main points: I didn't buy Guido's
rejection of list-after-def (and so argued against his points in
another thread here); but the decorator suite before the def (J2) is
quite nice also, and it doesn't hide the def when there are lots of
decorators (due to the indentation). I would strongly support either
proposal over the current @decorators.
From python at rcn.com  Fri Aug 20 00:53:20 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug 20 00:53:42 2004
Subject: [Python-Dev] decimal module portability to 2.3?
In-Reply-To: <16677.4152.404288.108320@montanaro.dyndns.org>
Message-ID: <001701c4863f$537bcb20$e841fea9@oemcomputer>

> The docstring for the decimal module says in part:
> 
>   This is a Py2.3 implementation of decimal floating point arithmetic
> based on
>   the General Decimal Arithmetic Specification:
> 
> Is it intended to remain 2.3-compatible for the forseeable future?
It's
> not
> mentioned in PEP 291, nor is there any discussion of portability to
2.3 in
> PEP 327.

I've added module comments to include an update policy and will update
the PEP 291 to match.  Nothing in Py2.4 warrants giving up 2.3
compatibility, so 2.3 will be kept at least until Py2.5.



> I ask because we're at 2.3.4 at work, but could use it today.  I'd
hate to
> have to track it to backport fixes though.  It would be ideal if I
could
> simply copy the 2.4 version as it matures.

Yes, that was the intention behind the 2.3 compatibility code I put in.
Also, I will keep testing it under 2.3.

Also, the policy states that spec updates will be treated as bugfixes
and backported.  There should be very few of these and they are likely
to not be major.  There were a couple of big changes on the table but
Mike Cowlishaw advised me that these are now very unlikely.


Raymond

From bac at OCF.Berkeley.EDU  Fri Aug 20 01:12:54 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug 20 01:13:01 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1092954772.8712.147.camel@geddy.wooz.org>
References: <1092177771.21839.54.camel@localhost>
	<cfd23o$lla$1@sea.gmane.org>	<59e9fd3a040811052152548b6b@mail.gmail.com>	<1092232683.9945.26.camel@localhost>	<1f7befae040811221916a955fd@mail.gmail.com>
	<1092954772.8712.147.camel@geddy.wooz.org>
Message-ID: <412533F6.4000703@ocf.berkeley.edu>

Barry Warsaw wrote:

> On Thu, 2004-08-12 at 01:19, Tim Peters wrote:
> 
> 
>>I do object to this part:
>>
>>     If the $ character appears at the end of the line, or is followed
>>     by any other character than those described above, it is treated
>>     as if it had been escaped, appearing in the resulting string
>>     unchanged.
>>
>>There's already a facility to escape $, and it's dead easy to use.  $
>>isn't a frequently needed character either in most apps.  So escaping
>>$ "by magic" too is, I think, more likely to make typing errors harder
>>to find than to do anyone real good.
> 
> 
> What would you do about $'s at the end of the string?  I think the
> implementation would be much more complex if you didn't have this rule,
> e.g. you'd have to match \$$ and the $-placeholder regexps would
> basically have to match everything.  Then the .sub call would have to be
> more complex too, because it would have to check for the existence of
> those match groups and then raise an exception.  Or something like that.
> 

Got an implementation and it's simple.  =)  It only required one 
additional group at the end (r"(?<!\$)(\$)$") in the regex and a test if 
it matched.

I tested against "$$" to be fine but for "blah $" and "$" to raise an 
exception.  Those all work and the tests in test_pep292 all pass.

If Barry is okay with this I can apply the patch and update the tests 
and the PEP.  Do need to know what exception to raise when this does 
occur, though.  ValueError?

-Brett
From barry at python.org  Fri Aug 20 01:17:06 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 01:16:41 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <007301c480b1$0867bec0$5229c797@oemcomputer>
References: <007301c480b1$0867bec0$5229c797@oemcomputer>
Message-ID: <1092957426.8714.182.camel@geddy.wooz.org>

On Thu, 2004-08-12 at 17:12, Raymond Hettinger wrote:

> After more thought, I would like to voice a strong -1 on the packaging
> of string.

Since few people have commented either way, and taking myself out of the
picture, it comes down to a battle royale between you and Brett. :)

Seriously, I'm willing to drop this non-PEP'd suggestion, since it just
makes my life easier, however see below.

> At the outset, I felt that it introduced unnecessary complexity.  Adding
> a new directory, an __init__.py, and two modules just to add a few lines
> of code is overkill.  Package logic should be reserved for organizing
> voluminous code like the email package.  Arguably, the logging package
> should have been just a single module -- putting it in a package just
> made it more difficult to read and maintain.

I think that logging would have been much more readable if more stuff
were taken out of its __init__.py and put into sub-package modules.  I
happen to like the way email does it, and I've come to believe that
package authors should strive for minimal __init__.py's, at most
exposing the public interface for the package but keeping code to a
minimum.  That's besides the point though.

> To test the existing sandbox code, I moved it into the library and found
> a circular import issue:  importing the string package, causes
> template.py to be imported, which in turn imports re which uses
> sre_parse.py, needs to import string.  So, the whole thing fails.

Something's wacky with either your, or my setup then:

Python 2.4a2 (#1, Aug 18 2004, 00:08:12)
[GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)] on
linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from string import Template
[25426 refs]

I'd like to know if anybody else has tried to wedge the sandbox
implementation into their Python tree, and whether it worked or not.  If
you do, don't forget to remove your old string.py and string.pyc files.

> My recommendation:  Create a stringtools module with the new template
> stuff in it and with room for growth.  Leave string.py alone and let it
> die in peace.

As I told Raymond in pvt email, I'm really not happy with creating
another string module.  I hate stringlib.py and stringtools.py seems too
contrived.  I'm putting this as an open issue in the PEP's next rev, but
to me the most logical place to put the Template and SafeTemplate
classes is in the string module.  So if we /don't/ reorganize string,
then I propose to just drop these two classes in string.py.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/59098ad9/attachment.pgp
From python at rcn.com  Fri Aug 20 01:22:09 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug 20 01:22:31 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1092954772.8712.147.camel@geddy.wooz.org>
Message-ID: <001f01c48643$5985faa0$e841fea9@oemcomputer>

[Tim]
> > I do object to this part:
> >
> >      If the $ character appears at the end of the line, or is
followed
> >      by any other character than those described above, it is
treated
> >      as if it had been escaped, appearing in the resulting string
> >      unchanged.
> >
> > There's already a facility to escape $, and it's dead easy to use.
$
> > isn't a frequently needed character either in most apps.  So
escaping
> > $ "by magic" too is, I think, more likely to make typing errors
harder
> > to find than to do anyone real good.

[Barry]
> What would you do about $'s at the end of the string?  I think the
> implementation would be much more complex if you didn't have this
rule,
> e.g. you'd have to match \$$ and the $-placeholder regexps would
> basically have to match everything.  Then the .sub call would have to
be
> more complex too, because it would have to check for the existence of
> those match groups and then raise an exception.  Or something like
that.

FWIW, Cheetah implements the behavior as described by Barry.

    IDLE 1.1a2      
    >>> from Cheetah.Template import Template
    >>> t = Template('the $rank is $@ a $$ dead ${rank}! $')
    >>> t.rank='king'
    >>> print t
    the king is $@ a $$ dead king! $

I think "escaping by magic" is the operational description.  A better
way to think of it is that only well formed substitutions are made.


Raymond

From python at rcn.com  Fri Aug 20 01:39:16 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug 20 01:39:38 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <1092957426.8714.182.camel@geddy.wooz.org>
Message-ID: <002401c48645$bdf482c0$e841fea9@oemcomputer>

> Since few people have commented either way, and taking myself out of
the
> picture, it comes down to a battle royale between you and Brett. :)

Hmm.  Brett is taller.  I'm better looking.
He's almost a master of computer science.  I'm just old.
He's needs a summer job to eat.  I've got a retirement account.

Perhaps, I can win him over through guile:

1.  Call up IDLE, then press File, Open Module, string.   Look at the
source.  Now, install your package and try again.  No source.  

2.  Run "cvs ann string.py" or "cvs log string.py" and look at who did
what and when.  Now, install the package and try again.  No history.

3.  Look at the new string docs, decide that you want to subclass
string.Template and want to look at the existing pattern.  Time how long
it takes you to find the source (change to the string subdirectory, look
in __init__.py, learn the template.py is the real source, yada yada
yada).



> > To test the existing sandbox code, I moved it into the library and
found
> > a circular import issue:  importing the string package, causes
> > template.py to be imported, which in turn imports re which uses
> > sre_parse.py, needs to import string.  So, the whole thing fails.
> 
> Something's wacky with either your, or my setup then:
> 
> Python 2.4a2 (#1, Aug 18 2004, 00:08:12)
> [GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)]
on
> linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from string import Template
> [25426 refs]
> 
> I'd like to know if anybody else has tried to wedge the sandbox
> implementation into their Python tree, and whether it worked or not.
If
> you do, don't forget to remove your old string.py and string.pyc
files.


How could this work?
Template has an re.compile that runs upon import.
The sre_parse module has an "import string" that runs when imported.


Raymond

From xscottg at yahoo.com  Fri Aug 20 02:02:04 2004
From: xscottg at yahoo.com (Scott Gilbert)
Date: Fri Aug 20 02:02:07 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <59e9fd3a040819154158b60833@mail.gmail.com>
Message-ID: <20040820000204.12671.qmail@web50201.mail.yahoo.com>


Please count a humble vote for list after def.

The following is simple:

    def foo(self, a, b) [classmethod]:
        pass

And the pathological cases can be made more readable with a little
formatting:

    def longMethodNameForEffect(
        longArgumentOne=None,
        longArgumentTwo=42
    )[
        staticmethod,
        funcattrs(
            grammar="'@' dotted_name [ '(' [arglist] ')' ]",
            status="experimental", author="BDFL"
        )
    ]:
        """This method blah, blah.

        It supports the following arguments:
        - longArgumentOne -- a string giving ...
        - longArgumentTwo -- a number giving ...

        blah, blah.

        """
        raise NotYetImplemented



Back to lurking...




From guido at python.org  Fri Aug 20 02:07:36 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug 20 02:07:44 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: Your message of "Fri, 20 Aug 2004 08:41:47 +1000."
	<59e9fd3a040819154158b60833@mail.gmail.com> 
References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz>
	<200408190558.i7J5wPP19708@guido.python.org>
	<200408191158.12738.gmccaughan@synaptics-uk.com> 
	<59e9fd3a040819154158b60833@mail.gmail.com> 
Message-ID: <200408200007.i7K07av21619@guido.python.org>

> I agree with Gareth on both of his main points: I didn't buy Guido's
> rejection of list-after-def (and so argued against his points in
> another thread here); but the decorator suite before the def (J2) is
> quite nice also, and it doesn't hide the def when there are lots of
> decorators (due to the indentation). I would strongly support either
> proposal over the current @decorators.

So pick a favorite already and argue with the folks on c.l.py about
which one will be presented to me.  I'm serious.  I don't want a
hundred people arguing their separate points with me.  If that's all I
get the current syntax stays.  If the community can *agree* on
something they can support, I will listen.  But it has to be a unified
voice.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Fri Aug 20 02:08:56 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug 20 02:09:03 2004
Subject: [Python-Dev] decimal module portability to 2.3?
In-Reply-To: Your message of "Thu, 19 Aug 2004 17:47:26 EDT."
	<000901c48636$1eb2b240$e841fea9@oemcomputer> 
References: <000901c48636$1eb2b240$e841fea9@oemcomputer> 
Message-ID: <200408200008.i7K08uC21635@guido.python.org>

> By keeping decimal 2.3 compatible, folks can make periodic downloads
> to take advantage of bug fixes and efforts to keep it in sync with
> the on-going evolution of the spec.

+1

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Fri Aug 20 02:17:36 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug 20 02:17:42 2004
Subject: [Python-Dev] PEP 318 has ReST errors
Message-ID: <200408200017.i7K0HaT21703@guido.python.org>

The cronjob on my home machine that formats the PEPs 4x daily
complains about syntax errors in PEP 318.  Last time that happened I
fixed them all, but today I don't have time.  Can somebody with
checkin perms and ReST foo please fix these?

pep-0318.txt:330: (ERROR/3) Unexpected indentation.
pep-0318.txt:336: (ERROR/3) Unexpected indentation.
pep-0318.txt:351: (WARNING/2) Inline substitution_reference start-string without end-string.
pep-0318.txt:106: (ERROR/3) Unknown target name: "java-style".
pep-0318.txt:132: (ERROR/3) Unknown target name: "gof book".
pep-0318.txt:169: (ERROR/3) Unknown target name: "in your face".
pep-0318.txt:218: (ERROR/3) Unknown target name: "a large number".
pep-0318.txt:218: (ERROR/3) Unknown target name: "each possible syntax".
pep-0318.txt:288: (ERROR/3) Unknown target name: "summarised the arguments".
pep-0318.txt:507: (ERROR/3) Unknown target name: "annotations".

--Guido van Rossum (home page: http://www.python.org/~guido/)
From skip at pobox.com  Fri Aug 20 02:40:33 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug 20 02:41:18 2004
Subject: [Python-Dev] decimal module portability to 2.3?
In-Reply-To: <000801c48635$46986e40$e841fea9@oemcomputer>
References: <A128D751272CD411BC9200508BC2194D053C7815@escpl.tcp.com.ar>
	<000801c48635$46986e40$e841fea9@oemcomputer>
Message-ID: <16677.18561.746876.718204@montanaro.dyndns.org>


    Raymond> Yes, I've been keeping it Py2.3 compatible and periodically
    Raymond> test it on 2.3 just to make sure.

    Raymond> Am hesitant to introduce this as a specific, on-going
    Raymond> restriction to the code, but I don't want to throw-away 2.3
    Raymond> compatibility unless there is a darned good offsetting gain.

Raymond,

Given that you've expended effort to keep decimal 2.3-compatible I sort of
doubt at this point that something will be done to make it incompatible with
2.3 before 2.4 is released.

Might I suggest an addition to PEP 291 like this:

    Package/Module     Maintainer(s)          Python Version     Notes
    --------------     -------------          --------------     -----
    ...
    decimal             Raymond, et al          2.3               [2]
    ...

with note 2 being something like:

    Compatibility with 2.3 maintained until at least 2.5 is released.

The thought there is that after 2.4 is released no new language features or
modules should be added to 2.4.n (n > 0) which you would want to use in the
decimal module.  If 2.4 is released with a decimal module that is
2.3-compatible, maintaining that compatibility until 2.5 is released
shouldn't be too difficult.

The above would satisfy me.

Thx,

Skip

From tim.peters at gmail.com  Fri Aug 20 03:10:56 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug 20 03:11:04 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1092954772.8712.147.camel@geddy.wooz.org>
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost>
	<1f7befae040811221916a955fd@mail.gmail.com>
	<1092954772.8712.147.camel@geddy.wooz.org>
Message-ID: <1f7befae0408191810474dc3de@mail.gmail.com>

[Tim]
>> I do object to this part:
>>
>>      If the $ character appears at the end of the line, or is followed
>>      by any other character than those described above, it is treated
>>      as if it had been escaped, appearing in the resulting string
>>      unchanged.
>>
>> There's already a facility to escape $, and it's dead easy to use.  $
>> isn't a frequently needed character either in most apps.  So escaping
>> $ "by magic" too is, I think, more likely to make typing errors harder
>> to find than to do anyone real good.

[Barry]
> What would you do about $'s at the end of the string?

Celebrate them, I suppose, if they're somehow better than $'s
elsewhere in the string <wink>.

> I think the implementation would be much more complex if you didn't have this
> rule, e.g. you'd have to match \$$ and the $-placeholder regexps would
> basically have to match everything.  Then the .sub call would have to be
> more complex too, because it would have to check for the existence of
> those match groups and then raise an exception.  Or something like that.

Na.  Add a 4th group to the regexp:

   |(\$)

That will reliably match any $, anywhere, that one of the first three
groups doesn't know what to do with.  Then add ", illegal" to the end
of the LHS of the line that unpacks the groups, and then, e.g.,

   if illegal:
       raise ValueError("unsubstitutable $ at index %d" % mo.start(4))

In return for that new line, I'll never have to worry about typing
"${barry)-rulz!" by mistake (depending on the font you're using, you
may or may not have a hard time seeing the typo in that -- and
assuming I didn't intend "${barry)-rulz!" to be the output too, which
it as things are now).
From andymac at bullseye.apana.org.au  Fri Aug 20 00:17:38 2004
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Fri Aug 20 03:13:44 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	compile.c, 2.319, 2.320
In-Reply-To: <1f7befae04081911192336c3b8@mail.gmail.com>
References: <000f01c48590$866705e0$e841fea9@oemcomputer>
	<200408190407.i7J47Ar19512@guido.python.org>
	<uu0uzowzg.fsf@boost-consulting.com>
	<1f7befae04081911192336c3b8@mail.gmail.com>
Message-ID: <20040820081535.X98652@bullseye.apana.org.au>

On Thu, 19 Aug 2004, Tim Peters wrote:

> It *used* to apply, and for a similar reason, when -O also controlled
> whether SET_LINENO opcodes were generated, and the Python debugger was
> blind without them.  That's no longer the case.  The only reason to
> avoid -O now is to retain
>
>     if __debug__:
>
> blocks and (same thing) assert statements.  That can make a big speed
> difference in Zope and ZEO, for example.  Life would definitely be
> worse if -O could introduce new bugs (other than programmer errors of
> putting essential code inside __debug__ blocks).

Seems to me that in this context -O would be more appropriately renamed to
-D to reflect the debug relationship...

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac@pcug.org.au             (alt) |        Belconnen  ACT  2616
Web:    http://www.andymac.org/               |        Australia
From goodger at python.org  Fri Aug 20 03:22:37 2004
From: goodger at python.org (David Goodger)
Date: Fri Aug 20 03:22:39 2004
Subject: [Python-Dev] Re: PEP 318 has ReST errors
In-Reply-To: <200408200017.i7K0HaT21703@guido.python.org>
References: <200408200017.i7K0HaT21703@guido.python.org>
Message-ID: <4125525D.5060902@python.org>

[Guido van Rossum]
> Can somebody with checkin perms and ReST foo please fix these?

Done.

-- 
David Goodger <http://python.net/~goodger>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040819/e8b9a386/signature.pgp
From greg at cosc.canterbury.ac.nz  Fri Aug 20 03:40:15 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug 20 03:40:21 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408200007.i7K07av21619@guido.python.org>
Message-ID: <200408200140.i7K1eFZh027162@cosc353.cosc.canterbury.ac.nz>

Guido:

> If the community can *agree* on
> something they can support, I will listen.

Even if it's []-after-args?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From neal at metaslash.com  Fri Aug 20 04:09:19 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Fri Aug 20 04:09:27 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules
	socketmodule.c, 1.298, 1.299
In-Reply-To: <E1Bu28N-0005RW-SO@sc8-pr-cvs1.sourceforge.net>
References: <E1Bu28N-0005RW-SO@sc8-pr-cvs1.sourceforge.net>
Message-ID: <20040820020919.GG31470@epoch.metaslash.com>

> Log Message:
> Patch #1003700: Add socketpair function to socket module.
> 
> Index: socketmodule.c

> + #ifdef HAVE_SOCKETPAIR
> + /* Create a pair of sockets using the socketpair() function.
> +    Arguments as for socket(). */
> + 
> + /*ARGSUSED*/
> + static PyObject *
> + socket_socketpair(PyObject *self, PyObject *args)
> + {
> + 	PySocketSockObject *s0 = NULL, *s1 = NULL;
> + 	SOCKET_T sv[2];
> + 	int family, type = SOCK_STREAM, proto = 0;
> + 	PyObject *res = NULL;
> + 
> + #if defined(AF_UNIX)
> + 	family = AF_UNIX;
> + #else
> + 	family = AF_INET;
> + #endif

The docstring (below) states the arguments are the same as socket().
However, in sock_initobj() line 2496, the family is initialized to
AF_INET.  I think the #if defined(AF_UNIX) code above should be
removed and family should be initialized to AF_INET.

I didn't look to see if the documentation agrees with the docstring.

> + 	if (!PyArg_ParseTuple(args, "|iii:socketpair",
> + 			      &family, &type, &proto))
> + 		return NULL;
> + 	/* Create a pair of socket fds */
> + 	if (socketpair(family, type, proto, sv) < 0)
> + 		return set_error();
> + #ifdef SIGPIPE
> + 	(void) signal(SIGPIPE, SIG_IGN);
> + #endif

I don't think the #ifdef SIGPIPE code is correct.  If the user
installed a signal handler calling signal() will remove it.  I 
think the call to signal() should be removed.

Neal

> + 	s0 = new_sockobject(sv[0], family, type, proto);
> + 	if (s0 == NULL)
> + 		goto finally;
> + 	s1 = new_sockobject(sv[1], family, type, proto);
> + 	if (s1 == NULL)
> + 		goto finally;
> + 	res = PyTuple_Pack(2, s0, s1);
> + 
> + finally:
> + 	if (res == NULL) {
> + 		if (s0 == NULL)
> + 			SOCKETCLOSE(sv[0]);
> + 		if (s1 == NULL)
> + 			SOCKETCLOSE(sv[1]);
> + 	}
> + 	Py_XDECREF(s0);
> + 	Py_XDECREF(s1);
> + 	return res;
> + }
> + 
> + PyDoc_STRVAR(socketpair_doc,
> + "socketpair([family[, type[, proto]]]) -> (socket object, socket object)\n\
> + \n\
> + Create a pair of socket objects from the sockets returned by the platform\n\
> + socketpair() function.\n\
> + The arguments are the same as for socket().");
> + 
> + #endif /* HAVE_SOCKETPAIR */
From bac at OCF.Berkeley.EDU  Fri Aug 20 04:10:44 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug 20 04:10:45 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <002401c48645$bdf482c0$e841fea9@oemcomputer>
References: <002401c48645$bdf482c0$e841fea9@oemcomputer>
Message-ID: <41255DA4.1070002@ocf.berkeley.edu>

Raymond Hettinger wrote:
>>Since few people have commented either way, and taking myself out of
> 
> the
> 
>>picture, it comes down to a battle royale between you and Brett. :)
> 
> 
> Hmm.  Brett is taller.  I'm better looking.

Don't let the photos from PyCON fool you; I never have a good look on my 
face and I tend to look stoned.

> He's almost a master of computer science.  I'm just old.

I hope you are not trying to use some "wiser" argument with that.  =)

> He's needs a summer job to eat.  I've got a retirement account.
> 

Got me there.

> Perhaps, I can win him over through guile:
> 
> 1.  Call up IDLE, then press File, Open Module, string.   Look at the
> source.  Now, install your package and try again.  No source.  
> 

Yeah, but you will notice the directory and realize it is a package so 
look in there.

> 2.  Run "cvs ann string.py" or "cvs log string.py" and look at who did
> what and when.  Now, install the package and try again.  No history.
> 

Eh.  That module has been only changed to keep up with the built-ins or 
tweak a docstring.  Not exactly major stuff I feel bad about losing some 
commit messages over.

> 3.  Look at the new string docs, decide that you want to subclass
> string.Template and want to look at the existing pattern.  Time how long
> it takes you to find the source (change to the string subdirectory, look
> in __init__.py, learn the template.py is the real source, yada yada
> yada).
> 

Only reason that could be considered an issue is because the __init__.py 
file has the constants and everything else that is not deprecated in there.



Basically I can live with having a single string module, but I would 
like to see some real deprecation happen.  The nice thing about the 
pacakge separation is it made it clear in the code what would be going 
away.  If we at least at PendingDeprecation to what is supposed to be 
taken out I will be happy.

-Brett
From anthony at interlink.com.au  Fri Aug 20 05:25:27 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug 20 05:25:31 2004
Subject: [Python-Dev] decimal module portability to 2.3?
In-Reply-To: <A128D751272CD411BC9200508BC2194D053C7815@escpl.tcp.com.ar>
References: <A128D751272CD411BC9200508BC2194D053C7815@escpl.tcp.com.ar>
Message-ID: <41256F27.8090602@interlink.com.au>

Batista, Facundo wrote:
> Could decimal be backported to 2.3 and be included in some 2.3.5? Don't know
> if this behaviour is common to Python.

Absolutely not. See PEP 0006. There's nothing stopping someone from
making a seperate package of the decimal code (complete with setup.py)
and distributing it for Python 2.3 users to download and add on. Barry
does this with the email package.

From python at rcn.com  Fri Aug 20 05:33:25 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug 20 05:33:52 2004
Subject: [Python-Dev] decimal module portability to 2.3?
In-Reply-To: <16677.18561.746876.718204@montanaro.dyndns.org>
Message-ID: <003e01c48666$73afbe20$e841fea9@oemcomputer>

>     Raymond> Yes, I've been keeping it Py2.3 compatible and
periodically
>     Raymond> test it on 2.3 just to make sure.
> 
>     Raymond> Am hesitant to introduce this as a specific, on-going
>     Raymond> restriction to the code, but I don't want to throw-away
2.3
>     Raymond> compatibility unless there is a darned good offsetting
gain.
> 
> Raymond,
> 
> Given that you've expended effort to keep decimal 2.3-compatible I
sort of
> doubt at this point that something will be done to make it
incompatible
> with
> 2.3 before 2.4 is released.
> 
> Might I suggest an addition to PEP 291 like this:
> 
>     Package/Module     Maintainer(s)          Python Version     Notes
>     --------------     -------------          --------------     -----
>     ...
>     decimal             Raymond, et al          2.3               [2]
>     ...
> 
> with note 2 being something like:
> 
>     Compatibility with 2.3 maintained until at least 2.5 is released.
> 
> The thought there is that after 2.4 is released no new language
features
> or
> modules should be added to 2.4.n (n > 0) which you would want to use
in
> the
> decimal module.  If 2.4 is released with a decimal module that is
> 2.3-compatible, maintaining that compatibility until 2.5 is released
> shouldn't be too difficult.
> 
> The above would satisfy me.

Done.




Raymond

From anthony at interlink.com.au  Fri Aug 20 05:34:39 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug 20 05:34:42 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408200007.i7K07av21619@guido.python.org>
References: <200408190455.i7J4t6CN025433@cosc353.cosc.canterbury.ac.nz>	<200408190558.i7J5wPP19708@guido.python.org>	<200408191158.12738.gmccaughan@synaptics-uk.com>
	<59e9fd3a040819154158b60833@mail.gmail.com>
	<200408200007.i7K07av21619@guido.python.org>
Message-ID: <4125714F.4000709@interlink.com.au>

Guido van Rossum wrote:
> So pick a favorite already and argue with the folks on c.l.py about
> which one will be presented to me.  I'm serious.  I don't want a
> hundred people arguing their separate points with me.  If that's all I
> get the current syntax stays.  If the community can *agree* on
> something they can support, I will listen.  But it has to be a unified
> voice.

More importantly, _don't_ go either collecting or posting votes on this
issue to python-dev. If all you have to add is either "+1 to this" or
restating prior arguments, _use_ _comp.lang.python_. I'd suggest the
thread "Alternative decorator syntax decision" in that group.

Anthony
From skip at pobox.com  Fri Aug 20 06:14:13 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Aug 20 06:14:25 2004
Subject: [Python-Dev] decimal module portability to 2.3?
In-Reply-To: <003e01c48666$73afbe20$e841fea9@oemcomputer>
References: <16677.18561.746876.718204@montanaro.dyndns.org>
	<003e01c48666$73afbe20$e841fea9@oemcomputer>
Message-ID: <16677.31381.599608.540220@montanaro.dyndns.org>


    >> Might I suggest an addition to PEP 291 like this:
    ...
    >> 
    >> Package/Module     Maintainer(s)          Python Version     Notes
    >> --------------     -------------          --------------     -----
    >> ...
    >> decimal             Raymond, et al          2.3               [2]
    >> ...
    ...
    Raymond> Done.

It would appear Raymond has his own time machine.  I was responding to email
as I read it.  About five messages further into my python-dev mailbox I saw
his checkin message...

Skip
From python at rcn.com  Fri Aug 20 06:17:12 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug 20 06:17:39 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1f7befae0408191810474dc3de@mail.gmail.com>
Message-ID: <004501c4866c$919ea620$e841fea9@oemcomputer>



> -----Original Message-----
> From: python-dev-bounces+python=rcn.com@python.org [mailto:python-dev-
> [Tim]
> >> I do object to this part:
> >>
> >>      If the $ character appears at the end of the line, or is
followed
> >>      by any other character than those described above, it is
treated
> >>      as if it had been escaped, appearing in the resulting string
> >>      unchanged.
> >>
> >> There's already a facility to escape $, and it's dead easy to use.
$
> >> isn't a frequently needed character either in most apps.  So
escaping
> >> $ "by magic" too is, I think, more likely to make typing errors
harder
> >> to find than to do anyone real good.

. . .

raise ValueError("unsubstitutable $ at index %d" % mo.start(4))
> 
> In return for that new line, I'll never have to worry about typing
> "${barry)-rulz!" by mistake (depending on the font you're using, you
> may or may not have a hard time seeing the typo in that -- and
> assuming I didn't intend "${barry)-rulz!" to be the output too, which
> it as things are now).

That was a good example.

However, I think we should follow Cheetah's example.  That's makes it
less of a pita when the template naturally contains dollar signs with
numbers:

Template("Dear ${donor}, send $100.00 to the PSF template fund.")



Raymond


From tim.peters at gmail.com  Fri Aug 20 06:34:31 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug 20 06:34:34 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <004501c4866c$919ea620$e841fea9@oemcomputer>
References: <004501c4866c$919ea620$e841fea9@oemcomputer>
Message-ID: <1f7befae0408192134734b6a38@mail.gmail.com>

[Raymond Hettinger]
> That was a good example.
>
> However, I think we should follow Cheetah's example.

Sorry, I never heard of Cheetah before you mentioned it (& still don't
know what it is), so feel no pressure to ape it.

> That's makes it less of a pita when the template naturally contains
> dollar signs with numbers:

I don't care.  The minority of Americans who want to use $ in a
minority of templated strings in a minority of apps can learn to type
$$, or use the ironically named SafeTemplate instead (as suggested
earlier).

> Template("Dear ${donor}, send $100.00 to the PSF template fund.")

Which would trigger an exception, after which they type one character
to repair it:

Template("Dear ${donor}, send $$100.00 to the PSF template fund.")

They'll live, but not at everyone else's expense.  They're already
confused, or they wouldn't have bothered with the unnecessary braces
around "donor" <wink>.
From andrewm at object-craft.com.au  Fri Aug 20 06:44:36 2004
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Fri Aug 20 06:44:40 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules
	socketmodule.c, 1.298, 1.299 
In-Reply-To: <20040820020919.GG31470@epoch.metaslash.com> 
References: <E1Bu28N-0005RW-SO@sc8-pr-cvs1.sourceforge.net>
	<20040820020919.GG31470@epoch.metaslash.com>
Message-ID: <20040820044436.0F8503C113@coffee.object-craft.com.au>

>> Patch #1003700: Add socketpair function to socket module.
>
>The docstring (below) states the arguments are the same as socket().
>However, in sock_initobj() line 2496, the family is initialized to
>AF_INET.  I think the #if defined(AF_UNIX) code above should be
>removed and family should be initialized to AF_INET.

I talked Dave into this - AF_UNIX is typically the only address family
that is valid for socketpair(). Using AF_INET under linux and OS X
results in EOPNOTSUPP.

>I don't think the #ifdef SIGPIPE code is correct.  If the user
>installed a signal handler calling signal() will remove it.  I 
>think the call to signal() should be removed.

I agree, but I think that was copied verbatim from elsewhere in
socketmodule.c, so it was left with the aim of being bug for bug
compatible with socket.socket().

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/
From djc at object-craft.com.au  Fri Aug 20 07:06:21 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Fri Aug 20 07:06:34 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Modules	socketmodule.c, 1.298, 1.299
In-Reply-To: <20040820044436.0F8503C113@coffee.object-craft.com.au>
References: <E1Bu28N-0005RW-SO@sc8-pr-cvs1.sourceforge.net>	<20040820020919.GG31470@epoch.metaslash.com>
	<20040820044436.0F8503C113@coffee.object-craft.com.au>
Message-ID: <412586CD.2000000@object-craft.com.au>

Andrew McNamara wrote:
>>>Patch #1003700: Add socketpair function to socket module.
>>
>>The docstring (below) states the arguments are the same as socket().
>>However, in sock_initobj() line 2496, the family is initialized to
>>AF_INET.  I think the #if defined(AF_UNIX) code above should be
>>removed and family should be initialized to AF_INET.
> 
> 
> I talked Dave into this - AF_UNIX is typically the only address family
> that is valid for socketpair(). Using AF_INET under linux and OS X
> results in EOPNOTSUPP.

I wondered about the AF_UNIX/AF_INET thing.  When looking through the 
code I noticed that all references to AF_UNIX were #ifdef enclosed.  I 
decided to allow for the possibility of socketpair() support on a 
platform that supports AF_INET but not AF_UNIX (no idea if such a thing 
exists).  The other alternative could have been to conditional include 
socketpair() only if both AF_UNIX and HAVE_SOCKETPAIR are defined.

Another thing to consider is the prior art in the form of eunuchs.  The 
implementation of socketpair() in eunuchs defaults to AF_UNIX.  Code 
using eunuchs is probably assuming AF_UNIX - not a bad assumption really.

Would a change to the documentation and docstring be sufficient it it 
explained that the default family is AF_UNIX if defined for the 
platform, AF_INET otherwise?

>>I don't think the #ifdef SIGPIPE code is correct.  If the user
>>installed a signal handler calling signal() will remove it.  I 
>>think the call to signal() should be removed.
> 
> 
> I agree, but I think that was copied verbatim from elsewhere in
> socketmodule.c, so it was left with the aim of being bug for bug
> compatible with socket.socket().

Indeed.  I was not going to change something as fundamental as the 
existing signal handling.  The onyl sensible thing to do was to copy the 
behaviour already in place when you create a socket the "normal" way.

-- 
http://www.object-craft.com.au
From lists at hlabs.spb.ru  Fri Aug 20 12:22:55 2004
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Fri Aug 20 08:16:34 2004
Subject: [Python-Dev] python-dev Summary for 2004-08-01 through 2004-08-15
	[draft]
In-Reply-To: <4124506F.6040503@ocf.berkeley.edu>
References: <4124506F.6040503@ocf.berkeley.edu>
Message-ID: <4125D0FF.3080307@hlabs.spb.ru>

Brett C. wrote:
> --------------------------------------
> Is an int/long combined type worth it?
> --------------------------------------
> language evolution
> 
> Dmitry Vasiliev pointed out that `PEP 237`_ (Unifying Long Integers and 
> Integers) mentioned that a a new type named 'integer' might be 
> introduced that subclassed both int and long.  The discussion waivered 
> between whether it was at all needed, and if it was if it should be a 
> true type or just a tuple containing both types for use with isinstance() .
> 
> No conclusion was reached in the end.
> 
> .. _PEP 237: http://www.python.org/peps/pep-0237.html
> 
> Contributing threads:
>   - `Unifying Long Integers and Integers: baseint 
> <http://mail.python.org/pipermail/python-dev/2004-August/047404.html>`__

The discussion has been closed at:

http://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=1007068

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru
From barry at python.org  Fri Aug 20 16:16:03 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 16:15:34 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <41255DA4.1070002@ocf.berkeley.edu>
References: <002401c48645$bdf482c0$e841fea9@oemcomputer>
	<41255DA4.1070002@ocf.berkeley.edu>
Message-ID: <1093011363.8761.57.camel@geddy.wooz.org>

On Thu, 2004-08-19 at 22:10, Brett C. wrote:

> Don't let the photos from PyCON fool you; I never have a good look on my 
> face and I tend to look stoned.

Just "look"?  Dang, I really thought Tim, Fred, and I had a new recruit
for our CrackPython project.  The first one's free, y'know.

> Basically I can live with having a single string module, but I would 
> like to see some real deprecation happen.  The nice thing about the 
> pacakge separation is it made it clear in the code what would be going 
> away.  If we at least at PendingDeprecation to what is supposed to be 
> taken out I will be happy.

At the very least, my rewrite of libstring.tex will make it clear which
inhabitants of the string module are going to be deprecated.  Besides,
since it's clear that Python 3000 will take a broader look at standard
library packagization, I'll drop this if we can agree that the PEP 292
classes should go in the existing string module.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/34b36756/attachment.pgp
From aahz at pythoncraft.com  Fri Aug 20 16:40:48 2004
From: aahz at pythoncraft.com (Aahz)
Date: Fri Aug 20 16:40:50 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <1092957426.8714.182.camel@geddy.wooz.org>
References: <007301c480b1$0867bec0$5229c797@oemcomputer>
	<1092957426.8714.182.camel@geddy.wooz.org>
Message-ID: <20040820144048.GB4952@panix.com>

On Thu, Aug 19, 2004, Barry Warsaw wrote:
> On Thu, 2004-08-12 at 17:12, Raymond Hettinger wrote:
>> 
>> After more thought, I would like to voice a strong -1 on the packaging
>> of string.
> 
> Since few people have commented either way, and taking myself out of the
> picture, it comes down to a battle royale between you and Brett. :)

I'm +1 on a package.  I'm a strong +0 on naming the package ``string``.

However, the reason I didn't speak up earlier was because I thought
Raymond had a good technical point about his failure in getting your
package to work, and I can't in good conscience support the package
until that's resolved.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From mwh at python.net  Fri Aug 20 17:03:14 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Aug 20 17:03:17 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1092955002.8714.149.camel@geddy.wooz.org> (Barry Warsaw's
	message of "Thu, 19 Aug 2004 18:36:42 -0400")
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost>
	<1f7befae040811221916a955fd@mail.gmail.com>
	<200408120539.i7C5dqL29226@guido.python.org>
	<59e9fd3a0408112305fa1b667@mail.gmail.com>
	<1092955002.8714.149.camel@geddy.wooz.org>
Message-ID: <2mbrh597il.fsf@starship.python.net>

Barry Warsaw <barry@python.org> writes:

> On Thu, 2004-08-12 at 02:05, Andrew Durdin wrote:
>> One thing the PEP doesn't make clear: do these templates only
>> interpolate string variables, or will they handle anything with a
>> __str__ method?
>> 
>> e.g., will the following raise an exception:
>> 
>> print Template("I am ${height} centimetres tall.") % {height: 183}
>
> In the current implementation, this will raise an exception.  

What, like a NameError for height? :-)

Sorry,
mwh

-- 
  I can't see a conspicuous evolutionary advantage in being good
  at higher mathematics.                           -- James Riden, asr
From barry at python.org  Fri Aug 20 17:05:27 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 17:04:59 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <001f01c48643$5985faa0$e841fea9@oemcomputer>
References: <001f01c48643$5985faa0$e841fea9@oemcomputer>
Message-ID: <1093014306.9254.3.camel@geddy.wooz.org>

On Thu, 2004-08-19 at 19:22, Raymond Hettinger wrote:

>     IDLE 1.1a2      
>     >>> from Cheetah.Template import Template
>     >>> t = Template('the $rank is $@ a $$ dead ${rank}! $')
>     >>> t.rank='king'
>     >>> print t
>     the king is $@ a $$ dead king! $

There's a neat idea buried in there: Templates taking an interpolation
mapping from their attributes.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/fe36fcea/attachment.pgp
From barry at python.org  Fri Aug 20 17:08:15 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 17:07:47 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <412533F6.4000703@ocf.berkeley.edu>
References: <1092177771.21839.54.camel@localhost>
	<cfd23o$lla$1@sea.gmane.org>	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost>
	<1f7befae040811221916a955fd@mail.gmail.com>
	<1092954772.8712.147.camel@geddy.wooz.org>
	<412533F6.4000703@ocf.berkeley.edu>
Message-ID: <1093014495.9258.6.camel@geddy.wooz.org>

On Thu, 2004-08-19 at 19:12, Brett C. wrote:

> Got an implementation and it's simple.  =)  It only required one 
> additional group at the end (r"(?<!\$)(\$)$") in the regex and a test if 
> it matched.
> 
> I tested against "$$" to be fine but for "blah $" and "$" to raise an 
> exception.  Those all work and the tests in test_pep292 all pass.

What do you think about this compared to Tim's pattern, which is more
strict than just complaining about $'s at the end of strings?

> If Barry is okay with this I can apply the patch and update the tests 
> and the PEP.  Do need to know what exception to raise when this does 
> occur, though.  ValueError?

ValueError seems right to me.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/131a7810/attachment.pgp
From barry at python.org  Fri Aug 20 17:11:11 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 17:10:43 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <2mbrh597il.fsf@starship.python.net>
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost>
	<1f7befae040811221916a955fd@mail.gmail.com>
	<200408120539.i7C5dqL29226@guido.python.org>
	<59e9fd3a0408112305fa1b667@mail.gmail.com>
	<1092955002.8714.149.camel@geddy.wooz.org>
	<2mbrh597il.fsf@starship.python.net>
Message-ID: <1093014670.9249.8.camel@geddy.wooz.org>

On Fri, 2004-08-20 at 11:03, Michael Hudson wrote:

> What, like a NameError for height? :-)

Heh.  :)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/3c98476a/attachment.pgp
From barry at python.org  Fri Aug 20 17:21:22 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 17:20:55 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <20040820144048.GB4952@panix.com>
References: <007301c480b1$0867bec0$5229c797@oemcomputer>
	<1092957426.8714.182.camel@geddy.wooz.org>
	<20040820144048.GB4952@panix.com>
Message-ID: <1093015282.9255.11.camel@geddy.wooz.org>

On Fri, 2004-08-20 at 10:40, Aahz wrote:

> I'm +1 on a package.  I'm a strong +0 on naming the package ``string``.
> 
> However, the reason I didn't speak up earlier was because I thought
> Raymond had a good technical point about his failure in getting your
> package to work, and I can't in good conscience support the package
> until that's resolved.

Did you try it?  I'm really interested in getting additional feedback,
because AFAIK only Raymond and I have.  It works for me, but doesn't for
him.

But if the package problem turns out to be real, we shouldn't support
it, or expend too much effort fixing it.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/73ea27df/attachment.pgp
From fumanchu at amor.org  Fri Aug 20 17:23:08 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Fri Aug 20 17:28:30 2004
Subject: [Python-Dev] Re: Update PEP 292
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E13@exchange.hqamor.amorhq.net>

Tim Peters wrote:
> I don't care.  The minority of Americans who want to use $ in a
> minority of templated strings in a minority of apps can learn to type
> $$, or use the ironically named SafeTemplate instead (as suggested
> earlier).
> 
> > Template("Dear ${donor}, send $100.00 to the PSF template fund.")
> 
> Which would trigger an exception, after which they type one character
> to repair it:
> 
> Template("Dear ${donor}, send $$100.00 to the PSF template fund.")

Yep. Just like they (I mean "I") have to remember to write:

td width="10%%"


FuManChu
From bac at OCF.Berkeley.EDU  Fri Aug 20 18:29:00 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug 20 18:28:58 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1093014495.9258.6.camel@geddy.wooz.org>
References: <1092177771.21839.54.camel@localhost>	
	<cfd23o$lla$1@sea.gmane.org>	<59e9fd3a040811052152548b6b@mail.gmail.com>	
	<1092232683.9945.26.camel@localhost>	
	<1f7befae040811221916a955fd@mail.gmail.com>	
	<1092954772.8712.147.camel@geddy.wooz.org>	
	<412533F6.4000703@ocf.berkeley.edu>
	<1093014495.9258.6.camel@geddy.wooz.org>
Message-ID: <412626CC.4050900@ocf.berkeley.edu>

Barry Warsaw wrote:

> On Thu, 2004-08-19 at 19:12, Brett C. wrote:
> 
> 
>>Got an implementation and it's simple.  =)  It only required one 
>>additional group at the end (r"(?<!\$)(\$)$") in the regex and a test if 
>>it matched.
>>
>>I tested against "$$" to be fine but for "blah $" and "$" to raise an 
>>exception.  Those all work and the tests in test_pep292 all pass.
> 
> 
> What do you think about this compared to Tim's pattern, which is more
> strict than just complaining about $'s at the end of strings?
> 

That is coming down to a question of convenience compared to 
explicitness.  Do we want "Give me $1,000,000 now!" to raise ValueError, 
or do we want to let it pass through?

"Explicit is better than implicit" is good since it will cut down on 
errors (which is what Tim is arguing for along with "Special cases 
aren't special enough to break the rules").

But Raymond is pushing for "practicality beats purity" which is more or 
less true, but I am starting to agree with Tim that dollar amounts in an 
interpolated string probably won't hapeen that often.  If anything the 
number will be what is interpolated in.  So going for a more explicit 
version is seeming to be the better solution.  Perk of the current 
implementation is that if someone wants a looser definition they can 
just change the regex themselves (you guys could even go as far as to 
break up the regex into individual class attributes and just have a 
property that does a '|'.join on them as a read method so it is even 
easier to just change the one group).

So I guess it's your call, Mr PEP Champion, on whether you want all 
individual, non-interpolating $ signs to lead to an error or just random 
orphaned ones at the end of the line.

-Brett

From tim.peters at gmail.com  Fri Aug 20 19:01:29 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug 20 19:01:35 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <412626CC.4050900@ocf.berkeley.edu>
References: <1092177771.21839.54.camel@localhost>	
	<cfd23o$lla$1@sea.gmane.org>	<59e9fd3a040811052152548b6b@mail.gmail.com>	
	<1092232683.9945.26.camel@localhost>	
	<1f7befae040811221916a955fd@mail.gmail.com>	
	<1092954772.8712.147.camel@geddy.wooz.org>	
	<412533F6.4000703@ocf.berkeley.edu>
	<1093014495.9258.6.camel@geddy.wooz.org>
	<412626CC.4050900@ocf.berkeley.edu>
Message-ID: <1f7befae04082010013adb0a85@mail.gmail.com>

[Brett]
> That is coming down to a question of convenience compared to
> explicitness.  Do we want "Give me $1,000,000 now!" to raise ValueError,
> or do we want to let it pass through?
>
> "Explicit is better than implicit" is good since it will cut down on
> errors (which is what Tim is arguing for along with "Special cases
> aren't special enough to break the rules").
>
> But Raymond is pushing for "practicality beats purity" which is more or
> less true,

It's far less true for non-Americans.  I know it took me several
decades to learn that "$ means currency" isn't an international
standard enforced by the death penalty -- although it should be
<wink>.

> but I am starting to agree with Tim that dollar amounts in an
> interpolated string probably won't hapeen that often.  If anything the
> number will be what is interpolated in.

And that's where "handy shortcuts" bite too.  "OK, I want to
interpolate the value of variable 'dollars'.  A $ sign on its own
means a $ sign, and to interpolate the value of 'dollars' I do
$dollars, so putting them those together I must need

    "Total: $$dollars"

".  Which doesn't do what they want, although

    "Total: $ $dollars"

might be close enough.  A *consistent* (if you want $ in the output,
you must have $$ in the input) rule may be surprising at first, but it
doesn't remain surprising.

> So going for a more explicit version is seeming to be the better solution.
> Perk of the current implementation is that if someone wants a looser definition
> they can just change the regex themselves (you guys could even go as far as to
> break up the regex into individual class attributes and just have a
> property that does a '|'.join on them as a read method so it is even
> easier to just change the one group).

We could indeed make this so complicated nobody will understand it
<wink>.  The use of named (as opposed to the current numbered)
capturing groups could make it a lot less dependent on details of how
the regexp happens to be written, though.

> So I guess it's your call, Mr PEP Champion, on whether you want all
> individual, non-interpolating $ signs to lead to an error or just random
> orphaned ones at the end of the line.

I still don't grasp why the end of the line is somehow special here. 
The real thrust of what I'm after is catching what will *certainly* be
common typos, like

    $(barry)
    ${barry)
    $[barry]
    $(barry}
    ${barry{
    ${barry

The specific gimmick I suggested flags those as errors, and flags an
odd number of trailing $;s too, but the latter isn't a kind of typo I
expect occurs in real life.
From python at rcn.com  Fri Aug 20 19:04:46 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug 20 19:05:09 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <1093015282.9255.11.camel@geddy.wooz.org>
Message-ID: <003e01c486d7$cbd5e5e0$e841fea9@oemcomputer>

> > However, the reason I didn't speak up earlier was because I thought
> > Raymond had a good technical point about his failure in getting your
> > package to work, and I can't in good conscience support the package
> > until that's resolved.
> 
> Did you try it?  I'm really interested in getting additional feedback,
> because AFAIK only Raymond and I have.  It works for me, but doesn't
for
> him.
> 
> But if the package problem turns out to be real, we shouldn't support
> it, or expend too much effort fixing it.

That part is not a package problem.  The circular import issue occurs
whether you put Template in string.py or in a string package.  Try a
quick experiment to confirm on your machine:

  Take your existing string.py, add a couple of lines:

       import re
       pattern = re.compile('bingo')

  Now, clear out your pyc files, start python and import re.

  Here's what I get:

C:\py24\Lib>python
Python 2.4a2 (#46, Aug 19 2004, 18:01:28) [MSC v.1200 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\PY24\lib\re.py", line 5, in ?
    from sre import *
  File "C:\PY24\lib\sre.py", line 98, in ?
    import sre_parse
  File "C:\PY24\lib\sre_parse.py", line 16, in ?
    import string, sys
  File "string.py", line 43, in ?
    pattern = re.compile("bingo")
AttributeError: 'module' object has no attribute 'compile'



Raymond

From python at rcn.com  Fri Aug 20 19:19:14 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug 20 19:19:39 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1f7befae0408192134734b6a38@mail.gmail.com>
Message-ID: <003f01c486d9$d1298180$e841fea9@oemcomputer>

> [Raymond Hettinger]
> > However, I think we should follow Cheetah's example.
> 
> Sorry, I never heard of Cheetah before you mentioned it (& still don't
> know what it is), so feel no pressure to ape it.

I'm surprised that we're introducing a new API into the standard library
and no one is showing the slightest interest in prior art.  Cheetah is
well thought out, fast, and mature (it's been through several evolutions
in the wild).  Also, the Cheetah designers started out by studying the
existing art from ASP, JSP, PHP, PSP, and such.  Templating and string
substituion is not a new, yet the discussion here has the flavor of
being thought up from scratch.



> > That's makes it less of a pita when the template naturally contains
> > dollar signs with numbers:
> 
> I don't care.  The minority of Americans who want to use $ in a
> minority of templated strings in a minority of apps can learn to type
> $$, or use the ironically named SafeTemplate instead (as suggested
> earlier).
> 
> > Template("Dear ${donor}, send $100.00 to the PSF template fund.")
> 
> Which would trigger an exception, after which they type one character
> to repair it:
> 
> Template("Dear ${donor}, send $$100.00 to the PSF template fund.")


FWIW, I'm fine with that.  It was a minor suggestion.  Either way works.

My thought was that simpler string substitutions were going to be
exposed to the end-user (non-programmers) and that they should not be
harassed unnecessarily.  However, these are easy errors to trap and
report back to the user -- the only question being how to assign a line
number to the exception (if I have a ten page user supplied template
with an error, it would be great to tell them where it is).



Raymond









> 
> They'll live, but not at everyone else's expense.  They're already
> confused, or they wouldn't have bothered with the unnecessary braces
> around "donor" <wink>.

From tim.peters at gmail.com  Fri Aug 20 19:50:41 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug 20 19:50:47 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <003f01c486d9$d1298180$e841fea9@oemcomputer>
References: <003f01c486d9$d1298180$e841fea9@oemcomputer>
Message-ID: <1f7befae04082010507d3c50d4@mail.gmail.com>

[Raymond Hettinger]
> I'm surprised that we're introducing a new API into the standard library
> and no one is showing the slightest interest in prior art.  Cheetah is
> well thought out, fast, and mature (it's been through several evolutions
> in the wild).  Also, the Cheetah designers started out by studying the
> existing art from ASP, JSP, PHP, PSP, and such.  Templating and string
> substituion is not a new, yet the discussion here has the flavor of
> being thought up from scratch.

The current discussion is also just the latest in years of discussion.
 I've had 25 years of using string-interpolation gimmicks in countless
contexts myself, and my preference is a reaction against the practical
problems I had with "too much magic" in them (e.g., I can't remember
the difference between {} vs (), so I know I'll make typos related to
that; I also know I'll have no trouble remembering "if you want $, say
$$", and despite that I'll rarely want $).

...

> FWIW, I'm fine with that.  It was a minor suggestion.  Either way works.

The question is which works better.  Let's make Guido decide <wink>.

> My thought was that simpler string substitutions were going to be
> exposed to the end-user (non-programmers) and that they should not be
> harassed unnecessarily.  However, these are easy errors to trap and
> report back to the user -- the only question being how to assign a line
> number to the exception (if I have a ten page user supplied template
> with an error, it would be great to tell them where it is).

Indeed!  match_object.string

contains the original template string, and

    match_object.start(4)

gives the index at which the unsubstitutable $ appeared.  A
string-relative line number can be computed from that, and/or
surrounding context can be inserted in the exception detail.
From jimjjewett at yahoo.com  Fri Aug 20 21:10:58 2004
From: jimjjewett at yahoo.com (Jim J Jewett)
Date: Fri Aug 20 21:11:01 2004
Subject: [Python-Dev] Reserved Characters
Message-ID: <20040820191058.51944.qmail@web50708.mail.yahoo.com>

Python does not currently reserve any characters 
for user extensions.

One of the objections to @decorators is that it 
will cause problems for applications that used 
'@', simply because of it was one of only three 
characters that *might* be available.

Newer String interpolation will also use up '$', 
which leaves only '?'.

Is it time to reserve a character to the user, or
at least to reserve some in certain contexts?  

Examples:  
"""Python will not use '!' except as part of '!='
""", 
or 
"""The '?' character is reserved for the user."""

-jJ



		
_______________________________
Do you Yahoo!?
Win 1 of 4,000 free domain names from Yahoo! Enter now.
http://promotions.yahoo.com/goldrush
From barry at python.org  Fri Aug 20 21:56:30 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 21:56:02 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1f7befae0408191810474dc3de@mail.gmail.com>
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost>
	<1f7befae040811221916a955fd@mail.gmail.com>
	<1092954772.8712.147.camel@geddy.wooz.org>
	<1f7befae0408191810474dc3de@mail.gmail.com>
Message-ID: <1093031790.9257.56.camel@geddy.wooz.org>

On Thu, 2004-08-19 at 21:10, Tim Peters wrote:

> Na.  Add a 4th group to the regexp:
> 
>    |(\$)
> 
> That will reliably match any $, anywhere, that one of the first three
> groups doesn't know what to do with.

Works for me.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/af550877/attachment.pgp
From barry at python.org  Fri Aug 20 21:59:30 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 21:59:06 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1f7befae04082010013adb0a85@mail.gmail.com>
References: <1092177771.21839.54.camel@localhost>
	<cfd23o$lla$1@sea.gmane.org>	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost>
	<1f7befae040811221916a955fd@mail.gmail.com>
	<1092954772.8712.147.camel@geddy.wooz.org>
	<412533F6.4000703@ocf.berkeley.edu>
	<1093014495.9258.6.camel@geddy.wooz.org>
	<412626CC.4050900@ocf.berkeley.edu>
	<1f7befae04082010013adb0a85@mail.gmail.com>
Message-ID: <1093031970.9257.61.camel@geddy.wooz.org>

On Fri, 2004-08-20 at 13:01, Tim Peters wrote:

> The use of named (as opposed to the current numbered)
> capturing groups could make it a lot less dependent on details of how
> the regexp happens to be written, though.

Nice!  I think this will make it easier to document how to do subclasses
too.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/3a496dca/attachment.pgp
From guido at python.org  Fri Aug 20 22:07:01 2004
From: guido at python.org (Guido van Rossum)
Date: Fri Aug 20 22:07:07 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: Your message of "Fri, 20 Aug 2004 13:40:15 +1200."
	<200408200140.i7K1eFZh027162@cosc353.cosc.canterbury.ac.nz> 
References: <200408200140.i7K1eFZh027162@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200408202007.i7KK71t23456@guido.python.org>

> > If the community can *agree* on
> > something they can support, I will listen.
> 
> Even if it's []-after-args?

If most people favor that over prefix @deco, sure, I'll give it
another look.  (It better come with an implementation though.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
From martin at v.loewis.de  Fri Aug 20 22:12:10 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug 20 22:12:02 2004
Subject: [Python-Dev] Reserved Characters
In-Reply-To: <20040820191058.51944.qmail@web50708.mail.yahoo.com>
References: <20040820191058.51944.qmail@web50708.mail.yahoo.com>
Message-ID: <41265B1A.6010405@v.loewis.de>

Jim J Jewett wrote:
> One of the objections to @decorators is that it 
> will cause problems for applications that used 
> '@', simply because of it was one of only three 
> characters that *might* be available.

I personally believe that the set of characters
available to programming languages will extend beyond
us-ascii sooner or later. So the number of available
characters is several thousand.

> Newer String interpolation will also use up '$', 
> which leaves only '?'.

That is not true. $ will have a special meaning only
inside string literals. It can be put into string
literals already, so the lexical rules don't change
at all with that change.

> Is it time to reserve a character to the user, or
> at least to reserve some in certain contexts?  

I don't think so. People who desire to put additional
stuff into Python files should use Python comments.
Indeed, #! already has a special meaning on some systems.

Regards,
Martin
From fperez528 at yahoo.com  Fri Aug 20 22:57:12 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Fri Aug 20 22:57:17 2004
Subject: [Python-Dev] Re: Re: Re: Update PEP 292
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<1092232536.9945.21.camel@localhost> <cfdbv6$pv5$1@sea.gmane.org>
	<1092802980.7810.29.camel@geddy.wooz.org>
Message-ID: <cg5oj8$rqm$1@sea.gmane.org>

Note: I'm resending this, b/c I'd like to see this particular suggestion
addressed as things seem to be finalizing on this PEP.  My original message
seems to have fallen below the radar, and I think it's a reasonable idea (or
at least I would like to hear why it isn't :)

Best to all,

f

------ my original post:

Barry Warsaw wrote:

> On Wed, 2004-08-11 at 10:54, Fernando Perez wrote:

>> It may be trivial to extend pep-292 for this, but not having to do it in
>> every
>> small utility you write/ship would be very useful, IMHO.??If?there?are?no
>> technical reasons to prevent it, is the 'too complex' argument really that
>> strong (especially since you state it's a trivial amount of code)?
> 
> Well, it's about 3 lines of Python, but what would you call the class,
> AttributeTemplate???I?wouldn't?be?opposed?to?adding?the?class,?since?I
> think it would be pretty useful.

I'd like to argue that this form may be the most useful for common tasks, so
you can mix and match "this is foo: $foo and this is foo.bar: $foo.bar"
without having to worry too much about which template class you are using.??

How about making a BaseTemplate class which does NOT allow $foo.bar, and having
the default Template implement attributes???This?would?give?us:

-??A?default?class?covering?what?I?think?is?a?reasonable,?common-case?behavior.?
I'm a big fan of covering most reasonable default behavior out-of-the-box,
and I think python's stdlib achieves this very well in most cases.??I'd?argue
this is one of the reasons for its success, and I have the feeling that in
this case (PEP-292) the proposed default would be sub-par.

- A clean base class for user-defined subclasses which want to be very exacting
about what they want to implement or not.

This sounds like a reasonable compromise to me, but ymmv.

Best, and thanks for keeping an open mind about this issue (string
interpolation is one of the very few areas where python's syntax bugs me, even
after years of daily use).

f


From barry at python.org  Fri Aug 20 23:17:21 2004
From: barry at python.org (Barry Warsaw)
Date: Fri Aug 20 23:16:54 2004
Subject: [Python-Dev] Reserved Characters
In-Reply-To: <20040820191058.51944.qmail@web50708.mail.yahoo.com>
References: <20040820191058.51944.qmail@web50708.mail.yahoo.com>
Message-ID: <1093036641.9258.64.camel@geddy.wooz.org>

On Fri, 2004-08-20 at 15:10, Jim J Jewett wrote:
> Newer String interpolation will also use up '$', 
> which leaves only '?'.

Not true.  PEP 292's $ characters are only special inside string
literals, so there's really nothing new here.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040820/e4d695d4/attachment-0001.pgp
From neal at metaslash.com  Sat Aug 21 01:40:05 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Sat Aug 21 01:40:08 2004
Subject: [Python-Dev] Ignoring SIGPIPE (was: [Python-checkins]
	python/dist/src/Modules	socketmodule.c, 1.298, 1.299)
In-Reply-To: <412586CD.2000000@object-craft.com.au>
References: <E1Bu28N-0005RW-SO@sc8-pr-cvs1.sourceforge.net>
	<20040820020919.GG31470@epoch.metaslash.com>
	<20040820044436.0F8503C113@coffee.object-craft.com.au>
	<412586CD.2000000@object-craft.com.au>
Message-ID: <20040820234005.GP31470@epoch.metaslash.com>

On Fri, Aug 20, 2004 at 03:06:21PM +1000, Dave Cole wrote:
> Andrew McNamara wrote:
> >>>Patch #1003700: Add socketpair function to socket module.
> >>
> >>The docstring (below) states the arguments are the same as socket().
> >>However, in sock_initobj() line 2496, the family is initialized to
> >>AF_INET.  I think the #if defined(AF_UNIX) code above should be
> >>removed and family should be initialized to AF_INET.
> >
> >I talked Dave into this - AF_UNIX is typically the only address family
> >that is valid for socketpair(). Using AF_INET under linux and OS X
> >results in EOPNOTSUPP.
> 
> Would a change to the documentation and docstring be sufficient it it 
> explained that the default family is AF_UNIX if defined for the 
> platform, AF_INET otherwise?

That would make me happy.  I just don't want the doc/docstring
to lie.  I don't have a problem with it defaulting to AF_UNIX.

> >>I don't think the #ifdef SIGPIPE code is correct.  If the user
> >>installed a signal handler calling signal() will remove it.  I 
> >>think the call to signal() should be removed.
> >
> >I agree, but I think that was copied verbatim from elsewhere in
> >socketmodule.c, so it was left with the aim of being bug for bug
> >compatible with socket.socket().
> 
> Indeed.  I was not going to change something as fundamental as the 
> existing signal handling.  The onyl sensible thing to do was to copy the 
> behaviour already in place when you create a socket the "normal" way.

You are correct.  There are two places that call 
signal(SIGPIPE, SIG_IGN); I find this surprising.

Guido, you added this code in version 1.4 from 1991.  
You do remember the reason why, don't you?  It's only 
13 years ago. :-)  I'm hesitant to remove the old calls,
but I'm not sure I want it added to new calls if it's
not necessary.

Any opinions?

Neal
From guido at python.org  Sat Aug 21 03:11:37 2004
From: guido at python.org (Guido van Rossum)
Date: Sat Aug 21 03:11:49 2004
Subject: [Python-Dev] Ignoring SIGPIPE (was: [Python-checkins]
	python/dist/src/Modules socketmodule.c, 1.298, 1.299)
In-Reply-To: Your message of "Fri, 20 Aug 2004 19:40:05 EDT."
	<20040820234005.GP31470@epoch.metaslash.com> 
References: <E1Bu28N-0005RW-SO@sc8-pr-cvs1.sourceforge.net>
	<20040820020919.GG31470@epoch.metaslash.com>
	<20040820044436.0F8503C113@coffee.object-craft.com.au>
	<412586CD.2000000@object-craft.com.au> 
	<20040820234005.GP31470@epoch.metaslash.com> 
Message-ID: <200408210111.i7L1Bb623898@guido.python.org>

> > >I agree, but I think that was copied verbatim from elsewhere in
> > >socketmodule.c, so it was left with the aim of being bug for bug
> > >compatible with socket.socket().
> > 
> > Indeed.  I was not going to change something as fundamental as the 
> > existing signal handling.  The onyl sensible thing to do was to copy the 
> > behaviour already in place when you create a socket the "normal" way.
> 
> You are correct.  There are two places that call 
> signal(SIGPIPE, SIG_IGN); I find this surprising.
> 
> Guido, you added this code in version 1.4 from 1991.  
> You do remember the reason why, don't you?  It's only 
> 13 years ago. :-)  I'm hesitant to remove the old calls,
> but I'm not sure I want it added to new calls if it's
> not necessary.
> 
> Any opinions?

The only real requirement is that SIGPIPE is being ignored by default,
and that is already taken care of by inisigs() in pythonrun.c.  So I
think both calls can be removed from socketmodule.c, heh heh.

Originally, I was being careful, and not ignoring SIGPIPE until the
first socket was created, hence the call there.  The idea being that
if you have a Python program that spits lots of stuff to stdout, and
you pipe it into head (e.g.), it won't give you an exception -- the
SIGPIPE will just kill it and the shell will ignore that; but if
you're using sockets, it's important to ignore SIGPIPE so too bad for
the other feature.  But we've been ignoring SIGPIPE in the
initialization forever, so the SIGPIPE call in socketmodule.c is
really not needed any more.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From fumanchu at amor.org  Sat Aug 21 10:31:05 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Sat Aug 21 10:36:27 2004
Subject: [Python-Dev] python-dev Summary for 2004-08-01 through
	2004-08-15[draft]
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E1D@exchange.hqamor.amorhq.net>

Brett C. wrote:
> Option two out of all of this is people just say, "summarize what you
> want, Brett."  Then I just cover what I find interesting and just
don't
> worry about covering a specific area.

+1. Bring back "Skipped Threads" if you can, just for the sake of
indexing completeness.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From pm_mon at yahoo.com  Sat Aug 21 18:29:46 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sat Aug 21 18:29:53 2004
Subject: [Python-Dev] __metaclass__ and __author__ are already decorators
Message-ID: <cg7t9r$r8r$4@sea.gmane.org>

[I posted this on comp.lang.python.general but I'm not sure how many of 
the folks here read that newsgroup, so my apologies if you're seeing 
this twice.]

Thinking about decorators, and looking at what we are already doing in 
our Python code, it seems that __metaclass__, __author__, __version__, 
etc. are all examples of decorators.  So we already have a decorator 
syntax.  What is the compelling reason to invent a new one?  And if we 
do, what's to become of the old one?

Here's my take on this.  We have two kinds of decorators:  those that 
are informational only (like __author__ and __version__), and those 
which have side-effects, i.e. those that actually *do* something (like 
__metaclass__).

     class Foo:
         """ This describes the Foo class as normal. """
         __metaclass__ = M
         __author__ = 'Paul Morrow'
         __version__ = '0.1'
         __automethods__ = True


         def baz(self, a, b):
             """ This describes the baz method. """
             __synchronized__ = True
             __returns__ = None
             __author__ = 'Neville Shunt'
             # body of baz goes here...

There, that looks pretty clear and pythonic.  Now how to define the 
decorators.

     def metaclass(decoratedClass, subType):
         """ This describes the 'metaclass' decorator. """
         __decorator__ = True
         __version__ = '1.9'
         # perform the metaclass operation on decoratedClass


     def synchronized(decoratedFunc, trueOrFalse):
         """ This describes the 'synchronized' decorator. """
         __decorator__ = True
         __author__ = 'Martin Curry'
         __version__ = '0.5'
         # perform the synchronized operation on decoratedFunc


     def returns(decoratedFunc, *args):
         """Docstring for 'returns' decorator."""
         __decorator__ = True
         # body of decorator goes here


Each decorator function receives the class|function|method being 
decorated as the first parameter of the function (e.g. decoratedClass 
and decoratedFunc above).  The remaining parameters are the values 
assigned to the __xxx__ variable in the definition of the decorated 
function.

So, in the above example, when the 'returns' decorator function is 
called to decorate the 'baz' method, decoratedFunc would recieve the baz 
object and args[0] would be set to None (because of the statement 
"__returns__ = None" in the definition of baz).

For a function to be used as a decorator function, it must be decorated 
as such, by setting its __decorator__ attribute to True.

Does this handle enough of the decorator concerns?

Paul


From pje at telecommunity.com  Sat Aug 21 18:54:37 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat Aug 21 18:54:29 2004
Subject: [Python-Dev] __metaclass__ and __author__ are already decorators
In-Reply-To: <cg7t9r$r8r$4@sea.gmane.org>
Message-ID: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>

At 12:29 PM 8/21/04 -0400, Paul Morrow wrote:
>[I posted this on comp.lang.python.general but I'm not sure how many of 
>the folks here read that newsgroup, so my apologies if you're seeing this 
>twice.]
>
>Thinking about decorators, and looking at what we are already doing in our 
>Python code, it seems that __metaclass__, __author__, __version__, etc. 
>are all examples of decorators.  So we already have a decorator syntax.

The items you describe are not decorators, they are attributes.  The 
current syntax for decorators is 'x = decorator(x)'.


>         def baz(self, a, b):
>             """ This describes the baz method. """
>             __synchronized__ = True
>             __returns__ = None
>             __author__ = 'Neville Shunt'
>             # body of baz goes here...

This syntax already has a different and conflicting meaning in today's 
Python.  A different such syntax:

     [synchronized(), returns(None), author("Neville Shunt")]
     def baz(self, a, b):
         """ This describes the baz method. """
         # body of baz goes here...

has already been rejected on the basis that this too has meaning in today's 
Python, and that the syntax produces "spooky action at a distance".  That 
is, it's "magical" that the function calls affect the definition below.

Following this logic, expecting assignments to magic names contained within 
a function's body to magically invoke decorator functions with different 
names *before and outside the block* where those names are assigned to in 
the code, seems far less likely to be considered acceptable.


From pm_mon at yahoo.com  Sat Aug 21 20:25:27 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sat Aug 21 20:25:43 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>
References: <cg7t9r$r8r$4@sea.gmane.org>
	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>
Message-ID: <cg842n$r8r$5@sea.gmane.org>

Phillip J. Eby wrote:
> At 12:29 PM 8/21/04 -0400, Paul Morrow wrote:
> 
>> [I posted this on comp.lang.python.general but I'm not sure how many 
>> of the folks here read that newsgroup, so my apologies if you're 
>> seeing this twice.]
>>
>> Thinking about decorators, and looking at what we are already doing in 
>> our Python code, it seems that __metaclass__, __author__, __version__, 
>> etc. are all examples of decorators.  So we already have a decorator 
>> syntax.
> 
> 
> The items you describe are not decorators, they are attributes.  The 
> current syntax for decorators is 'x = decorator(x)'.
> 

In today's Python that's of course true, but these 'attributes' are used 
to supply meta info about the class. They *describe* the class.  They 
are never intended to be inherited by instances of the class.  So in 
this sense, they are very much more like decorators than (normal) class 
attributes.


> 
>>         def baz(self, a, b):
>>             """ This describes the baz method. """
>>             __synchronized__ = True
>>             __returns__ = None
>>             __author__ = 'Neville Shunt'
>>             # body of baz goes here...
> 
> 
> This syntax already has a different and conflicting meaning in today's 
> Python.  

Yes.  There is a big conflict here.  When I say

     def Foo:
         __author__ = 'me'
         __metaclass__ = M
         var1 = 10
         var2 = 20

although I'm technically defining four class variables, my intention 
behind the first two is very different than my intention behind the 
second two. The first two supply meta information about Foo.  It is 
*not* my intention that those variables be inherited by any instances of 
Foo, because they *don't apply* to instances of Foo --- they apply to 
Foo itself.  They enhance the definition of Foo.  They decorate Foo.

This is true of all magic attributes (as far as I can tell), magic 
methods included.  We define them right alongside instance methods.  But 
we don't intend for the magic methods to be *inherited* by the instances 
so that they become part of the instances interface.  We want to affect 
something much deeper than that (what happens when the instance is 
constructed, which operators may be used on the instance, etc.)

So providing definitions for __xxx__ attributes is not done for the same 
purpose as the other attributes.  They are special.  Therefore, let's 
formally acknowledge that and call them 'decorators'.

> 
> A different such syntax:
>     [synchronized(), returns(None), author("Neville Shunt")]
>     def baz(self, a, b):
>         """ This describes the baz method. """
>         # body of baz goes here...
> 
> has already been rejected on the basis that this too has meaning in 
> today's Python, and that the syntax produces "spooky action at a 
> distance".  That is, it's "magical" that the function calls affect the 
> definition below.
> 

The problem with this example isn't that there's magic involved, it's 
that 1) there's no precedent for idioms like the statement in your 
example to do anything special (to have magical side effects), and 2) 
its not intuitive that the declarations affect the upcoming def.

I don't believe that we have an aversion to magic, where it's clear and 
intuitive what's going to happen.


> Following this logic, expecting assignments to magic names contained 
> within a function's body to magically invoke decorator functions with 
> different names *before and outside the block* where those names are 
> assigned to in the code, seems far less likely to be considered acceptable.
> 

1. The decorator names aren't different than the decorator functions, 
they're the same (minus the leading and trailing double underscores).

2. All magic methods in today's Python are invoked 'magically' (are 
called indirectly; not called directly by any user code).

3. All of the interesting uses of decorators that I've seen proposed 
involve calling functions defined elsewhere (certainly outside of the 
current block of code).

From chris.cavalaria at free.fr  Sat Aug 21 22:14:07 2004
From: chris.cavalaria at free.fr (Christophe Cavalaria)
Date: Sat Aug 21 22:14:12 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
References: <cg7t9r$r8r$4@sea.gmane.org>
Message-ID: <cg8aef$dek$1@sea.gmane.org>

Paul Morrow wrote:

>      class Foo:
>          """ This describes the Foo class as normal. """
>          __metaclass__ = M
>          __author__ = 'Paul Morrow'
>          __version__ = '0.1'
>          __automethods__ = True
> 
> 
>          def baz(self, a, b):
>              """ This describes the baz method. """
>              __synchronized__ = True
>              __returns__ = None
>              __author__ = 'Neville Shunt'
>              # body of baz goes here...

It's a function call that masquerades as an attribute assignment. How worse
can it be ? There's also the fact that it can't handle named parameters
like a regular function call. You can't write that :

def foo():
    __decoration__ = (1,1,param=True)


From pm_mon at yahoo.com  Sat Aug 21 23:15:09 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sat Aug 21 23:15:15 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <cg8aef$dek$1@sea.gmane.org>
References: <cg7t9r$r8r$4@sea.gmane.org> <cg8aef$dek$1@sea.gmane.org>
Message-ID: <cg8e0u$r8r$9@sea.gmane.org>

Christophe Cavalaria wrote:

> Paul Morrow wrote:
> 
> 
>>     class Foo:
>>         """ This describes the Foo class as normal. """
>>         __metaclass__ = M
>>         __author__ = 'Paul Morrow'
>>         __version__ = '0.1'
>>         __automethods__ = True
>>
>>
>>         def baz(self, a, b):
>>             """ This describes the baz method. """
>>             __synchronized__ = True
>>             __returns__ = None
>>             __author__ = 'Neville Shunt'
>>             # body of baz goes here...
> 
> 
> It's a function call that masquerades as an attribute assignment. How worse
> can it be ? 

How about we create new syntax that uses an @ and special words, where 
the words correspond with functions to be called.  That's what is 
seriously being considered.  And that would be worse (IMO).



> can it be ? There's also the fact that it can't handle named parameters
> like a regular function call. You can't write that :
> 
> def foo():
>     __decoration__ = (1,1,param=True)
> 

As far as I know, we can't do that with the current decorator proposals 
either.


[But that is something that I've often wanted to do (create a tuple that 
contains named arguments).]

From kbk at shore.net  Sat Aug 21 23:18:08 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Sat Aug 21 23:18:11 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <4124FD72.4090901@v.loewis.de> (Martin v.
	=?iso-8859-1?q?L=F6wis's?= message of "Thu, 19 Aug 2004 21:20:18 +0200")
References: <200408190529.i7J5TZF7010126@h006008a7bda6.ne.client2.attbi.com>
	<16676.47823.382278.737098@montanaro.dyndns.org>
	<4124FD72.4090901@v.loewis.de>
Message-ID: <87wtzs5gxb.fsf@hydra.localdomain>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> Skip Montanaro wrote:
>> Any thoughts about posting this to c.l.py as well as python-dev?  Maybe it
>> would encourage some more participation...
>
> +1.

OK, if no one objects, I'll add c.l.p to the distribution.

-- 
KBK
From pje at telecommunity.com  Sat Aug 21 23:21:28 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat Aug 21 23:21:14 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: <cg8e0u$r8r$9@sea.gmane.org>
References: <cg8aef$dek$1@sea.gmane.org> <cg7t9r$r8r$4@sea.gmane.org>
	<cg8aef$dek$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>

At 05:15 PM 8/21/04 -0400, Paul Morrow wrote:
>Christophe Cavalaria wrote:
>
>>can it be ? There's also the fact that it can't handle named parameters
>>like a regular function call. You can't write that :
>>def foo():
>>     __decoration__ = (1,1,param=True)
>
>As far as I know, we can't do that with the current decorator proposals 
>either.

@decoration(1,1,param=True)
def foo(whatever):
     pass


From pm_mon at yahoo.com  Sat Aug 21 23:34:16 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sat Aug 21 23:34:21 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>
References: <cg8aef$dek$1@sea.gmane.org>
	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>
	<cg8e0u$r8r$9@sea.gmane.org>
	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>
Message-ID: <cg8f4p$r8r$11@sea.gmane.org>

Phillip J. Eby wrote:

> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote:
> 
>> Christophe Cavalaria wrote:
>>
>>> can it be ? There's also the fact that it can't handle named parameters
>>> like a regular function call. You can't write that :
>>> def foo():
>>>     __decoration__ = (1,1,param=True)
>>
>>
>> As far as I know, we can't do that with the current decorator 
>> proposals either.
> 
> 
> @decoration(1,1,param=True)
> def foo(whatever):
>     pass
> 
> 

Ok, then whatever changes you've made to the Python system to support 
that would allow the same syntax to be used in what I'm suggesting.

     def foo(whatever):
         __decoration__ = (1,1,param=True)

     def decoration(decoratedFunc, a, b, param=False):
         __decorator__ = True
         __version__ = '0.1'
         # body of 'decoration' decorator function goes here...


From bob at redivi.com  Sat Aug 21 23:47:18 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sat Aug 21 23:48:01 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: <cg8f4p$r8r$11@sea.gmane.org>
References: <cg8aef$dek$1@sea.gmane.org>
	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>
	<cg8e0u$r8r$9@sea.gmane.org>
	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>
	<cg8f4p$r8r$11@sea.gmane.org>
Message-ID: <ACA4E2BD-F3BB-11D8-845D-000A95686CD8@redivi.com>


On Aug 21, 2004, at 5:34 PM, Paul Morrow wrote:

> Phillip J. Eby wrote:
>
>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote:
>>> Christophe Cavalaria wrote:
>>>
>>>> can it be ? There's also the fact that it can't handle named 
>>>> parameters
>>>> like a regular function call. You can't write that :
>>>> def foo():
>>>>     __decoration__ = (1,1,param=True)
>>>
>>>
>>> As far as I know, we can't do that with the current decorator 
>>> proposals either.
>> @decoration(1,1,param=True)
>> def foo(whatever):
>>     pass
>
> Ok, then whatever changes you've made to the Python system to support 
> that would allow the same syntax to be used in what I'm suggesting.
>
>     def foo(whatever):
>         __decoration__ = (1,1,param=True)
>
>     def decoration(decoratedFunc, a, b, param=False):
>         __decorator__ = True
>         __version__ = '0.1'
>         # body of 'decoration' decorator function goes here...

Congratulations, this by far the worst suggestion yet!  I'm -Inf on 
this :)

@decoration(1,1,param=True) makes no changes whatsoever to the Python 
system.  Everything after the @ is really just an expression.  The @ 
just implies that the result of the next class or def gets put through 
the result of the @expression before it is thrown into the namespace.

-bob
From pje at telecommunity.com  Sat Aug 21 23:48:43 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat Aug 21 23:48:27 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: <cg8f4p$r8r$11@sea.gmane.org>
References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>
	<cg8aef$dek$1@sea.gmane.org> <cg7t9r$r8r$4@sea.gmane.org>
	<cg8aef$dek$1@sea.gmane.org> <cg8e0u$r8r$9@sea.gmane.org>
	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040821174009.02531670@mail.telecommunity.com>

At 05:34 PM 8/21/04 -0400, Paul Morrow wrote:
>Phillip J. Eby wrote:
>
>>At 05:15 PM 8/21/04 -0400, Paul Morrow wrote:
>>
>>>Christophe Cavalaria wrote:
>>>
>>>>can it be ? There's also the fact that it can't handle named parameters
>>>>like a regular function call. You can't write that :
>>>>def foo():
>>>>     __decoration__ = (1,1,param=True)
>>>
>>>
>>>As far as I know, we can't do that with the current decorator proposals 
>>>either.
>>
>>@decoration(1,1,param=True)
>>def foo(whatever):
>>     pass
>
>Ok, then whatever changes you've made to the Python system to support that 
>would allow the same syntax to be used in what I'm suggesting.

Huh?  @decoration(1,1,param=True) is evaluated at the place where it 
appears.  The *return value* of that expression is then called, passing in 
the foo function.  In other words, the above is equivalent to today's:

     def foo(whatever):
         pass

     foo = decoration(1,1,param=True)(foo)

except that the first assignment to 'foo' doesn't happen, only the second 
one.  If the 'foo' function is a single expression, of course, today you 
can do the straightforward:

     foo = decoration(1,1,param=True)(lambda whatever: something)

So, "@x func" is effectively a macro for "func = x(func)", where 'func' may 
be a function, or another decorator.  That is:

     @x
     @y
     @z
     def foo():
         ...

is shorthand for 'foo = x(y(z(foo)))', no matter what the x/y/z expressions 
actually are.  So there are *no* "changes to the Python system" here, just 
a very small amount of syntax sugar.

By contrast, your attribute assignment approach isn't nearly so simple, as 
you will see if you attempt to write an implementation.

From pm_mon at yahoo.com  Sun Aug 22 00:15:43 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sun Aug 22 00:15:50 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <5.1.1.6.0.20040821174009.02531670@mail.telecommunity.com>
References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8aef$dek$1@sea.gmane.org>
	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>
	<cg8e0u$r8r$9@sea.gmane.org>	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>
	<cg8f4p$r8r$11@sea.gmane.org>
	<5.1.1.6.0.20040821174009.02531670@mail.telecommunity.com>
Message-ID: <cg8hig$r8r$12@sea.gmane.org>

Phillip J. Eby wrote:

> At 05:34 PM 8/21/04 -0400, Paul Morrow wrote:
> 
>> Phillip J. Eby wrote:
>>
>>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote:
>>>
>>>> Christophe Cavalaria wrote:
>>>>
>>>>> can it be ? There's also the fact that it can't handle named 
>>>>> parameters
>>>>> like a regular function call. You can't write that :
>>>>> def foo():
>>>>>     __decoration__ = (1,1,param=True)
>>>>
>>>>
>>>>
>>>> As far as I know, we can't do that with the current decorator 
>>>> proposals either.
>>>
>>>
>>> @decoration(1,1,param=True)
>>> def foo(whatever):
>>>     pass
>>
>>
>> Ok, then whatever changes you've made to the Python system to support 
>> that would allow the same syntax to be used in what I'm suggesting.
> 
> 
> Huh?  @decoration(1,1,param=True) is evaluated at the place where it 
> appears.  The *return value* of that expression is then called, passing 
> in the foo function.  In other words, the above is equivalent to today's:
> 
>     def foo(whatever):
>         pass
> 
>     foo = decoration(1,1,param=True)(foo)
> 
> except that the first assignment to 'foo' doesn't happen, only the 
> second one.  If the 'foo' function is a single expression, of course, 
> today you can do the straightforward:
> 
>     foo = decoration(1,1,param=True)(lambda whatever: something)
> 
> So, "@x func" is effectively a macro for "func = x(func)", where 'func' 
> may be a function, or another decorator.  That is:
> 
>     @x
>     @y
>     @z
>     def foo():
>         ...
> 
> is shorthand for 'foo = x(y(z(foo)))', 

Wouldn't that actually be shorthand for foo = x()(y()(z()(foo))) ?


From bob at redivi.com  Sun Aug 22 00:22:04 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Aug 22 00:22:51 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: <cg8hig$r8r$12@sea.gmane.org>
References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8aef$dek$1@sea.gmane.org>
	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>
	<cg8e0u$r8r$9@sea.gmane.org>	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>
	<cg8f4p$r8r$11@sea.gmane.org>
	<5.1.1.6.0.20040821174009.02531670@mail.telecommunity.com>
	<cg8hig$r8r$12@sea.gmane.org>
Message-ID: <87E9F4E0-F3C0-11D8-A072-000A95686CD8@redivi.com>


On Aug 21, 2004, at 6:15 PM, Paul Morrow wrote:

> Phillip J. Eby wrote:
>
>> At 05:34 PM 8/21/04 -0400, Paul Morrow wrote:
>>> Phillip J. Eby wrote:
>>>
>>>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote:
>>>>
>>>>> Christophe Cavalaria wrote:
>>>>>
>>>>>> can it be ? There's also the fact that it can't handle named 
>>>>>> parameters
>>>>>> like a regular function call. You can't write that :
>>>>>> def foo():
>>>>>>     __decoration__ = (1,1,param=True)
>>>>>
>>>>>
>>>>>
>>>>> As far as I know, we can't do that with the current decorator 
>>>>> proposals either.
>>>>
>>>>
>>>> @decoration(1,1,param=True)
>>>> def foo(whatever):
>>>>     pass
>>>
>>>
>>> Ok, then whatever changes you've made to the Python system to 
>>> support that would allow the same syntax to be used in what I'm 
>>> suggesting.
>> Huh?  @decoration(1,1,param=True) is evaluated at the place where it 
>> appears.  The *return value* of that expression is then called, 
>> passing in the foo function.  In other words, the above is equivalent 
>> to today's:
>>     def foo(whatever):
>>         pass
>>     foo = decoration(1,1,param=True)(foo)
>> except that the first assignment to 'foo' doesn't happen, only the 
>> second one.  If the 'foo' function is a single expression, of course, 
>> today you can do the straightforward:
>>     foo = decoration(1,1,param=True)(lambda whatever: something)
>> So, "@x func" is effectively a macro for "func = x(func)", where 
>> 'func' may be a function, or another decorator.  That is:
>>     @x
>>     @y
>>     @z
>>     def foo():
>>         ...
>> is shorthand for 'foo = x(y(z(foo)))',
>
> Wouldn't that actually be shorthand for foo = x()(y()(z()(foo))) ?

No.

-bob

From pm_mon at yahoo.com  Sun Aug 22 00:24:09 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sun Aug 22 00:24:16 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <ACA4E2BD-F3BB-11D8-845D-000A95686CD8@redivi.com>
References: <cg8aef$dek$1@sea.gmane.org>	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>	<cg8e0u$r8r$9@sea.gmane.org>	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8f4p$r8r$11@sea.gmane.org>
	<ACA4E2BD-F3BB-11D8-845D-000A95686CD8@redivi.com>
Message-ID: <cg8i2a$r8r$13@sea.gmane.org>

Bob Ippolito wrote:

> 
> On Aug 21, 2004, at 5:34 PM, Paul Morrow wrote:
> 
>> Phillip J. Eby wrote:
>>
>>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote:
>>>
>>>> Christophe Cavalaria wrote:
>>>>
>>>>> can it be ? There's also the fact that it can't handle named 
>>>>> parameters
>>>>> like a regular function call. You can't write that :
>>>>> def foo():
>>>>>     __decoration__ = (1,1,param=True)
>>>>
>>>>
>>>>
>>>> As far as I know, we can't do that with the current decorator 
>>>> proposals either.
>>>
>>> @decoration(1,1,param=True)
>>> def foo(whatever):
>>>     pass
>>
>>
>> Ok, then whatever changes you've made to the Python system to support 
>> that would allow the same syntax to be used in what I'm suggesting.
>>
>>     def foo(whatever):
>>         __decoration__ = (1,1,param=True)
>>
>>     def decoration(decoratedFunc, a, b, param=False):
>>         __decorator__ = True
>>         __version__ = '0.1'
>>         # body of 'decoration' decorator function goes here...
> 
> 
> Congratulations, this by far the worst suggestion yet!  I'm -Inf on this :)
> 

Thanks.


> @decoration(1,1,param=True) makes no changes whatsoever to the Python 
> system.  Everything after the @ is really just an expression.  The @ 
> just implies that the result of the next class or def gets put through 
> the result of the @expression before it is thrown into the namespace.
> 

It seems that writing a decorator is going to be a bizarre experience. 
In the example, I would need to write a function named 'decoration' that 
returns a function that will recieve a function (foo) to be decorated 
and then return a function.  Does that sound about right?

What would functions like 'decoration' typically look like?  Could you 
show a short code snippet?


From bob at redivi.com  Sun Aug 22 00:28:37 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Aug 22 00:29:13 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: <cg8i2a$r8r$13@sea.gmane.org>
References: <cg8aef$dek$1@sea.gmane.org>	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>	<cg8e0u$r8r$9@sea.gmane.org>	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8f4p$r8r$11@sea.gmane.org>
	<ACA4E2BD-F3BB-11D8-845D-000A95686CD8@redivi.com>
	<cg8i2a$r8r$13@sea.gmane.org>
Message-ID: <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com>


On Aug 21, 2004, at 6:24 PM, Paul Morrow wrote:

> Bob Ippolito wrote:
>
>> On Aug 21, 2004, at 5:34 PM, Paul Morrow wrote:
>>> Phillip J. Eby wrote:
>>>
>>>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote:
>>>>
>>>>> Christophe Cavalaria wrote:
>>>>>
>>>>>> can it be ? There's also the fact that it can't handle named 
>>>>>> parameters
>>>>>> like a regular function call. You can't write that :
>>>>>> def foo():
>>>>>>     __decoration__ = (1,1,param=True)
>>>>>
>>>>>
>>>>>
>>>>> As far as I know, we can't do that with the current decorator 
>>>>> proposals either.
>>>>
>>>> @decoration(1,1,param=True)
>>>> def foo(whatever):
>>>>     pass
>>>
>>>
>>> Ok, then whatever changes you've made to the Python system to 
>>> support that would allow the same syntax to be used in what I'm 
>>> suggesting.
>>>
>>>     def foo(whatever):
>>>         __decoration__ = (1,1,param=True)
>>>
>>>     def decoration(decoratedFunc, a, b, param=False):
>>>         __decorator__ = True
>>>         __version__ = '0.1'
>>>         # body of 'decoration' decorator function goes here...
>> Congratulations, this by far the worst suggestion yet!  I'm -Inf on 
>> this :)
>
> Thanks.
>
>
>> @decoration(1,1,param=True) makes no changes whatsoever to the Python 
>> system.  Everything after the @ is really just an expression.  The @ 
>> just implies that the result of the next class or def gets put 
>> through the result of the @expression before it is thrown into the 
>> namespace.
>
> It seems that writing a decorator is going to be a bizarre experience. 
> In the example, I would need to write a function named 'decoration' 
> that returns a function that will recieve a function (foo) to be 
> decorated and then return a function.  Does that sound about right?

Yes that is correct.

> What would functions like 'decoration' typically look like?  Could you 
> show a short code snippet?

http://python.org/peps/pep-0318.html

-bob
From pm_mon at yahoo.com  Sun Aug 22 00:29:56 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sun Aug 22 00:30:01 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <cg8i2a$r8r$13@sea.gmane.org>
References: <cg8aef$dek$1@sea.gmane.org>	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>	<cg8e0u$r8r$9@sea.gmane.org>	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8f4p$r8r$11@sea.gmane.org>	<ACA4E2BD-F3BB-11D8-845D-000A95686CD8@redivi.com>
	<cg8i2a$r8r$13@sea.gmane.org>
Message-ID: <cg8id5$r8r$15@sea.gmane.org>

Paul Morrow wrote:

> Bob Ippolito wrote:
> 
>>
>> On Aug 21, 2004, at 5:34 PM, Paul Morrow wrote:
>>
>>> Phillip J. Eby wrote:
>>>
>>>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote:
>>>>
>>>>> Christophe Cavalaria wrote:
>>>>>
>>>>>> can it be ? There's also the fact that it can't handle named 
>>>>>> parameters
>>>>>> like a regular function call. You can't write that :
>>>>>> def foo():
>>>>>>     __decoration__ = (1,1,param=True)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> As far as I know, we can't do that with the current decorator 
>>>>> proposals either.
>>>>
>>>>
>>>> @decoration(1,1,param=True)
>>>> def foo(whatever):
>>>>     pass
>>>
>>>
>>>
>>> Ok, then whatever changes you've made to the Python system to support 
>>> that would allow the same syntax to be used in what I'm suggesting.
>>>
>>>     def foo(whatever):
>>>         __decoration__ = (1,1,param=True)
>>>
>>>     def decoration(decoratedFunc, a, b, param=False):
>>>         __decorator__ = True
>>>         __version__ = '0.1'
>>>         # body of 'decoration' decorator function goes here...
>>
>>
>>
>> Congratulations, this by far the worst suggestion yet!  I'm -Inf on 
>> this :)
>>
> 
> Thanks.
> 
> 
>> @decoration(1,1,param=True) makes no changes whatsoever to the Python 
>> system.  Everything after the @ is really just an expression.  The @ 
>> just implies that the result of the next class or def gets put through 
>> the result of the @expression before it is thrown into the namespace.
>>
> 
> It seems that writing a decorator is going to be a bizarre experience. 
> In the example, I would need to write a function named 'decoration' that 
> returns a function that will recieve a function (foo) to be decorated 
> and then return a function.  Does that sound about right?
> 
> What would functions like 'decoration' typically look like?  Could you 
> show a short code snippet?
> 

Nevermind.  I got wrapped around the axle.  I get it now.  Thanks.

From pm_mon at yahoo.com  Sun Aug 22 00:28:36 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sun Aug 22 00:30:49 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <87E9F4E0-F3C0-11D8-A072-000A95686CD8@redivi.com>
References: <5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8aef$dek$1@sea.gmane.org>	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>	<cg8e0u$r8r$9@sea.gmane.org>	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8f4p$r8r$11@sea.gmane.org>	<5.1.1.6.0.20040821174009.02531670@mail.telecommunity.com>	<cg8hig$r8r$12@sea.gmane.org>
	<87E9F4E0-F3C0-11D8-A072-000A95686CD8@redivi.com>
Message-ID: <cg8ial$r8r$14@sea.gmane.org>

Bob Ippolito wrote:

> 
> On Aug 21, 2004, at 6:15 PM, Paul Morrow wrote:
> 
>> Phillip J. Eby wrote:
>>
>>> At 05:34 PM 8/21/04 -0400, Paul Morrow wrote:
>>>
>>>> Phillip J. Eby wrote:
>>>>
>>>>> At 05:15 PM 8/21/04 -0400, Paul Morrow wrote:
>>>>>
>>>>>> Christophe Cavalaria wrote:
>>>>>>
>>>>>>> can it be ? There's also the fact that it can't handle named 
>>>>>>> parameters
>>>>>>> like a regular function call. You can't write that :
>>>>>>> def foo():
>>>>>>>     __decoration__ = (1,1,param=True)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> As far as I know, we can't do that with the current decorator 
>>>>>> proposals either.
>>>>>
>>>>>
>>>>>
>>>>> @decoration(1,1,param=True)
>>>>> def foo(whatever):
>>>>>     pass
>>>>
>>>>
>>>>
>>>> Ok, then whatever changes you've made to the Python system to 
>>>> support that would allow the same syntax to be used in what I'm 
>>>> suggesting.
>>>
>>> Huh?  @decoration(1,1,param=True) is evaluated at the place where it 
>>> appears.  The *return value* of that expression is then called, 
>>> passing in the foo function.  In other words, the above is equivalent 
>>> to today's:
>>>     def foo(whatever):
>>>         pass
>>>     foo = decoration(1,1,param=True)(foo)
>>> except that the first assignment to 'foo' doesn't happen, only the 
>>> second one.  If the 'foo' function is a single expression, of course, 
>>> today you can do the straightforward:
>>>     foo = decoration(1,1,param=True)(lambda whatever: something)
>>> So, "@x func" is effectively a macro for "func = x(func)", where 
>>> 'func' may be a function, or another decorator.  That is:
>>>     @x
>>>     @y
>>>     @z
>>>     def foo():
>>>         ...
>>> is shorthand for 'foo = x(y(z(foo)))',
>>
>>
>> Wouldn't that actually be shorthand for foo = x()(y()(z()(foo))) ?
> 
> 
> No.
> 

Ok, I see, nevermind.  Thanks.


From pm_mon at yahoo.com  Sun Aug 22 00:54:28 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sun Aug 22 00:54:34 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com>
References: <cg8aef$dek$1@sea.gmane.org>	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>	<cg8e0u$r8r$9@sea.gmane.org>	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8f4p$r8r$11@sea.gmane.org>	<ACA4E2BD-F3BB-11D8-845D-000A95686CD8@redivi.com>	<cg8i2a$r8r$13@sea.gmane.org>
	<7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com>
Message-ID: <cg8jr5$r8r$16@sea.gmane.org>

Bob Ippolito wrote:

> 
> On Aug 21, 2004, at 6:24 PM, Paul Morrow wrote:
> 
>>
>> It seems that writing a decorator is going to be a bizarre experience. 
>> In the example, I would need to write a function named 'decoration' 
>> that returns a function that will recieve a function (foo) to be 
>> decorated and then return a function.  Does that sound about right?
> 
> 
> Yes that is correct.
> 
>> What would functions like 'decoration' typically look like?  Could you 
>> show a short code snippet?
> 
> 
> http://python.org/peps/pep-0318.html
> 

Thanks.  Of the 5 examples there, the first two are apparently not 
implemented correctly, as they expect that the function/class to be 
decorated is passed directly to them, rather than to the function they 
return.  Would you agree?  I pasted them here for your consideration...


    1. Define a function to be executed at exit. Note that the function 
isn't actually "wrapped" in the usual sense.

def onexit(f):
     import atexit
     atexit.register(f)
     return f

@onexit
def func():
     ...

    2. Define a class with a singleton instance. Note that once the 
class disappears enterprising programmers would have to be more creative 
to create more instances. (From Shane Hathaway on python-dev.)

def singleton(cls):
     instances = {}
     def getinstance():
         if cls not in instances:
             instances[cls] = cls()
         return instances[cls]
     return getinstance

@singleton
class MyClass:
     ...


From bob at redivi.com  Sun Aug 22 01:11:12 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Aug 22 01:11:50 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: <cg8jr5$r8r$16@sea.gmane.org>
References: <cg8aef$dek$1@sea.gmane.org>	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>	<cg8e0u$r8r$9@sea.gmane.org>	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8f4p$r8r$11@sea.gmane.org>	<ACA4E2BD-F3BB-11D8-845D-000A95686CD8@redivi.com>	<cg8i2a$r8r$13@sea.gmane.org>
	<7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com>
	<cg8jr5$r8r$16@sea.gmane.org>
Message-ID: <64E90FD9-F3C7-11D8-A072-000A95686CD8@redivi.com>


On Aug 21, 2004, at 6:54 PM, Paul Morrow wrote:

> Bob Ippolito wrote:
>
>> On Aug 21, 2004, at 6:24 PM, Paul Morrow wrote:
>>>
>>> It seems that writing a decorator is going to be a bizarre 
>>> experience. In the example, I would need to write a function named 
>>> 'decoration' that returns a function that will recieve a function 
>>> (foo) to be decorated and then return a function.  Does that sound 
>>> about right?
>> Yes that is correct.
>>> What would functions like 'decoration' typically look like?  Could 
>>> you show a short code snippet?
>> http://python.org/peps/pep-0318.html
>
> Thanks.  Of the 5 examples there, the first two are apparently not 
> implemented correctly, as they expect that the function/class to be 
> decorated is passed directly to them, rather than to the function they 
> return.  Would you agree?  I pasted them here for your 
> consideration...

No, they are correct.  You are confused.  What is expected is that the 
result of the expression after @ is callable and takes one parameter.  
If the expression after @ is just a name, then nothing particularly 
exciting happens at that time.

@bar # NOTE THE LACK OF PARENTHESES
def foo():
	....

is equivalent to:

_tmp = bar
def foo():
	....
foo = _tmp(foo)

_tmp = bar clearly doesn't do anything special

However, you're confusing that with examples that look more like:

@bar()  # NOTE THE PARENTHESES
def foo():
	....

which are equivalent to:

_tmp = bar()
def foo():
	....
foo = _tmp(foo)

-bob
From pje at telecommunity.com  Sun Aug 22 01:12:57 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun Aug 22 01:12:42 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: <cg8jr5$r8r$16@sea.gmane.org>
References: <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com>
	<cg8aef$dek$1@sea.gmane.org> <cg7t9r$r8r$4@sea.gmane.org>
	<cg8aef$dek$1@sea.gmane.org> <cg8e0u$r8r$9@sea.gmane.org>
	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>
	<cg8f4p$r8r$11@sea.gmane.org>
	<ACA4E2BD-F3BB-11D8-845D-000A95686CD8@redivi.com>
	<cg8i2a$r8r$13@sea.gmane.org>
	<7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com>
Message-ID: <5.1.1.6.0.20040821190916.03c6e610@mail.telecommunity.com>

At 06:54 PM 8/21/04 -0400, Paul Morrow wrote:
>Thanks.  Of the 5 examples there, the first two are apparently not 
>implemented correctly, as they expect that the function/class to be 
>decorated is passed directly to them, rather than to the function they 
>return.  Would you agree?  I pasted them here for your consideration...

They're correct.  You're missing the fact that '@x' and '@x()' are not the 
same thing.  '@x' means 'func=x(func)', while '@x()' means 'func = 
x()(func)'.  There's no inconsistency here at all, it's just ordinary 
Python semantics.

The only time a decorator needs to return a function is if it needs 
arguments other than the function being decorated.  In which case, it might 
properly be termed a decorator factory, i.e. a function returning a 
decorator.  Thus, in the '@' syntax, a decorator expression is always 
either '@decorator' or '@decorator_factory(arguments)'.

From pm_mon at yahoo.com  Sun Aug 22 01:25:30 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sun Aug 22 01:25:39 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <5.1.1.6.0.20040821190916.03c6e610@mail.telecommunity.com>
References: <7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com>	<cg8aef$dek$1@sea.gmane.org>
	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>
	<cg8e0u$r8r$9@sea.gmane.org>	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8f4p$r8r$11@sea.gmane.org>	<ACA4E2BD-F3BB-11D8-845D-000A95686CD8@redivi.com>	<cg8i2a$r8r$13@sea.gmane.org>	<7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com>
	<cg8jr5$r8r$16@sea.gmane.org>
	<5.1.1.6.0.20040821190916.03c6e610@mail.telecommunity.com>
Message-ID: <cg8llb$r8r$17@sea.gmane.org>

Phillip J. Eby wrote:

> At 06:54 PM 8/21/04 -0400, Paul Morrow wrote:
> 
>> Thanks.  Of the 5 examples there, the first two are apparently not 
>> implemented correctly, as they expect that the function/class to be 
>> decorated is passed directly to them, rather than to the function they 
>> return.  Would you agree?  I pasted them here for your consideration...
> 
> 
> They're correct.  You're missing the fact that '@x' and '@x()' are not 
> the same thing.  '@x' means 'func=x(func)', while '@x()' means 'func = 
> x()(func)'.  There's no inconsistency here at all, it's just ordinary 
> Python semantics.
> 
> The only time a decorator needs to return a function is if it needs 
> arguments other than the function being decorated.  In which case, it 
> might properly be termed a decorator factory, i.e. a function returning 
> a decorator.  Thus, in the '@' syntax, a decorator expression is always 
> either '@decorator' or '@decorator_factory(arguments)'.
> 

That's clear.  Thank you.

From pm_mon at yahoo.com  Sun Aug 22 01:28:01 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sun Aug 22 01:30:54 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <64E90FD9-F3C7-11D8-A072-000A95686CD8@redivi.com>
References: <cg8aef$dek$1@sea.gmane.org>	<cg7t9r$r8r$4@sea.gmane.org>	<cg8aef$dek$1@sea.gmane.org>	<cg8e0u$r8r$9@sea.gmane.org>	<5.1.1.6.0.20040821172009.029908c0@mail.telecommunity.com>	<cg8f4p$r8r$11@sea.gmane.org>	<ACA4E2BD-F3BB-11D8-845D-000A95686CD8@redivi.com>	<cg8i2a$r8r$13@sea.gmane.org>	<7256D31C-F3C1-11D8-A072-000A95686CD8@redivi.com>	<cg8jr5$r8r$16@sea.gmane.org>
	<64E90FD9-F3C7-11D8-A072-000A95686CD8@redivi.com>
Message-ID: <cg8lq2$r8r$18@sea.gmane.org>

Bob Ippolito wrote:

> 
> On Aug 21, 2004, at 6:54 PM, Paul Morrow wrote:
> 
>> Bob Ippolito wrote:
>>
>>> On Aug 21, 2004, at 6:24 PM, Paul Morrow wrote:
>>>
>>>>
>>>> It seems that writing a decorator is going to be a bizarre 
>>>> experience. In the example, I would need to write a function named 
>>>> 'decoration' that returns a function that will recieve a function 
>>>> (foo) to be decorated and then return a function.  Does that sound 
>>>> about right?
>>>
>>> Yes that is correct.
>>>
>>>> What would functions like 'decoration' typically look like?  Could 
>>>> you show a short code snippet?
>>>
>>> http://python.org/peps/pep-0318.html
>>
>>
>> Thanks.  Of the 5 examples there, the first two are apparently not 
>> implemented correctly, as they expect that the function/class to be 
>> decorated is passed directly to them, rather than to the function they 
>> return.  Would you agree?  I pasted them here for your consideration...
> 
> 
> No, they are correct.  You are confused.  What is expected is that the 
> result of the expression after @ is callable and takes one parameter.  
> If the expression after @ is just a name, then nothing particularly 
> exciting happens at that time.
> 
> @bar # NOTE THE LACK OF PARENTHESES
> def foo():
>     ....
> 
> is equivalent to:
> 
> _tmp = bar
> def foo():
>     ....
> foo = _tmp(foo)
> 
> _tmp = bar clearly doesn't do anything special
> 
> However, you're confusing that with examples that look more like:
> 
> @bar()  # NOTE THE PARENTHESES
> def foo():
>     ....
> 
> which are equivalent to:
> 
> _tmp = bar()
> def foo():
>     ....
> foo = _tmp(foo)
> 
> -bob

Yes, I was missing that.  Thanks for the detailed explanation.


From martin at v.loewis.de  Sun Aug 22 09:21:39 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug 22 09:21:43 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: <cg842n$r8r$5@sea.gmane.org>
References: <cg7t9r$r8r$4@sea.gmane.org>	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>
	<cg842n$r8r$5@sea.gmane.org>
Message-ID: <41284983.3000303@v.loewis.de>

Paul Morrow wrote:
>> The items you describe are not decorators, they are attributes.  The 
>> current syntax for decorators is 'x = decorator(x)'.
>>
> 
> In today's Python that's of course true, but these 'attributes' are used 
> to supply meta info about the class. They *describe* the class.  They 
> are never intended to be inherited by instances of the class.  So in 
> this sense, they are very much more like decorators than (normal) class 
> attributes.

No, that makes them completely unlike function decorators. Function
decorators do not describe something, they modify something (namely,
the thing that is bound to the function's name).

Some attributes have a run-time meaning, like __metaclass__; others,
like __author__, have not. Unfortunately, the meaning of __metaclass__
is conceptually different from decorators: the metaclass is evaluated
*before* the class is constructed; the decorators are evaluated
*after* the function is constructed.

> So providing definitions for __xxx__ attributes is not done for the same 
> purpose as the other attributes.  They are special.  Therefore, let's 
> formally acknowledge that and call them 'decorators'.

I acknowledge that they are special, and call them "special attributes".
I won't call that decorators, because that means something else.

> 2. All magic methods in today's Python are invoked 'magically' (are 
> called indirectly; not called directly by any user code).

Correct. For each special attribute, the interpreter needs advance
knowledge of the name of the attribute. Not so for decorators:
the interpreter does not need advance knowledge - you can define
your own function decorators, and Python will support them because
of the leading @.

Regards,
Martin
From jlgijsbers at planet.nl  Sun Aug 22 13:08:35 2004
From: jlgijsbers at planet.nl (Johannes Gijsbers)
Date: Sun Aug 22 13:07:10 2004
Subject: [Python-Dev] Adding 'lexists()' to os.path
Message-ID: <20040822110835.GA7452@mail.planet.nl>

I was just reviewing the patch at http://python.org/sf/941486, which proposes
adding a new 'lexists()' function to os.path. This function would return True
for dangling symbolic links, unlike os.path.exists() which returns False. One
could use 'os.path.islink(path) or os.path.exists()' as well, but that's two
stat calls instead of one.

This function is useful and efficient for fixing a bug in glob.py (which the
patch does as well) and it seems like it could be useful in tarfile.py and
tempfile.py as well. Also, a high-quality patch is already available. So, any
objections to adding this to os.path?

Johannes
From martin at v.loewis.de  Sun Aug 22 14:00:20 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug 22 14:00:24 2004
Subject: [Python-Dev] Adding 'lexists()' to os.path
In-Reply-To: <20040822110835.GA7452@mail.planet.nl>
References: <20040822110835.GA7452@mail.planet.nl>
Message-ID: <41288AD4.7060407@v.loewis.de>

Johannes Gijsbers wrote:
> This function is useful and efficient for fixing a bug in glob.py (which the
> patch does as well) and it seems like it could be useful in tarfile.py and
> tempfile.py as well. Also, a high-quality patch is already available. So, any
> objections to adding this to os.path?

Go ahead. "Equivallent" contains a typo, though.

Martin
From pf_moore at yahoo.co.uk  Sun Aug 22 15:17:00 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sun Aug 22 15:17:02 2004
Subject: [Python-Dev] Re: Adding 'lexists()' to os.path
References: <20040822110835.GA7452@mail.planet.nl>
Message-ID: <fz6f48j7.fsf@yahoo.co.uk>

Johannes Gijsbers <jlgijsbers@planet.nl> writes:

> I was just reviewing the patch at http://python.org/sf/941486, which proposes
> adding a new 'lexists()' function to os.path. This function would return True
> for dangling symbolic links, unlike os.path.exists() which returns False. One
> could use 'os.path.islink(path) or os.path.exists()' as well, but that's two
> stat calls instead of one.

As an alternative, would there be any value in cacheing the last stat
result? In a sequence like this, with multiple calls for the same
argument, many stat calls would be replaced by one. And it's likely
that the checking overhead on cache misses would be minimal compared
to the cost of the stat call, so it's an overall win.

Of course, actual tests beat idle speculation like this...

Paul.
-- 
Any sufficiently advanced technology is indistinguishable from magic.
-- Arthur C. Clarke

From pm_mon at yahoo.com  Sun Aug 22 16:15:11 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sun Aug 22 16:15:17 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <41284983.3000303@v.loewis.de>
References: <cg7t9r$r8r$4@sea.gmane.org>	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>	<cg842n$r8r$5@sea.gmane.org>
	<41284983.3000303@v.loewis.de>
Message-ID: <cga9pf$qgp$1@sea.gmane.org>

Martin v. L?wis wrote:
> Paul Morrow wrote:
> 
>>> The items you describe are not decorators, they are attributes.  The 
>>> current syntax for decorators is 'x = decorator(x)'.
>>>
>>
>> In today's Python that's of course true, but these 'attributes' are 
>> used to supply meta info about the class. They *describe* the class.  
>> They are never intended to be inherited by instances of the class.  So 
>> in this sense, they are very much more like decorators than (normal) 
>> class attributes.
> 
> 
> No, that makes them completely unlike function decorators. Function
> decorators do not describe something, they modify something (namely,
> the thing that is bound to the function's name).
> 

That's a terrible way to 'think' about this kind of programming --- 
programming thru imperative mutation.  Ugh.  It's much better to say 
"this function is synchronized and memoized" than to say "apply the 
synchronized mutator to this function, then apply the memoized mutator 
to the resulting function".   So regardless of what is going on under 
the hood, it's better to think of this technique as a function 
annotation --- something additionally stated in its description --- than 
something that we *do* to the function.


> Some attributes have a run-time meaning, like __metaclass__; others,
> like __author__, have not. Unfortunately, the meaning of __metaclass__
> is conceptually different from decorators: the metaclass is evaluated
> *before* the class is constructed; the decorators are evaluated
> *after* the function is constructed.
> 

*When* a feature gets added/applied to a class/function shouldn't matter 
to the programmer.  We've screwed up if we make them think that hard. 
They should simply be able to state which features they want a 
class/function to have (as they can now with __metaclass__, __lt__, 
etc.), and the system makes sure they are there when the class/function 
is called.  It's magical, but its easy to think about.

>> So providing definitions for __xxx__ attributes is not done for the 
>> same purpose as the other attributes.  They are special.  Therefore, 
>> let's formally acknowledge that and call them 'decorators'.
> 
> 
> I acknowledge that they are special, and call them "special attributes".
> I won't call that decorators, because that means something else.
>

But it shouldn't, because they ("special attributes" and "decorators") 
fundamentally have the same purpose --- to make something magical happen 
to the function/class.  You're getting hung up on what's going on under 
the covers and looking at the differences there.  That should be 
irrelevant to the programmer.  It's magic.  All they need to know is 
what magic happens when they use an __xxx__ attribute and what they must 
do to use it properly --- and the later should be as easy as possible.



>> 2. All magic methods in today's Python are invoked 'magically' (are 
>> called indirectly; not called directly by any user code).
> 
> 
> Correct. For each special attribute, the interpreter needs advance
> knowledge of the name of the attribute. Not so for decorators:
> the interpreter does not need advance knowledge - you can define
> your own function decorators, and Python will support them because
> of the leading @.
> 

Another way to look at that is that the interpretor has some built-in 
magic and you can create your own.  But it's all still magic.  And that 
makes it easier to think about.





From jepler at unpythonic.net  Sun Aug 22 16:18:19 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Sun Aug 22 16:18:31 2004
Subject: [Python-Dev] Re: Adding 'lexists()' to os.path
In-Reply-To: <fz6f48j7.fsf@yahoo.co.uk>
References: <20040822110835.GA7452@mail.planet.nl> <fz6f48j7.fsf@yahoo.co.uk>
Message-ID: <20040822141819.GC2348@unpythonic.net>

On Sun, Aug 22, 2004 at 02:17:00PM +0100, Paul Moore wrote:
> As an alternative, would there be any value in cacheing the last stat
> result?

Well, code like this would break:

    def wait_for_change(filename, delay=.1):
        stamp = os.stat(filename).st_mtime
        while 1:
            time.sleep(delay)
            if os.stat(filename).st_mtime != stamp: break

Hm ... 
>>> import statcache
/usr/lib/python2.3/statcache.py:8: DeprecationWarning: The statcache module is obsolete.  Use os.stat() instead.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040822/c4236a3d/attachment.pgp
From tino.lange at isg.de  Sun Aug 22 16:25:24 2004
From: tino.lange at isg.de (Tino Lange)
Date: Sun Aug 22 16:25:26 2004
Subject: [Python-Dev] Re: Adding 'lexists()' to os.path
In-Reply-To: <fz6f48j7.fsf@yahoo.co.uk>
References: <20040822110835.GA7452@mail.planet.nl> <fz6f48j7.fsf@yahoo.co.uk>
Message-ID: <4128ACD4.1090609@isg.de>

Paul Moore wrote:
> 
>>I was just reviewing the patch at http://python.org/sf/941486, which proposes
>>adding a new 'lexists()' function to os.path. This function would return True
>>for dangling symbolic links, unlike os.path.exists() which returns False. One
>>could use 'os.path.islink(path) or os.path.exists()' as well, but that's two
>>stat calls instead of one.
> 
> 
> As an alternative, would there be any value in cacheing the last stat
> result? In a sequence like this, with multiple calls for the same
> argument, many stat calls would be replaced by one. And it's likely
> that the checking overhead on cache misses would be minimal compared
> to the cost of the stat call, so it's an overall win.

Hi Paul,

you will never know how long the cached result is true. Files on disk
can change outside your python script....

I doubt that we can find a good expire time for the caching? Someone
will come up with an application in which this caching will give wrong
results...

If someone knows what he does and likes efficient programming he can
cache the result of os.stat() himself and work with that. (Like this
lexists() function is programmed internally...)

My personal opinion:
+0.5 for os.lexists() [it's useful, but we can also live without it...]
-1 for a stat cache in os.*.

Cheers,

Tino


From martin at v.loewis.de  Sun Aug 22 16:58:39 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug 22 16:58:42 2004
Subject: [Python-Dev] Re: Adding 'lexists()' to os.path
In-Reply-To: <fz6f48j7.fsf@yahoo.co.uk>
References: <20040822110835.GA7452@mail.planet.nl> <fz6f48j7.fsf@yahoo.co.uk>
Message-ID: <4128B49F.3020505@v.loewis.de>

Paul Moore wrote:
> As an alternative, would there be any value in cacheing the last stat
> result? 

That won't help (in addition to the problems it causes). One of the
stat calls must be stat(2), the other one lstat(2).

Regards,
Martin
From martin at v.loewis.de  Sun Aug 22 17:10:48 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug 22 17:10:52 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: <cga9pf$qgp$1@sea.gmane.org>
References: <cg7t9r$r8r$4@sea.gmane.org>	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>	<cg842n$r8r$5@sea.gmane.org>	<41284983.3000303@v.loewis.de>
	<cga9pf$qgp$1@sea.gmane.org>
Message-ID: <4128B778.1080503@v.loewis.de>

Paul Morrow wrote:
> That's a terrible way to 'think' about this kind of programming --- 
> programming thru imperative mutation. 

This is Pythonic. In Python, we don't have any declarations (well,
perhaps except for global). Instead, everything in a module is
executed, including class and def statements. This is an important
property, as it allows:
- conditional function definition:
   if some_condition:
     def foo():some_code
   else:
     def foo():other_code
- computed inheritance:
   if some_condition:
     base = B1
   else:
     base = B2

   class D(base):
     body
- and more

Decorators extend this pattern.

> So regardless of what is going on under 
> the hood, it's better to think of this technique as a function 
> annotation --- something additionally stated in its description --- than 
> something that we *do* to the function.

No. It is best to simultaneously think of this as descriptive *and*
imperative. I.e. for most purposes, it is sufficient to consider it
descriptive, but one always needs to be aware when it is more correct
to consider it imperative. For example, you have to understand that

   @foo
   def bar():self

might cause bar *not* to be a function. This can only be understood
if you don't think of decorators as kind of attribute.

> *When* a feature gets added/applied to a class/function shouldn't matter 
> to the programmer.  We've screwed up if we make them think that hard. 

Depends on the situation. In some cases, it does matter, and then the
programmer needs to think hard. However, she can only succeed in that
case if she is even *aware* that the declarative view on these things
is a simplification.

> They should simply be able to state which features they want a 
> class/function to have (as they can now with __metaclass__, __lt__, 
> etc.), and the system makes sure they are there when the class/function 
> is called.  It's magical, but its easy to think about.

Yes, it works most of the time. If this simplification is wrong, it
gets so terribly wrong that the programmer needs somebody else who
really understands it solves the problem. To avoid that, we should not
try to hide the semantics, but make it obvious - and simultaneously
explain that in most cases, the simplification will serve fine.

> That should be 
> irrelevant to the programmer.  It's magic.  

No, no, no. In a well-designed programming language, there is no place
for magic, in the sense that what really happens is deliberately hidden
for nobody to find. Instead, people should *know* the inner workings.
Many people know the inner workings of the current language, and they
can easily extend their mental model to decorators.

Regards,
Martin
From pm_mon at yahoo.com  Sun Aug 22 18:10:33 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sun Aug 22 18:10:44 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <4128B778.1080503@v.loewis.de>
References: <cg7t9r$r8r$4@sea.gmane.org>	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>	<cg842n$r8r$5@sea.gmane.org>	<41284983.3000303@v.loewis.de>	<cga9pf$qgp$1@sea.gmane.org>
	<4128B778.1080503@v.loewis.de>
Message-ID: <cgaghp$qgp$2@sea.gmane.org>

Martin v. L?wis wrote:

> Paul Morrow wrote:
> 
>> That's a terrible way to 'think' about this kind of programming --- 
>> programming thru imperative mutation. 
> 
> 
> This is Pythonic. In Python, we don't have any declarations (well,
> perhaps except for global). Instead, everything in a module is
> executed, including class and def statements. This is an important
> property, as it allows:
> - conditional function definition:
>   if some_condition:
>     def foo():some_code
>   else:
>     def foo():other_code
> - computed inheritance:
>   if some_condition:
>     base = B1
>   else:
>     base = B2
> 
>   class D(base):
>     body
> - and more
> 
> Decorators extend this pattern.
> 

That pattern (conditional definitions) is a recipe for disaster, and you 
should not use it if it can be avoided.  It's always best to use a 
declarative solution whenever possible and not disturb the natural 
resolution mechanism.  What you show is possible in Python (gosh, what 
isn't), but it does not 'define' Python.  It's not a salient feature of 
Python.  So it is not Pythonic (IMO).

>> So regardless of what is going on under the hood, it's better to think 
>> of this technique as a function annotation --- something additionally 
>> stated in its description --- than something that we *do* to the 
>> function.
> 
> 
> No. It is best to simultaneously think of this as descriptive *and*
> imperative. I.e. for most purposes, it is sufficient to consider it
> descriptive, but one always needs to be aware when it is more correct
> to consider it imperative. For example, you have to understand that
> 
>   @foo
>   def bar():self
> 
> might cause bar *not* to be a function. This can only be understood
> if you don't think of decorators as kind of attribute.
> 

Another recipe for disaster.  If foo can change bar into something that 
is no longer a function, then you can create defects that are 
unacceptably hard to find.  Never use that pattern either, if there is a 
less 'fancy' alternative (and there almost always will be).


>> *When* a feature gets added/applied to a class/function shouldn't 
>> matter to the programmer.  We've screwed up if we make them think that 
>> hard. 
> 
> 
> Depends on the situation. In some cases, it does matter, and then the
> programmer needs to think hard. However, she can only succeed in that
> case if she is even *aware* that the declarative view on these things
> is a simplification.
> 

Using magic is a kind of declarative programming.  But note to all 
programmers: magic isn't really 'magic', there's actually an underlying 
mechanism making the magic happen.  You can alter the way magic works if 
you really, really, really need to.  But we encourage you to first see 
if there isn't a way to solve your problem that doesn't change the magic 
(so that others have a prayer of understanding what you're doing).

>> They should simply be able to state which features they want a 
>> class/function to have (as they can now with __metaclass__, __lt__, 
>> etc.), and the system makes sure they are there when the 
>> class/function is called.  It's magical, but its easy to think about.
> 
> 
> Yes, it works most of the time. If this simplification is wrong, it
> gets so terribly wrong that the programmer needs somebody else who
> really understands it solves the problem. To avoid that, we should not
> try to hide the semantics, but make it obvious - and simultaneously
> explain that in most cases, the simplification will serve fine.
> 

No, 'how' magic works (details of the innerworkings of the interpreter) 
isn't necessary for most programmers to understand (unless of course 
you're working on the interpreter or in some other way trying to extend 
the Python system).  They only need to know what each type of magic does 
and how to use it properly.  That needs to be thoroughly documented. 
But the implementation should only be of interest to the low-level 
developers of the Python system (with rare exceptions).


>> That should be irrelevant to the programmer.  It's magic.  
> 
> 
> No, no, no. In a well-designed programming language, there is no place
> for magic, in the sense that what really happens is deliberately hidden
> for nobody to find. Instead, people should *know* the inner workings.
> Many people know the inner workings of the current language, and they
> can easily extend their mental model to decorators.
> 

Well, define magic.  If you mean automatic behavior, then I strongly 
disagree. You need the system to do things automatically for you, 
otherwise you're writing in nothing more than assembly language.  What 
kind of magic are you referring to?


From martin at v.loewis.de  Sun Aug 22 18:53:47 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug 22 18:53:51 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: <cgaghp$qgp$2@sea.gmane.org>
References: <cg7t9r$r8r$4@sea.gmane.org>	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>	<cg842n$r8r$5@sea.gmane.org>	<41284983.3000303@v.loewis.de>	<cga9pf$qgp$1@sea.gmane.org>	<4128B778.1080503@v.loewis.de>
	<cgaghp$qgp$2@sea.gmane.org>
Message-ID: <4128CF9B.4070107@v.loewis.de>

Paul Morrow wrote:
>>   @foo
>>   def bar():self
>>
>> might cause bar *not* to be a function. This can only be understood
>> if you don't think of decorators as kind of attribute.
>>
> 
> Another recipe for disaster.  If foo can change bar into something that 
> is no longer a function, then you can create defects that are 
> unacceptably hard to find.  Never use that pattern either, if there is a 
> less 'fancy' alternative (and there almost always will be).

Yet, this is what happens for staticmethod and classmethod: the result
of applying the decorator will cause bar not to be a function anymore
(but a staticmethod or a classmethod, respectively). There is no
alternative for this.

> No, 'how' magic works (details of the innerworkings of the interpreter) 
> isn't necessary for most programmers to understand (unless of course 
> you're working on the interpreter or in some other way trying to extend 
> the Python system).  They only need to know what each type of magic does 
> and how to use it properly.  That needs to be thoroughly documented. But 
> the implementation should only be of interest to the low-level 
> developers of the Python system (with rare exceptions).

In the case of decorators, atleast anybody designing one should know how
this works. They are intentionally extensible, so many more than the
Python developers proper will need to know how decorators work.

> Well, define magic.  If you mean automatic behavior, then I strongly 
> disagree. You need the system to do things automatically for you, 
> otherwise you're writing in nothing more than assembly language.  What 
> kind of magic are you referring to?

Your proposal specifically: use __foo__ for decorators, instead of
@foo. You haven't actually specified how this works, but merely asserted
that it will work, and (apparently) declared important details as
irrelevant implementation details.

Regards,
Martin
From guido at python.org  Sun Aug 22 20:58:31 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Aug 22 20:58:37 2004
Subject: [Python-Dev] Adding 'lexists()' to os.path
In-Reply-To: Your message of "Sun, 22 Aug 2004 13:08:35 +0200."
	<20040822110835.GA7452@mail.planet.nl> 
References: <20040822110835.GA7452@mail.planet.nl> 
Message-ID: <200408221858.i7MIwV502491@guido.python.org>

> I was just reviewing the patch at http://python.org/sf/941486, which
> proposes adding a new 'lexists()' function to os.path. This function
> would return True for dangling symbolic links, unlike
> os.path.exists() which returns False. One could use
> 'os.path.islink(path) or os.path.exists()' as well, but that's two
> stat calls instead of one.
> 
> This function is useful and efficient for fixing a bug in glob.py
> (which the patch does as well) and it seems like it could be useful
> in tarfile.py and tempfile.py as well. Also, a high-quality patch is
> already available. So, any objections to adding this to os.path?

Looks like you already got the go-ahead.  Just make sure with any API
changes/additions to os.path, that there are many individual *path
modules that need to be updated -- if a platform doesn't support
symlinks, lexists() should probably just be an alias for exists().

posixpath.py, ntpath.py, macpath.py, os2emxpath.py.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Sun Aug 22 21:02:54 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Aug 22 21:03:00 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: Your message of "Sun, 22 Aug 2004 12:10:33 EDT."
	<cgaghp$qgp$2@sea.gmane.org> 
References: <cg7t9r$r8r$4@sea.gmane.org>
	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>
	<cg842n$r8r$5@sea.gmane.org> <41284983.3000303@v.loewis.de>
	<cga9pf$qgp$1@sea.gmane.org> <4128B778.1080503@v.loewis.de> 
	<cgaghp$qgp$2@sea.gmane.org> 
Message-ID: <200408221902.i7MJ2sd02547@guido.python.org>

Who is Paul Morrow and why is he asserting what is Pythonic and isn't?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From barry at python.org  Sun Aug 22 21:17:21 2004
From: barry at python.org (Barry Warsaw)
Date: Sun Aug 22 21:17:04 2004
Subject: [Python-Dev] Re: Re: Re: Update PEP 292
In-Reply-To: <cg05u5$qfh$1@sea.gmane.org>
References: <1092177771.21839.54.camel@localhost>
	<cfd23o$lla$1@sea.gmane.org> <1092232536.9945.21.camel@localhost>
	<cfdbv6$pv5$1@sea.gmane.org> <1092802980.7810.29.camel@geddy.wooz.org>
	<cg05u5$qfh$1@sea.gmane.org>
Message-ID: <1093202241.25618.19.camel@geddy.wooz.org>

On Wed, 2004-08-18 at 14:07, Fernando Perez wrote:

> I'd like to argue that this form may be the most useful for common tasks, so
> you can mix and match "this is foo: $foo and this is foo.bar: $foo.bar"
> without having to worry too much about which template class you are using.  

It might be, but using attribute lookup syntax can be problematic.  I
can only relate my experiences with Mailman, where I've been using
something similar for several years, albeit with traditional
%-placeholders.

With attribute placeholders, you typically want the mapping to be
automatically created from the call site's locals and globals, otherwise
you end up having to type many things three times.  That's fine, but
there are gotchas.  First, the exact behavior of the mapping is somewhat
application dependent (do you want locals only, globals and locals, only
some specially named attributes in those namespaces, etc.).  That makes
it difficult to provide a generally useful mapping implementation in the
standard library.  And without a special mapping instance, I think
attribute placeholders are less useful.

Second, you want to be really careful about the context that attributes
are looked up in.  For example, in a CGI context, you can't simply
expose the names of variables that hold data extracted from the form,
otherwise you open yourself up to cross-site scripting attacks.  Then,
as is the case with Mailman, if the template string is really a key into
a translation catalog, you have to be careful that your translators
aren't accidentally or intentionally exposing unsafe objects through
their translated message strings.

So at the very least, I don't think we have enough application
independent use cases to know what kinds of attribute path template
support should go in the standard library.  That may change a year and a
half from now for Python 2.5, but for 2.4 I think it's enough to get the
basic support and interface right, and see what use cases emerge and
become general enough for the standard library.

> How about making a BaseTemplate class which does NOT allow $foo.bar, and having
> the default Template implement attributes?

I'm leery of making this hierarchy too deep.  Really, the only reason
PEP 292's Template and SafeTemplate classes have an inheritance
hierarchy is so that the pattern doesn't have to be written twice. 
Their __mod__()'s are quite different, and other than the pattern and
that method, there ain't much to those classes.

> Best, and thanks for keeping an open mind about this issue (string
> interpolation is one of the very few areas where python's syntax bugs me, even
> after years of daily use).

And thank you for your suggestions.  Hopefully by Python 2.5 time frame
we'll have enough experience with this stuff to be able to extend them
with other useful classes.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040822/6808d0ad/attachment.pgp
From pm_mon at yahoo.com  Sun Aug 22 22:56:19 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Sun Aug 22 22:56:27 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <200408221902.i7MJ2sd02547@guido.python.org>
References: <cg7t9r$r8r$4@sea.gmane.org>	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>	<cg842n$r8r$5@sea.gmane.org>
	<41284983.3000303@v.loewis.de>	<cga9pf$qgp$1@sea.gmane.org>
	<4128B778.1080503@v.loewis.de> <cgaghp$qgp$2@sea.gmane.org>
	<200408221902.i7MJ2sd02547@guido.python.org>
Message-ID: <cgb19j$qgp$3@sea.gmane.org>

Guido van Rossum wrote:

> Who is Paul Morrow and why is he asserting what is Pythonic and isn't?
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

I'm sorry Guido if I was out of line.  I'm just a Python user (a strong 
proponent of the language), and have not seen the definition of 
'pythonic' published anywhere, so I assumed that it was open to 
interpretation, as I've seen others make assertions that feature x or y 
"is pythonic".

Is there a sanctioned definition for that term?

Paul

From guido at python.org  Sun Aug 22 23:10:35 2004
From: guido at python.org (Guido van Rossum)
Date: Sun Aug 22 23:10:41 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already
	decorators
In-Reply-To: Your message of "Sun, 22 Aug 2004 16:56:19 EDT."
	<cgb19j$qgp$3@sea.gmane.org> 
References: <cg7t9r$r8r$4@sea.gmane.org>
	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>
	<cg842n$r8r$5@sea.gmane.org> <41284983.3000303@v.loewis.de>
	<cga9pf$qgp$1@sea.gmane.org> <4128B778.1080503@v.loewis.de>
	<cgaghp$qgp$2@sea.gmane.org>
	<200408221902.i7MJ2sd02547@guido.python.org> 
	<cgb19j$qgp$3@sea.gmane.org> 
Message-ID: <200408222110.i7MLAZJ16423@guido.python.org>

> I'm sorry Guido if I was out of line.  I'm just a Python user (a
> strong proponent of the language), and have not seen the definition
> of 'pythonic' published anywhere, so I assumed that it was open to
> interpretation, as I've seen others make assertions that feature x
> or y "is pythonic".
> 
> Is there a sanctioned definition for that term?

No, and that's intentionally so, because beauty is not absolute or
quantifyable (even if there are extreme cases where we can all agree).

All uses of pythonic should be explicitly qualified by "IMO" or
something equivalent.  And you can't assume that everybody
automatically assumes this!

Your recent posts seemed out of line to me only because you were being
very argumentative and at the same time exposed serious
misunderstanding (*).  That's usually a clue to back off...  High
frequency posters are only okay if their posts have a high information
content.

(*) Which you acknowledged, but in messages showing more poor form by
containing many dozens of lines of quoted material followed by a
single "I see".

--Guido van Rossum (home page: http://www.python.org/~guido/)
From tdelaney at avaya.com  Mon Aug 23 00:31:28 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Mon Aug 23 00:31:34 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01D59429@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

>>> If the community can *agree* on
>>> something they can support, I will listen.
>> 
>> Even if it's []-after-args?
> 
> If most people favor that over prefix @deco, sure, I'll give it
> another look.  (It better come with an implementation though.)

Wasn't this the first implementation?

Tim Delaney
From pf_moore at yahoo.co.uk  Mon Aug 23 00:37:56 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Mon Aug 23 00:37:54 2004
Subject: [Python-Dev] Re: Adding 'lexists()' to os.path
References: <20040822110835.GA7452@mail.planet.nl> <fz6f48j7.fsf@yahoo.co.uk>
	<4128ACD4.1090609@isg.de>
Message-ID: <oel223zv.fsf@yahoo.co.uk>

Tino Lange <tino.lange@isg.de> writes:

> Paul Moore wrote:

>> As an alternative, would there be any value in cacheing the last stat
>> result? In a sequence like this, with multiple calls for the same
>> argument, many stat calls would be replaced by one. And it's likely
>> that the checking overhead on cache misses would be minimal compared
>> to the cost of the stat call, so it's an overall win.

> you will never know how long the cached result is true. Files on disk
> can change outside your python script....

Doh. Sorry about that - I wasn't thinking hard enough before posting.

Back to sleep...
Paul.
-- 
Instant gratification takes too long -- Carrie Fisher

From mkc at mathdogs.com  Mon Aug 23 00:53:34 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Mon Aug 23 00:53:40 2004
Subject: [Python-Dev] Re: re.split on empty patterns
References: <20040807145142.GB2529@rogue.amk.ca>
	<lkd4qner2xq.fsf@panix2.panix.com>
	<411C2B72.4090809@ocf.berkeley.edu>
Message-ID: <lkdk6vqwzrl.fsf@panix2.panix.com>

"Brett C." <bac@OCF.Berkeley.EDU> writes:
> Mike Coleman wrote:
> 
> [SNIP]
> >     # alternative 2:
> >     re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', 'bbczz']
>                                                                 ^
> >     re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz']
> >     # alternative 3:
> >     re.structmatch(r'xxx|(?=abc)', 'zzxxxabczz') --> ['zz', '', 'bbczz']
>                                                                     ^
> >     re.structmatch(r'xxx|(?=abc)', 'zzxxxbbczz') --> ['zz', 'bbczz']
> >
> 
> I take it the first 'b' in both of the first examples for each alternative
> were supposed to be 'a'?

Yes, that's correct.  Oops.

> And as for which version, I actually like Mike's version more than the one AMK
> and Tim like.  The reason is that the '' in the middle of the example in
> question in the OP tells you where the split would have occurred had split0 (I
> like that or 'split_empty') not been turned on. That way there is no real loss
> of data between the two, but a gain with the new feature being used.


Is there something we can do to move this forward?  It seems like a couple of
people like one option and a couple the other, but I think at least we all
agree that the general feature would be a good idea.  So, should we take a
vote?  Or just go with the more conservative option, in order to get something
in the tree for 2.4?

Mike


From fperez528 at yahoo.com  Mon Aug 23 01:35:36 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Mon Aug 23 01:24:03 2004
Subject: [Python-Dev] Re: Re: Re: Re: Update PEP 292
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<1092232536.9945.21.camel@localhost> <cfdbv6$pv5$1@sea.gmane.org>
	<1092802980.7810.29.camel@geddy.wooz.org>
	<cg05u5$qfh$1@sea.gmane.org>
	<1093202241.25618.19.camel@geddy.wooz.org>
Message-ID: <cgb9ue$h70$1@sea.gmane.org>

Barry Warsaw wrote:

> On Wed, 2004-08-18 at 14:07, Fernando Perez wrote:
> 
>> I'd like to argue that this form may be the most useful for common tasks, so
>> you can mix and match "this is foo: $foo and this is foo.bar: $foo.bar"
>> without having to worry too much about which template class you are using.
> 
> It might be, but using attribute lookup syntax can be problematic.  I
> can only relate my experiences with Mailman, where I've been using
> something similar for several years, albeit with traditional
> %-placeholders.

[...]

I trust your judgment an openness on this, and indeed prudence has served python
well in the past.  I'd just like this to be available so I could handle a very
common case like:

'a local var: $var, and some object params: $self.x, $self.y, $self.z'

This can't easily be solved with the current syntax of %()s, since the
namespaces for self (self.__dict__) and locals are different, and there's no
attribute lookup.

How about making template objects callable (or providing an eval() method)?  It
would allow the following:

tpl = template('a var $foo')

print tpl -> evaluates under the current rules, by calling __str__

but:

tpl2 = template('local: $var and attr: $self.x')

print tpl2()  # or tpl2.eval(), or whatever other name

where the call has _optional_ local/global arguments?  This could satisfy the
common case while allowing evaluation in restricted contexts:

print tpl2(my_locals,my_globals)

Anyway, just some thoughts on this...

Best regards,

Fernando

From mkc at mathdogs.com  Mon Aug 23 01:25:52 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Mon Aug 23 01:26:03 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com>
	<lkdwu0bcz7r.fsf@panix2.panix.com>
	<87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp>
	<lkdacx37oxr.fsf@panix2.panix.com>
	<873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp>
	<lkdpt5x3rul.fsf@panix2.panix.com> <cfgc6b$pt4$1@sea.gmane.org>
Message-ID: <lkdfz6ewy9r.fsf@panix2.panix.com>

"Fredrik Lundh" <fredrik@pythonware.com> writes:
> well, the examples in your PEP can be written as:
> 
>     data = [line[:-1].split(":") for line in open(filename)]

Yes, in practice I would write this, too.  The example was for pedagogical
purposes, but perhaps the fact that it's not particularly useful in practice
makes it a bad example.

> and
> 
>     import ConfigParser
> 
>     c = ConfigParser.ConfigParser()
>     c.read(filename)
> 
>     data = []
>     for section in c.sections():
>         data.append((section, c.items(section)))
> 
> both of which are shorter than your structparse examples.

Hmm.  In this case it doesn't seem fair to compare a call to ConfigParser,
rather than the code in the ConfigParser module itself (or at least the subset
of it that would provide the equivalent functionality).

I used this as an example because I thought most people would be familiar with
this file format, thus saving them having to figure out some new file format
in order to follow the PEP.  In practice, there are lots of other file formats
of similar complexity that are not handled by any such special purpose module,
and structmatch would make it easy to parse them.  For example, this "SQT"
format is output by a certain mass spec analysis program:

        http://fields.scripps.edu/sequest/SQTFormat.html

There are a number of other bioinformatics programs whose output unfortunately
must be scraped at present.  The structmatch feature would also be useful for
these cases.  (This is what motivated the PEP.)

> and most of the one-liners in your pre-PEP can be handled with a
> combination of "match" and "finditer".

I think "findall" and "finditer" are almost useless for this kind of thing, as
they are essentially "searching" rather than "matching".  That is, they'll
happily, silently skip over garbage to get to something they like.  Since
finditer returns matches, you can always inspect the match to determine
whether anything was skipped, but this seems kind of lame compared to just
doing the right thing in the first place (i.e., matching).

> here's a 16-line helper that
> parses strings matching the "a(b)*c" pattern into a prefix/list/tail tuple.
> 
>     import re
> 
>     def parse(string, pat1, pat2):
>         """Parse a string having the form pat1(pat2)*"""
>         m = re.match(pat1, string)
>         i = m.end()
>         a = m.group(1)
>         b = []
>         for m in re.compile(pat2 + "|.").finditer(string, i):
>             try:
>                 token = m.group(m.lastindex)
>             except IndexError:
>                 break
>             b.append(token)
>             i = m.end()
>         return a, b, string[i:]
> 
> >>> parse("hello 1 2 3 4 # 5", "(\w+)", "\s*(\d+)")
> ('hello', ['1', '2', '3', '4'], ' # 5')

No offense, but this code makes me cringe.  The "|." trick seems like a
horrific hack, and I'd need to stare at this code for quite a while to
convince myself that it doesn't have some subtle flaw.  And even then it just
handles the "a(b)*c" case.  It seems like the code for more complex patterns
parsed this way would just explode in size, and would have to be written
custom for each pattern.

Mike

From mkc at mathdogs.com  Mon Aug 23 01:47:43 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Mon Aug 23 01:47:52 2004
Subject: [Python-Dev] Re: python-dev Summary for 2004-08-01 through
	2004-08-15[draft]
References: <4124506F.6040503@ocf.berkeley.edu>
Message-ID: <lkdbrh2wx9c.fsf@panix2.panix.com>

> Mike Coleman presented a pre-PEP on adding a function to re that would create
> ...
> Contributing threads:
>    - `pre-PEP: Complete, Structured Regular Expression Group Matching
> <http://mail.python.org/pipermail/python-dev/2004-August/047042.html>`__

The original PEP (which you link to here) is almost incomprehensible due to my
typos, etc.  You might want to link to the corrected version here

   http://mail.python.org/pipermail/python-dev/2004-August/047238.html

in addition, or instead.


> Mike Coleman wrote up a patch for re.split() that AM Kuchling presented to the
> list.  It adds an argument to the function to allow an empty string to be put
> into the resulting list when a match occurs, even if the match consumes no
> characters.

On this, it might be clearer to add something like

   The point is that splitting on empty patterns lets you break up text
   without losing the delimiting parts.  So, for example, splitting on
   r'(?=^S)' breaks a string into "paragraphs", where a paragraph begins with
   a line that starts with 'S'.


BTW, FWIW, I read your python-dev summaries religiously.  No way would I have
time to read python-dev itself, so your summaries are the only thing that
allows me to follow along.  Thank you!!!

Mike


From pm_mon at yahoo.com  Mon Aug 23 01:49:11 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Mon Aug 23 01:49:22 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <200408222110.i7MLAZJ16423@guido.python.org>
References: <cg7t9r$r8r$4@sea.gmane.org>	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>	<cg842n$r8r$5@sea.gmane.org>
	<41284983.3000303@v.loewis.de>	<cga9pf$qgp$1@sea.gmane.org>
	<4128B778.1080503@v.loewis.de>	<cgaghp$qgp$2@sea.gmane.org>	<200408221902.i7MJ2sd02547@guido.python.org>
	<cgb19j$qgp$3@sea.gmane.org>
	<200408222110.i7MLAZJ16423@guido.python.org>
Message-ID: <412930F7.9080605@yahoo.com>

Guido van Rossum wrote:

> 
> Your recent posts seemed out of line to me only because you were being
> very argumentative and at the same time exposed serious
> misunderstanding (*).  That's usually a clue to back off...  High
> frequency posters are only okay if their posts have a high information
> content.
> 
> (*) Which you acknowledged, but in messages showing more poor form by
> containing many dozens of lines of quoted material followed by a
> single "I see".
> 

My apologies to all.  I'll be more careful in the future.

Paul

From mkc at mathdogs.com  Mon Aug 23 01:57:20 2004
From: mkc at mathdogs.com (Mike Coleman)
Date: Mon Aug 23 01:57:30 2004
Subject: [Python-Dev] Re: Threading in the Standard Library Tour Part II
References: <002001c48325$436d9460$e841fea9@oemcomputer>
	<87fz6n3g7d.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <lkd4qmuwwtb.fsf@panix2.panix.com>

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
> >>>>> "Raymond" == Raymond Hettinger <python@rcn.com> writes:
>     Raymond> Also, multiple processors cannot boost performance
>     Raymond> because Python's Global Interpreter Lock (GIL) precludes
>     Raymond> more than one thread from running in the interpreter at
>     Raymond> the same time (this was done to simplify re-entrancy
>     Raymond> issues).
> 
> This confused me because "running" is ambiguous.  How about something
> like:
> 
>     The Python interpreter itself is not fully reentrant, so threading
>     is accomplished by interleaving the execution of Python code from
>     different threads.  Thus, use of multiple processors cannot boost
>     performance of threaded Python code, because a single instance of
>     the interpreter can only execute code on one processor at a time.
> 
> (Season to taste, fix technical inaccuracies.)  I see no reason to
> mention the GIL, an implementation detail, at all.

I somewhat prefer Raymond's version, myself.  As soon as I read "execute code
on one processor at a time", I immediately want to ask "Why?".  The mention of
the GIL more or less answers this, depending on how much you know about
interpreters.

Mike


From djc at object-craft.com.au  Mon Aug 23 02:01:02 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Mon Aug 23 02:01:21 2004
Subject: [Python-Dev] Ignoring SIGPIPE
In-Reply-To: <200408210111.i7L1Bb623898@guido.python.org>
References: <E1Bu28N-0005RW-SO@sc8-pr-cvs1.sourceforge.net>	<20040820020919.GG31470@epoch.metaslash.com>	<20040820044436.0F8503C113@coffee.object-craft.com.au>	<412586CD.2000000@object-craft.com.au>
	<20040820234005.GP31470@epoch.metaslash.com>
	<200408210111.i7L1Bb623898@guido.python.org>
Message-ID: <412933BE.6020309@object-craft.com.au>

Guido van Rossum wrote:
>>>>I agree, but I think that was copied verbatim from elsewhere in
>>>>socketmodule.c, so it was left with the aim of being bug for bug
>>>>compatible with socket.socket().
>>>
>>>Indeed.  I was not going to change something as fundamental as the 
>>>existing signal handling.  The onyl sensible thing to do was to copy the 
>>>behaviour already in place when you create a socket the "normal" way.
>>
>>You are correct.  There are two places that call 
>>signal(SIGPIPE, SIG_IGN); I find this surprising.
>>
>>Guido, you added this code in version 1.4 from 1991.  
>>You do remember the reason why, don't you?  It's only 
>>13 years ago. :-)  I'm hesitant to remove the old calls,
>>but I'm not sure I want it added to new calls if it's
>>not necessary.
>>
>>Any opinions?
> 
> 
> The only real requirement is that SIGPIPE is being ignored by default,
> and that is already taken care of by inisigs() in pythonrun.c.  So I
> think both calls can be removed from socketmodule.c, heh heh.
> 
> Originally, I was being careful, and not ignoring SIGPIPE until the
> first socket was created, hence the call there.  The idea being that
> if you have a Python program that spits lots of stuff to stdout, and
> you pipe it into head (e.g.), it won't give you an exception -- the
> SIGPIPE will just kill it and the shell will ignore that; but if
> you're using sockets, it's important to ignore SIGPIPE so too bad for
> the other feature.  But we've been ignoring SIGPIPE in the
> initialization forever, so the SIGPIPE call in socketmodule.c is
> really not needed any more.

So should I remove both signal() calls and go offline for a few months? 
  I should also fix the docstring and documentation to mention that the 
default socketpair() family is AF_UNIX.

- Dave

-- 
http://www.object-craft.com.au
From barry at python.org  Mon Aug 23 02:30:37 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 23 02:30:32 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <412626CC.4050900@ocf.berkeley.edu>
References: <1092177771.21839.54.camel@localhost>
	<cfd23o$lla$1@sea.gmane.org>	<59e9fd3a040811052152548b6b@mail.gmail.com>
	<1092232683.9945.26.camel@localhost>
	<1f7befae040811221916a955fd@mail.gmail.com>
	<1092954772.8712.147.camel@geddy.wooz.org>
	<412533F6.4000703@ocf.berkeley.edu>
	<1093014495.9258.6.camel@geddy.wooz.org>
	<412626CC.4050900@ocf.berkeley.edu>
Message-ID: <1093221037.25618.90.camel@geddy.wooz.org>

On Fri, 2004-08-20 at 12:29, Brett C. wrote:
> > What do you think about this compared to Tim's pattern, which is more
> > strict than just complaining about $'s at the end of strings?
> > 

> But Raymond is pushing for "practicality beats purity" which is more or 
> less true, but I am starting to agree with Tim that dollar amounts in an 
> interpolated string probably won't hapeen that often.  If anything the 
> number will be what is interpolated in.  So going for a more explicit 
> version is seeming to be the better solution. [...]

I agree that explicit is better than implicit.  I've added this as an
open issue in the PEP, but the implementation (patch) I'm going to post
will require throw the ValueError for all bad placeholders.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040822/15e88b1b/attachment.pgp
From barry at python.org  Mon Aug 23 02:32:56 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 23 02:32:28 2004
Subject: [Python-Dev] Re: Re: Re: Re: Update PEP 292
In-Reply-To: <cgb9ue$h70$1@sea.gmane.org>
References: <1092177771.21839.54.camel@localhost>
	<cfd23o$lla$1@sea.gmane.org> <1092232536.9945.21.camel@localhost>
	<cfdbv6$pv5$1@sea.gmane.org> <1092802980.7810.29.camel@geddy.wooz.org>
	<cg05u5$qfh$1@sea.gmane.org> <1093202241.25618.19.camel@geddy.wooz.org>
	<cgb9ue$h70$1@sea.gmane.org>
Message-ID: <1093221175.25616.93.camel@geddy.wooz.org>

On Sun, 2004-08-22 at 19:35, Fernando Perez wrote:

> Anyway, just some thoughts on this...

Good thoughts, and thanks for them, but for now I'm going to stick with
the basics and put this PEP to bed.  Feel free to write a follow on of
PEP 292 for Python 2.5 <wink>.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040822/69ac757b/attachment-0001.pgp
From fperez528 at yahoo.com  Mon Aug 23 03:00:27 2004
From: fperez528 at yahoo.com (Fernando Perez)
Date: Mon Aug 23 02:47:00 2004
Subject: [Python-Dev] Re: Re: Re: Re: Re: Update PEP 292
References: <1092177771.21839.54.camel@localhost> <cfd23o$lla$1@sea.gmane.org>
	<1092232536.9945.21.camel@localhost> <cfdbv6$pv5$1@sea.gmane.org>
	<1092802980.7810.29.camel@geddy.wooz.org>
	<cg05u5$qfh$1@sea.gmane.org>
	<1093202241.25618.19.camel@geddy.wooz.org>
	<cgb9ue$h70$1@sea.gmane.org>
	<1093221175.25616.93.camel@geddy.wooz.org>
Message-ID: <cgbepv$rn$1@sea.gmane.org>

Barry Warsaw wrote:

> On Sun, 2004-08-22 at 19:35, Fernando Perez wrote:
> 
>> Anyway, just some thoughts on this...
> 
> Good thoughts, and thanks for them, but for now I'm going to stick with
> the basics and put this PEP to bed.  Feel free to write a follow on of
> PEP 292 for Python 2.5 <wink>.

No prob.  Thanks for listening, but don't drop these emails from your pep-292
folder :)

Best,

f

From python at rcn.com  Mon Aug 23 02:47:14 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug 23 02:47:55 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1093221037.25618.90.camel@geddy.wooz.org>
Message-ID: <006d01c488aa$bbf91e60$e841fea9@oemcomputer>

> ... the implementation (patch) I'm going to post
> will require throw the ValueError for all bad placeholders.

Since you're overriding the mod operator, it may be best to emulate its
behavior which is to throw a KeyError:



>>> '%(bingo)s' % {'bongo':'xxx'}

Traceback (most recent call last):
  File "<pyshell#1>", line 1, in -toplevel-
    '%(bingo)s' % {'bongo':'xxx'}
KeyError: 'bingo'



Raymond

From barry at python.org  Mon Aug 23 02:50:32 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 23 02:50:04 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <006d01c488aa$bbf91e60$e841fea9@oemcomputer>
References: <006d01c488aa$bbf91e60$e841fea9@oemcomputer>
Message-ID: <1093222232.28570.1.camel@geddy.wooz.org>

On Sun, 2004-08-22 at 20:47, Raymond Hettinger wrote:
> > ... the implementation (patch) I'm going to post
> > will require throw the ValueError for all bad placeholders.
> 
> Since you're overriding the mod operator, it may be best to emulate its
> behavior which is to throw a KeyError:
> 
> 
> 
> >>> '%(bingo)s' % {'bongo':'xxx'}
> 
> Traceback (most recent call last):
>   File "<pyshell#1>", line 1, in -toplevel-
>     '%(bingo)s' % {'bongo':'xxx'}
> KeyError: 'bingo'

That's a different error, and one that Template still throws, if a
placeholder can't be found in the mapping.  The ValueError is thrown
when an invalid placeholder specification is found.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040822/372a6fde/attachment.pgp
From fumanchu at amor.org  Mon Aug 23 04:40:31 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Mon Aug 23 04:45:54 2004
Subject: [Python-Dev] Multiple decorators per line
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E32@exchange.hqamor.amorhq.net>

Guido,

Are multiple decorators per line allowed in the current (pie) syntax?
Should they be? They seem to be enabled in CVS, but nobody is verbally
promoting them, and the wiki actually points out the "single deco per
line" as a positive feature of the current syntax (A1).


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From python at rcn.com  Mon Aug 23 04:54:18 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug 23 04:55:02 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <1093222232.28570.1.camel@geddy.wooz.org>
Message-ID: <001a01c488bc$7bdc97a0$e841fea9@oemcomputer>

> That's a different error, and one that Template still throws, if a
> placeholder can't be found in the mapping.  The ValueError is thrown
> when an invalid placeholder specification is found.

Got it!


Raymond

From guido at python.org  Mon Aug 23 05:17:29 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug 23 05:17:40 2004
Subject: [Python-Dev] Ignoring SIGPIPE
In-Reply-To: Your message of "Mon, 23 Aug 2004 10:01:02 +1000."
	<412933BE.6020309@object-craft.com.au> 
References: <E1Bu28N-0005RW-SO@sc8-pr-cvs1.sourceforge.net>
	<20040820020919.GG31470@epoch.metaslash.com>
	<20040820044436.0F8503C113@coffee.object-craft.com.au>
	<412586CD.2000000@object-craft.com.au>
	<20040820234005.GP31470@epoch.metaslash.com>
	<200408210111.i7L1Bb623898@guido.python.org> 
	<412933BE.6020309@object-craft.com.au> 
Message-ID: <200408230317.i7N3HT716938@guido.python.org>

> So should I remove both signal() calls and go offline for a few months? 

Yes to the former; to the latter, I don't know what that's referring to.

>   I should also fix the docstring and documentation to mention that the 
> default socketpair() family is AF_UNIX.

Of course.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org  Mon Aug 23 05:27:07 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug 23 05:27:14 2004
Subject: [Python-Dev] Multiple decorators per line
In-Reply-To: Your message of "Sun, 22 Aug 2004 19:40:31 PDT."
	<3A81C87DC164034AA4E2DDFE11D258E3022E32@exchange.hqamor.amorhq.net> 
References: <3A81C87DC164034AA4E2DDFE11D258E3022E32@exchange.hqamor.amorhq.net>
Message-ID: <200408230327.i7N3R7717007@guido.python.org>

> Are multiple decorators per line allowed in the current (pie) syntax?

Yes in the 2.4a2 release, no in the PEP and current CVS.  This was
decided quickly and with full agreement of all around shortly after a2
was released.

> Should they be? They seem to be enabled in CVS, but nobody is verbally
> promoting them, and the wiki actually points out the "single deco per
> line" as a positive feature of the current syntax (A1).

I originally thought it would be useful to have a compact form for
multiple short parameterless decorators, but in the end decided that
it wasn't worth it.

This is along the same lines of my current thinking that allowing

  if condition: statement

instead of

  if condition:
      statement

was a mistake (the statement is too hidden, especially if it's a break
or return).

--Guido van Rossum (home page: http://www.python.org/~guido/)
From djc at object-craft.com.au  Mon Aug 23 05:42:30 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Mon Aug 23 05:42:37 2004
Subject: [Python-Dev] Ignoring SIGPIPE
In-Reply-To: <200408230317.i7N3HT716938@guido.python.org>
References: <E1Bu28N-0005RW-SO@sc8-pr-cvs1.sourceforge.net>
	<20040820020919.GG31470@epoch.metaslash.com>
	<20040820044436.0F8503C113@coffee.object-craft.com.au>
	<412586CD.2000000@object-craft.com.au>
	<20040820234005.GP31470@epoch.metaslash.com>
	<200408210111.i7L1Bb623898@guido.python.org>
	<412933BE.6020309@object-craft.com.au>
	<200408230317.i7N3HT716938@guido.python.org>
Message-ID: <412967A6.7080002@object-craft.com.au>

Guido van Rossum wrote:
>>So should I remove both signal() calls and go offline for a few months? 
> 
> 
> Yes to the former; to the latter, I don't know what that's referring to.

Go offline so I can't see the fallout from the damage I have caused :-).

>>  I should also fix the docstring and documentation to mention that the 
>>default socketpair() family is AF_UNIX.
> 
> 
> Of course.

Good.  I will make the changes.

- Dave

-- 
http://www.object-craft.com.au
From guido at python.org  Mon Aug 23 07:45:21 2004
From: guido at python.org (Guido van Rossum)
Date: Mon Aug 23 07:45:28 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/nondist/peps pep-0020.txt, NONE, 1.1
In-Reply-To: Your message of "Sun, 22 Aug 2004 20:41:24 PDT."
	<E1Bz5i0-0004Ob-Nw@sc8-pr-cvs1.sourceforge.net> 
References: <E1Bz5i0-0004Ob-Nw@sc8-pr-cvs1.sourceforge.net> 
Message-ID: <200408230545.i7N5jMC17212@guido.python.org>

A good idea!  What made you think of doing this?

--Guido van Rossum (home page: http://www.python.org/~guido/)
From mwh at python.net  Mon Aug 23 13:08:10 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug 23 13:08:12 2004
Subject: [Python-Dev] PEP 318: Can't we all just get along?
In-Reply-To: <200408202007.i7KK71t23456@guido.python.org> (Guido van
	Rossum's message of "Fri, 20 Aug 2004 13:07:01 -0700")
References: <200408200140.i7K1eFZh027162@cosc353.cosc.canterbury.ac.nz>
	<200408202007.i7KK71t23456@guido.python.org>
Message-ID: <2my8k66rj9.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> > If the community can *agree* on
>> > something they can support, I will listen.
>> 
>> Even if it's []-after-args?
>
> If most people favor that over prefix @deco, sure, I'll give it
> another look.  (It better come with an implementation though.)

It has one, mine (well, at least the grammar and compiler changes).

Anthony keeps accusing my posting of this patch two-and-a-half years
ago (!) of starting this whole mudpie...

Cheers,
mwh

-- 
  About the use of language: it is impossible to sharpen a
  pencil with a blunt axe.  It is equally vain to try to do
  it with ten blunt axes instead.
      -- E.W.Dijkstra, 18th June 1975. Perl did not exist at the time.
From fredrik at pythonware.com  Mon Aug 23 13:41:26 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Aug 23 13:39:45 2004
Subject: [Python-Dev] Re: pre-PEP [corrected]: Complete,
	Structured Regular Expression Group Matching
References: <lkdekmlvv8k.fsf@panix2.panix.com><lkdwu0bcz7r.fsf@panix2.panix.com><87r7qgkccc.fsf@tleepslib.sk.tsukuba.ac.jp><lkdacx37oxr.fsf@panix2.panix.com><873c2ugir2.fsf@tleepslib.sk.tsukuba.ac.jp><lkdpt5x3rul.fsf@panix2.panix.com>
	<cfgc6b$pt4$1@sea.gmane.org> <lkdfz6ewy9r.fsf@panix2.panix.com>
Message-ID: <cgcl1r$7gf$1@sea.gmane.org>

Mike Coleman wrote:

>> >>> parse("hello 1 2 3 4 # 5", "(\w+)", "\s*(\d+)")
>> ('hello', ['1', '2', '3', '4'], ' # 5')
>
> No offense, but this code makes me cringe.  The "|." trick seems like a
> horrific hack

"or any other character" a horrific hack?  that's RE 101, really.

and you're supposed to *use* the code, not stare at the implementation.
programming is all about abstractions, remember?

</F> 



From cce at clarkevans.com  Mon Aug 23 14:58:48 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Mon Aug 23 14:58:50 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <001f01c48643$5985faa0$e841fea9@oemcomputer>
References: <1092954772.8712.147.camel@geddy.wooz.org>
	<001f01c48643$5985faa0$e841fea9@oemcomputer>
Message-ID: <20040823125848.GA47863@prometheusresearch.com>

On Thu, Aug 19, 2004 at 07:22:09PM -0400, Raymond Hettinger wrote:
|     IDLE 1.1a2      
|     >>> from Cheetah.Template import Template
|     >>> t = Template('the $rank is $@ a $$ dead ${rank}! $')
|     >>> t.rank='king'
|     >>> print t
|     the king is $@ a $$ dead king! $

I second this behavior.  Besides the missing 's', the second most common
error is not escaping the %, for example, <table width="100%"> is
painful, especially when you go through two or three stages of
substitution.  The behavior of Cheetah reflects real-world experience.
Most of the time your templates are edited by HTML monkeys or people
writing form letters or help screens.   If a substitution fails, it will
be obvious. This is one case where being explict causes more problems
than it fixes.

I've found Cheetah's implementation to be quite useful and would hope
that any 'standard' template mechanism would be advised by the good work
Chuck Esterbrook and Mike Orr.  The other mechanism I like is the dotted
notation to traverse objects.

Also, I really like Cheetah's mechanism where the template object
doubles as a dictionary of values.  In particular, it would be nice to
put default values, during substitution, it would first look in the
dictionary applied via the % operator.  If a key is not found in the
applied dictionary, it would use the values given by the template
itself.   And, if cast to a string, it would only use the template's key
values.  If this feature was implemented, it'd be great to use keyword
arguments as default mapping values.   

  >>> t = Template("$foo $bar $bing",foo='wibbles',bar='womble')
  >>> print t
  wibbles womble $bing
  >>> print t % {'bar': 'bingles'}
  wibbles bingles $bing
  >>> t.bing = 'wicken'
  >>> t
  wibbles womble wicken

Cheers,

Clark
From pinard at iro.umontreal.ca  Mon Aug 23 15:20:43 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Mon Aug 23 16:02:51 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <1093232832.28573.16.camel@geddy.wooz.org>
References: <1093232832.28573.16.camel@geddy.wooz.org>
Message-ID: <20040823132043.GA4501@titan.progiciels-bpi.ca>

[Barry Warsaw]

> Attached is the latest version of PEP 292, which describes simpler
> string substitutions.  This PEP is slated for inclusion in Python 2.4
> and will likely be checked in before the next alpha release.

Hi, people.

The PEP was sent to python-list, but I'm replying to python-dev.  As
the PEP is "slated for inclusion in Python 2.4", I presume it was sent
for information purposes, and not at all as a call for discussion or
feedback.  Nevertheless, here at least, I dare a few comments. :-)

I like the avenue taken by this PEP, especially about _not_ altering
Python syntax, and caring about internationalisation issues.  Yet
I'm rather -0.1 on the whole thing, considering that implementation
might not fully meet its official goal, which is making string less
error-prone, as per quote 1) below.  However, consider quote 2) and 3):

Quote 1)

>     This PEP is "simpler" in two respects:

>     1. Python's current string substitution feature (i.e.
>     %-substitution) is complicated and error prone.

Quote 2)

> Open Issues

>       >>> from string import Template
>       >>> Template('The cost was $amount euros') % {'amount': 7}

>       Proposed resolution: no automatic stringification.

Quote 3)

>     - Should the $-placeholder rules be more strict?

>       Proposed resolution: There seems to be consensus for strictness
>       on the grounds of explicit is better than implicit.

These two quotes let me think that people will often forget to stringify
numbers and various other value types, or to escape a few isolated `$'.
This is a lost compared to the current `%(key)s' which is much more
forgiving and helpful, in that it always stringify the value.  So, the
feature implemented by this PEP will _also_ be a bit complicated and
error prone.  We might not be gaining much overall, and yet, we loose
the simplicity of having only one main interpolation mechanism.

Of course, the PEP is accepted, and my opinion has no effect, so I'll
stick on one or two details.

I have the fuzzy feeling that one unstated, but important goal of the
PEP is to find some way to push `$' forward for stringification in
Python.  The whole PEP might even be a stunt or justification for this
unstated goal.  However, as this goal is well reached, it might somehow
have contributed to the PEP acceptation and counterweight the deficiency
stated in the preceding paragraph.  If my feeling is right, then the PEP
should clearly explicit this goal, it will make the PEP stronger.

>       There seems little consensus around either suggestion, and
>       since the classes are just a few lines of Python, I propose no
>       string module re-organization, but to add these two classes to
>       string.py.

I witnessed a serious effort in this developers' group to "empty" the
`string' module in favour of string methods.  That effort has been
abandoned?  I also personally think the word `string' should be left
free, in the long run, for Python programmers to use, on the ground that
modules should not be named after common words programmers would like to
keep to themselves as variable names.  So, I think _nothing_ should be
added to the `string' module, with the effect of nailing it deeper in
the library, even if there is a need to rush some solution in.

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard
From ilya at bluefir.net  Mon Aug 23 17:07:42 2004
From: ilya at bluefir.net (Ilya Sandler)
Date: Mon Aug 23 17:07:45 2004
Subject: [Python-Dev] #ifdeffery 
Message-ID: <Pine.LNX.4.58.0408230806520.770@bagira>


On Fri, 13 Aug 2004, Michael Hudson wrote:

> "Raymond Hettinger" <python@rcn.com> writes:
>
> > P.S.  Side rant:  Also, in the last few months, the #ifdefs have
> > multiplied.  While I understand that some things like time stamping are
> > worthwhile, the code is getting harder to read and maintain because of
> > all this b.s.
>
> Is it really getting worse?
>
> I think there are two qualitatively different sorts of #ifdef mess:
> feature ones, like the WITH_TSC I've recently been fiddling with
> recently and portability ones.

Actually, I have a suggestion regarding
#ifdef WITH_TSC

blocks in ceval.c

Most of these blocks look like this:

 #ifdef WITH_TSC
   rdtscll(inst0);
 #endif

where rdtscll() is defined in the beginning of ceval.c as follows

 #ifdef WITH_TSC
 ...
 #define rdtscll(var) ppc_getcounter(&var)
 ...
 #else /* this section is for linux/x86 */
 ...
 #endif

so why not to add a dummy rtdscll() macro to the else section of the
above ifdef:

 #else /* this section is for linux/x86 */
 #define rdtscll(var)
 #endif

Then one can simply omit ifdef brackets for rdtscll invocations, removing
about 20 ifdefs....

there are probably other places in the code where this trick could be used

Does this make sense or am I missing something here?

Ilya







> The feature #ifdefs aren't (or shouldn't be) that much of a big deal.
> Ideally, they are documented in Misc/SpecialBuilds.txt and depending
> on whether they are defined or not, bits and pieces of code are or are
> not included.
>

>
From mwh at python.net  Mon Aug 23 17:14:01 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug 23 17:14:02 2004
Subject: [Python-Dev] #ifdeffery
In-Reply-To: <Pine.LNX.4.58.0408230806520.770@bagira> (Ilya Sandler's
	message of "Mon, 23 Aug 2004 08:07:42 -0700 (PDT)")
References: <Pine.LNX.4.58.0408230806520.770@bagira>
Message-ID: <2mpt5h7upy.fsf@starship.python.net>

Ilya Sandler <ilya@bluefir.net> writes:

> On Fri, 13 Aug 2004, Michael Hudson wrote:
>
>> "Raymond Hettinger" <python@rcn.com> writes:
>>
>> > P.S.  Side rant:  Also, in the last few months, the #ifdefs have
>> > multiplied.  While I understand that some things like time stamping are
>> > worthwhile, the code is getting harder to read and maintain because of
>> > all this b.s.
>>
>> Is it really getting worse?
>>
>> I think there are two qualitatively different sorts of #ifdef mess:
>> feature ones, like the WITH_TSC I've recently been fiddling with
>> recently and portability ones.
>
> Actually, I have a suggestion regarding
> #ifdef WITH_TSC
>
> blocks in ceval.c
>
> Most of these blocks look like this:
>
>  #ifdef WITH_TSC
>    rdtscll(inst0);
>  #endif
>
> where rdtscll() is defined in the beginning of ceval.c as follows
>
>  #ifdef WITH_TSC
>  ...
>  #define rdtscll(var) ppc_getcounter(&var)
>  ...
>  #else /* this section is for linux/x86 */
>  ...
>  #endif
>
> so why not to add a dummy rtdscll() macro to the else section of the
> above ifdef:
>
>  #else /* this section is for linux/x86 */
>  #define rdtscll(var)
>  #endif
>
> Then one can simply omit ifdef brackets for rdtscll invocations, removing
> about 20 ifdefs....
>
> there are probably other places in the code where this trick could be used
>
> Does this make sense or am I missing something here?

No, this definitely makes sense to me.  I'd prefer to call the macro
something more transparent, though.  One of the other things I didn't
do when porting this stuff to PPC was changing the names so that it's
less Pentium specific.  Patches welcome :-)

Cheers,
mwh

-- 
  Python enjoys making tradeoffs that drive *someone* crazy <wink>.
                                       -- Tim Peters, comp.lang.python
From cce at clarkevans.com  Mon Aug 23 17:39:23 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Mon Aug 23 17:39:25 2004
Subject: [Python-Dev] Minimal 'stackless' PEP using generators?
Message-ID: <20040823153923.GA50408@prometheusresearch.com>

I just read the thread 'Stackless Python' in June 2004 on python-dev and
was wondering if you'd comment on a simpler cooperative mechanism, via a
small hack to generators:

1.  The PEP would introduce a new 'builtin' class called 'Cooperate'

2.  Generator semantics would be altered so that 'yield X', where X
    is an instance of Cooperate, would automagically propigate to
    the outer-most non-generator.

In for example,

    >>> def MyCooperate(magic.Cooperate):
    >>>     __str__(self): return "cooperate"
    >>>
    >>> def top():
    >>>     yield "one"
    >>>     yield MyCooperate()
    >>>     yield "two"
    >>>
    >>> def middle():
    >>>     """ intermediate generator _only_ sees one and two """
    >>>     for x in top():
    >>>         print "middle", x
    >>>         yield x
    >>>
    >>> def lower():
    >>>     """ this is not a generator, so it sees cooperate """
    >>>     for x in middle():
    >>>         print "lower", x
    >>>
    >>> lower()

    middle one
    lower one
    lower cooperate
    middle two
    lower two

With these two changes, the "lower" function could be an async reactor
like those found in Twisted, or async core.  While the result isn't true
coutines, it would be a huge improvement for those who would like to do
async coding.   I've done something similar with Twisted called Flow [1]
and it works well, with the exception of being a painful syntax hack and
being quite slow.   If this was moved into Python's core, we'd get most
of the advantages of coroutines without the full cost.

Thoughts?

Clark

[1] http://twistedmatrix.com/documents/current/howto/flow

-- 
Clark C. Evans                      Prometheus Research, LLC.
                                    http://www.prometheusresearch.com/
    o                               office: +1.203.777.2550 
  ~/ ,                              mobile: +1.203.444.0557 
 //
((   Prometheus Research: Transforming Data Into Knowledge
 \\  ,
   \/    - Research Exchange Database
   /\    - Survey & Assessment Technologies
   ` \   - Software Tools for Researchers
    ~ *
From mcherm at mcherm.com  Mon Aug 23 17:54:46 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Aug 23 17:51:30 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
Message-ID: <1093276486.412a1346e30e9@mcherm.com>

Fran?ois Pinard writes:
> [quotations from the PEP motivation] let me think that people will often
> forget to stringify numbers and various other value types,

I agree. I'd like to raise a small voice in favor of automatically
calling str() on all arguments before interpolation. I'd rather
type this:

    # ... code ...
    out.write( template("$foo\t$bar\t$baz\n") % locals() )

than this:

    # ... code ...
    out.write( template("$foo\t$bar\t$baz\n") %
        {'foo': str(foo), 'bar': str(bar), 'baz': str(baz)} )

-- Michael Chermside

From pje at telecommunity.com  Mon Aug 23 17:56:04 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug 23 17:55:50 2004
Subject: [Python-Dev] Minimal 'stackless' PEP using generators?
In-Reply-To: <20040823153923.GA50408@prometheusresearch.com>
Message-ID: <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>

At 11:39 AM 8/23/04 -0400, Clark C. Evans wrote:
>I just read the thread 'Stackless Python' in June 2004 on python-dev and
>was wondering if you'd comment on a simpler cooperative mechanism, via a
>small hack to generators:
>
>1.  The PEP would introduce a new 'builtin' class called 'Cooperate'
>
>2.  Generator semantics would be altered so that 'yield X', where X
>     is an instance of Cooperate, would automagically propigate to
>     the outer-most non-generator.

Perhaps you mean "inner-most"?

>With these two changes, the "lower" function could be an async reactor
>like those found in Twisted, or async core.  While the result isn't true
>coutines, it would be a huge improvement for those who would like to do
>async coding.   I've done something similar with Twisted called Flow [1]
>and it works well, with the exception of being a painful syntax hack and
>being quite slow.   If this was moved into Python's core, we'd get most
>of the advantages of coroutines without the full cost.
>
>Thoughts?

It doesn't seem to me to actually help anything.  You can already do this 
using a simple wrapper object that maintains a stack of active generators, 
as I do in 'peak.events'.

I was hoping that you had actually come up with a solution for the more 
complex problem of suspending *non* generator functions, in a way that 
would work with CPython.  :(

From cce at clarkevans.com  Mon Aug 23 18:12:33 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Mon Aug 23 18:12:36 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <1093276486.412a1346e30e9@mcherm.com>
References: <1093276486.412a1346e30e9@mcherm.com>
Message-ID: <20040823161233.GB71531@prometheusresearch.com>

+1 to automatic str() -- this is a common newbie mistake and I can't
think of a case where you wouldn't want this to happen

On Mon, Aug 23, 2004 at 08:54:46AM -0700, Michael Chermside wrote:
| Fran?ois Pinard writes:
| > [quotations from the PEP motivation] let me think that people will often
| > forget to stringify numbers and various other value types,
| 
| I agree. I'd like to raise a small voice in favor of automatically
| calling str() on all arguments before interpolation. I'd rather
| type this:
| 
|     # ... code ...
|     out.write( template("$foo\t$bar\t$baz\n") % locals() )
| 
| than this:
| 
|     # ... code ...
|     out.write( template("$foo\t$bar\t$baz\n") %
|         {'foo': str(foo), 'bar': str(bar), 'baz': str(baz)} )
From mcherm at mcherm.com  Mon Aug 23 18:22:03 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Aug 23 18:18:47 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <1093278123.412a19ab99d0b@mcherm.com>

Skip writes:
> Any thoughts about posting this to c.l.py as well as python-dev?  Maybe it
> would encourage some more participation...

I don't want to exercise a veto here or anything, but I don't think
it would be helpful. The barrier to more participation isn't that people
don't know there's work to be done. I suspect that anyone who was going
to consider helping with development would either be scratching their
own itch (ie, contributing a patch and wouldn't care about this report),
or would already be reading python-dev (after all, it's not hard to
qualify to receive it!). I worry that posting this to c.l.py would be
unlikely to generate anything other than occasional bellyaching about
how slow we are about addressing bug reports.

What might be more helpful would be a call for volunteers. We could
form a "patch reviewers" group that would review patches on SF. Or
something. Actually, figuring out how to recruit volunteers is a
difficult problem. But I'm not sure the weekly bug/patch summary is the
solution.

-- Michael Chermside

From tim.peters at gmail.com  Mon Aug 23 18:32:39 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug 23 18:32:46 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <20040823161233.GB71531@prometheusresearch.com>
References: <1093276486.412a1346e30e9@mcherm.com>
	<20040823161233.GB71531@prometheusresearch.com>
Message-ID: <1f7befae040823093264275843@mail.gmail.com>

[Clark C. Evans]
> +1 to automatic str() -- this is a common newbie mistake and I can't
> think of a case where you wouldn't want this to happen

Apologies to Barry, but I'm +1 on auto-str() too.  It's a string
interpolation -- the user is explicitly asking for a string.  If they
made a mistake, it was in asking for a string to begin with, not in
feeding it a non-string.

The same applies to string.join(iterable), for that matter.
From cce at clarkevans.com  Mon Aug 23 18:34:20 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Mon Aug 23 18:34:23 2004
Subject: [Python-Dev] Minimal 'stackless' PEP using generators?
In-Reply-To: <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
References: <20040823153923.GA50408@prometheusresearch.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
Message-ID: <20040823163420.GA94018@prometheusresearch.com>

On Mon, Aug 23, 2004 at 11:56:04AM -0400, Phillip J. Eby wrote:
| >1.  The PEP would introduce a new 'builtin' class called 'Cooperate'
| >
| >2.  Generator semantics would be altered so that 'yield X', where X
| >    is an instance of Cooperate, would automagically propigate to
| >    the outer-most non-generator.
| 
| Perhaps you mean "inner-most"?

Yes.  The top-most non-generator on the stack.

| It doesn't seem to me to actually help anything.  You can already do this 
| using a simple wrapper object that maintains a stack of active 
| generators, as I do in 'peak.events'.

Could you provide an example?  The problem this proposal solves is 
straight-foward -- it is tedious and slow to have intermediate 
generators do stuff like:

    def middle():
    """ intermediate generator _only_ sees one and two """
    for x in top():
!       if isinstance(x,X):
!           yield x
        print "middle", x
        yield x

This extra step is tedious and also slow; especially if one has lots of
yield statements that cooperate.  It could be standardized and made a
bit snappier if it was built-in behavior.   This is an 80/5 proposal.
One gets 80% of the happiness, with 5% of the pain.

| I was hoping that you had actually come up with a solution for the more 
| complex problem of suspending *non* generator functions, in a way that 
| would work with CPython.  :(

Yes, I know.  I'm trying to avoid this much harder problem.

Clark
From nas at arctrix.com  Mon Aug 23 18:38:29 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon Aug 23 18:38:33 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <1f7befae040823093264275843@mail.gmail.com>
References: <1093276486.412a1346e30e9@mcherm.com>
	<20040823161233.GB71531@prometheusresearch.com>
	<1f7befae040823093264275843@mail.gmail.com>
Message-ID: <20040823163829.GA3411@mems-exchange.org>

On Mon, Aug 23, 2004 at 12:32:39PM -0400, Tim Peters wrote:
> The same applies to string.join(iterable), for that matter.

Perhaps someone can add this as a wish for Python 3k.

  Neil
From cce at clarkevans.com  Mon Aug 23 18:41:09 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Mon Aug 23 18:41:12 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <1f7befae040823093264275843@mail.gmail.com>
References: <1093276486.412a1346e30e9@mcherm.com>
	<20040823161233.GB71531@prometheusresearch.com>
	<1f7befae040823093264275843@mail.gmail.com>
Message-ID: <20040823164109.GB94018@prometheusresearch.com>

On Mon, Aug 23, 2004 at 12:32:39PM -0400, Tim Peters wrote:
| The same applies to string.join(iterable), for that matter.

This code-snippet is littered everwhere in my applications:

    string.join([str(x) for x in iterable])

Its tedious and makes code hard to read.  Do we need a PEP to fix this?

Clark
From python at rcn.com  Mon Aug 23 18:54:33 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug 23 18:55:02 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <20040823164109.GB94018@prometheusresearch.com>
Message-ID: <002401c48931$dda69060$e841fea9@oemcomputer>

> This code-snippet is littered everwhere in my applications:
> 
>     string.join([str(x) for x in iterable])
> 
> Its tedious and makes code hard to read.  Do we need a PEP to fix
this?

A PEP would be overkill.

Still, it would be helpful to do PEP-like things such as reference
implementation, soliticing comments, keep an issue list, etc.

A minor issue is that the implementation automatically shifts to Unicode
upon encountering a Unicode string.  So you would need to test for this
before coercing to a string.

Also, join works in multiple passes.  The proposal should be specific
about where stringizing occurs.  IIRC, you need the object length on the
first pass, but the error handling and switchover to Unicode occur on
the second.



Raymond

From mcherm at mcherm.com  Mon Aug 23 19:04:55 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Aug 23 19:01:38 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
Message-ID: <1093280695.412a23b7865d8@mcherm.com>

Tim Peters writes:
> The same applies to string.join(iterable), for that matter.

Clark Evans writes:
> This code-snippet is littered everwhere in my applications:
>
>     string.join([str(x) for x in iterable])
>
> Its tedious and makes code hard to read.  Do we need a PEP to fix this?

I'm convinced! Of course any code like the following would break:

   class Item:
       # ...
       def __eq__(self, other):
           "All Items are equal!"
           return isinstance(other, Item)

   def process_item(item):
       """Processes an item, returning a string."""
       # ...

   def process_items(items):
       """This processes each Item in the list and
       returns a string containing the entire thing."""
       while True:
           try:
               return ','.join(items)
           except TypeError:
               pos = items.index(Item()) # find first Item
               items[pos] = process_item( items[pos] )
               continue

<Big Wink>

Seriously, I can't think of a sane use for <String>.join() raising
a TypeError, so I'd say run it by Guido and if it doesn't offend
his design intuition, let's fix it!

-- Michael Chermside

From pje at telecommunity.com  Mon Aug 23 19:18:28 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug 23 19:18:14 2004
Subject: [Python-Dev] Minimal 'stackless' PEP using generators?
In-Reply-To: <20040823163420.GA94018@prometheusresearch.com>
References: <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<20040823153923.GA50408@prometheusresearch.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>

At 12:34 PM 8/23/04 -0400, Clark C. Evans wrote:
>On Mon, Aug 23, 2004 at 11:56:04AM -0400, Phillip J. Eby wrote:
>| It doesn't seem to me to actually help anything.  You can already do this
>| using a simple wrapper object that maintains a stack of active
>| generators, as I do in 'peak.events'.
>
>Could you provide an example?  The problem this proposal solves is
>straight-foward -- it is tedious and slow to have intermediate
>generators do stuff like:
>
>     def middle():
>     """ intermediate generator _only_ sees one and two """
>     for x in top():
>!       if isinstance(x,X):
>!           yield x
>         print "middle", x
>         yield x
>
>This extra step is tedious and also slow; especially if one has lots of
>yield statements that cooperate.

'peak.events' uses "Task" objects that maintain a stack of active 
generators.  The Task receives yields from the "innermost" generator 
directly, without them being passed through by intermediate generators.  If 
the value yielded is *not* a control value, the Task object pops the 
generator stack, and resumes the previously suspended generator.  A "magic" 
function, 'events.resume()' retrieves the value from the Task inside the 
stopped generator.

Basically, this mechanism doesn't pass control values through multiple 
tests and generator frames: control values are consumed immediately by the 
Task.  This makes it easy to suspend nested generators while waiting for 
some event, such as socket readability, a timeout, a Twisted "Deferred", 
etc.  Yielding an "event" object like one of the aforementioned items 
causes the Task to return to its caller (the event loop) after requesting a 
callback for the appropriate event.  When the callback re-invokes the 
thread, it saves the value associated with the event, if any, for 
'events.resume()' to retrieve when the topmost generator is resumed.

Also, 'events.resume()' supports passing errors from one generator to the 
next, so that it's "as if" the generators execute in a nested fashion.  The 
drawback is that you must invoke events.resume() after each yield, but this 
is *much* less intrusive than requiring generators to pass through results 
from all nested generators.  Take a look at:

     http://cvs.eby-sarna.com/PEAK/src/peak/events/

In particular, the 'interfaces' and 'event_threads' modules.  Here's a 
usage example, a simple Task procedure:

     @events.taskFactory

     def monitorBusy(self):

         # get a "readable" event on this socket
         untilReadable = self.eventLoop.readable(self)

         while True:
             # Wait until we have stream activity
             yield untilReadable; events.resume()

             # Is everybody busy?
             if self.busyCount()==self.childCount():
                 self.supervisor.requestStart()

             # Wait until the child or busy count changes before proceeding
             yield events.AnyOf(self.busyCount,self.childCount); 
events.resume()

This task waits until a listener socket is readable (i.e. an incoming 
connection is pending), and then asks the process supervisor to start more 
processes if all the child processes are busy.  It then waits until either 
the busy count or the child process count changes, before it waits for 
another incoming connection.

Basically, if you're invoking a sub-generator, you do:

     yield subGenerator(arguments); result=events.resume()

This is if you're calling a sub-generator that only returns one "real" 
result.  You needn't worry about passing through control values, because 
the current generator won't be resumed until the subgenerator yields a 
non-control value.

If you're invoking a sub-generator that you intend to *iterate over*, 
however, and that generator can suspend on events, it's a bit more complex:

     iterator = subGenerator(arguments)

     while True:
         yield iterator; nextItem = events.resume()

         if nextItem is NOT_GIVEN:   # sentinel value
              break

         # body of loop goes here, using 'nextItem'

This is not very convenient, but I don't find it all that common to have 
data I'm iterating over in such a fashion, because 'peak.events' programs 
tends to have "infinite" streams that are organized as event sources in 
"pipe and filter" fashion.  So, you tend to end up with Tasks that only 
have one generator running anyway, except for things that are more like 
"subroutines" than real generators, because you only expect one real return 
value from them, anyway.

peak.events can work with Twisted, by the way, if you have it 
installed.  For example, this:

     yield aDeferred; result = events.resume()

suspends the generator until the Deferred fires, and then the result will 
be placed in 'result' upon resumption of the generator.  If the Deferred 
triggers an "errback", the call to 'events.resume()' will reraise the 
error, inside the current generator.

It would be nice if there were some way to "accept" data and exceptions 
within a generator that didn't require the 'events.resume' hack, e.g.:

     result = yield aDeferred

would be really nice, especially if 'result' could cause an exception to be 
raised.  I was hoping that this was something along the lines of what you 
were proposing.  E.g. if generator-iterators could take arguments to 
'next()' that would let you do this.  I believe there's already a rejected 
PEP covering the issue of communicating "into" generators.

Perhaps there should be a "simple coroutines" PEP, that doesn't try to 
extend generators into coroutines, but instead treats coroutines as a 
first-class animal that just happens to be implemented using some of the 
same techniques "under the hood".

From amk at amk.ca  Mon Aug 23 19:38:35 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Aug 23 19:38:46 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <1093278123.412a19ab99d0b@mcherm.com>
References: <1093278123.412a19ab99d0b@mcherm.com>
Message-ID: <20040823173835.GA10408@rogue.amk.ca>

On Mon, Aug 23, 2004 at 09:22:03AM -0700, Michael Chermside wrote:
> I don't want to exercise a veto here or anything, but I don't think
> it would be helpful. The barrier to more participation isn't that people
> don't know there's work to be done. I suspect that anyone who was going

I disagree.  I think there's a pool of volunteer labour in the Python
community that we aren't tapping yet.  For example, at the first bug
day the person who looked at the most bugs was someone who'd never
done anything with the Python core before (and, AFAICT, hasn't done
anything since).  In a discussion on that first day, he said
"... maybe ppl think that the tracker is only for core developers
etc. I never would have even looked there if it wasn't for today".
(That was in a discussion about automatically closing bugs once they
reach a certain age; see the transcript at
http://www.amk.ca/python/bugday/2004-06-05.html for the context.)

Posting the bug summary may result in people getting an idea of the
rate at which bugs and patches are processed.  Readers may also see a
new bug that's relevant to them, and offer debugging assistance or
commentary.  I can't see how posting the summaries to c.l.py would
*hurt*.  I say "do it".

--amk
From mwh at python.net  Mon Aug 23 19:58:02 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Aug 23 19:58:04 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <20040823173835.GA10408@rogue.amk.ca> (A. M. Kuchling's message
	of "Mon, 23 Aug 2004 13:38:35 -0400")
References: <1093278123.412a19ab99d0b@mcherm.com>
	<20040823173835.GA10408@rogue.amk.ca>
Message-ID: <2meklx7n4l.fsf@starship.python.net>

"A.M. Kuchling" <amk@amk.ca> writes:

> On Mon, Aug 23, 2004 at 09:22:03AM -0700, Michael Chermside wrote:
>> I don't want to exercise a veto here or anything, but I don't think
>> it would be helpful. The barrier to more participation isn't that people
>> don't know there's work to be done. I suspect that anyone who was going
>
> I disagree.  I think there's a pool of volunteer labour in the Python
> community that we aren't tapping yet.  For example, at the first bug
> day the person who looked at the most bugs was someone who'd never
> done anything with the Python core before (and, AFAICT, hasn't done
> anything since).  In a discussion on that first day, he said
> "... maybe ppl think that the tracker is only for core developers
> etc. I never would have even looked there if it wasn't for today".
> (That was in a discussion about automatically closing bugs once they
> reach a certain age; see the transcript at
> http://www.amk.ca/python/bugday/2004-06-05.html for the context.)
>
> Posting the bug summary may result in people getting an idea of the
> rate at which bugs and patches are processed.  Readers may also see a
> new bug that's relevant to them, and offer debugging assistance or
> commentary.  I can't see how posting the summaries to c.l.py would
> *hurt*.  I say "do it".

This mail expresses my opinions much better than the one I didn't get
around to writing :-)

Cheers,
mwh

-- 
  Presumably pronging in the wrong place zogs it.
                                        -- Aldabra Stoddart, ucam.chat
From cce at clarkevans.com  Mon Aug 23 20:33:15 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Mon Aug 23 20:33:19 2004
Subject: [Python-Dev] Minimal 'stackless' PEP using generators?
In-Reply-To: <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>
References: <5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<20040823153923.GA50408@prometheusresearch.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>
Message-ID: <20040823183315.GA84465@prometheusresearch.com>

On Mon, Aug 23, 2004 at 01:18:28PM -0400, Phillip J. Eby wrote:
| It would be nice if there were some way to "accept" data and exceptions 
| within a generator that didn't require the 'events.resume' hack, e.g.:
| 
|     result = yield aDeferred
| 
| would be really nice, especially if 'result' could cause an exception to 
| be raised.  I was hoping that this was something along the lines of what 
| you were proposing.

Perhaps it would be nice to add an alternative syntax to call a
generator when you are expecting exactly one value.

    def generator():
        yield 'one value'
    def consumer():
         value = generator()

This, when combined with the previous proposal would give:

    >>> def top():
    >>>     yield cooperate
    >>>     yield "one value"
    >>>
    >>> def middle():
    >>>     """ intermediate generator _only_ sees 'one value' """
    >>>     bing = top()
    >>>     # do something with bing
    >>>     yield bing
    >>>
    >>> def lower():
    >>>     """ this is not a generator, so it sees cooperate """
    >>>     for x in middle():
    >>>         print x
    >>>
    >>> lower()

    cooperate
    one value


| Perhaps there should be a "simple coroutines" PEP, that doesn't try to 
| extend generators into coroutines, but instead treats coroutines as a 
| first-class animal that just happens to be implemented using some of the 
| same techniques "under the hood".

The problem is maintaining a 'stack' of generators requires that
intermediate generators in the stack need a tedious hack (in both
peek.event and twisted.flow) for them to work.  If we can have python
allow messages to be sent from the top-most generator to the inner-most
non-generator (via yield cooperate, or some other magic), this
difficulty is resolved -- intermediate generators can then be written in
a operational style, without icky hacks for managing deferred execution.

Full-blown corountines arn't necessary.  A small tweak to generators
will do.

Best,

Clark
From pje at telecommunity.com  Mon Aug 23 20:59:18 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug 23 20:59:05 2004
Subject: [Python-Dev] Minimal 'stackless' PEP using generators?
In-Reply-To: <20040823183315.GA84465@prometheusresearch.com>
References: <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<20040823153923.GA50408@prometheusresearch.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com>

At 02:33 PM 8/23/04 -0400, Clark C. Evans wrote:
>Perhaps it would be nice to add an alternative syntax to call a
>generator when you are expecting exactly one value.
>
>     def generator():
>         yield 'one value'
>     def consumer():
>          value = generator()

We have that today:

     value, = generator()

Or do I misunderstand you?


>Full-blown corountines arn't necessary.  A small tweak to generators
>will do.

I don't think this is true.  Your hypothetical example can't resume 'top()' 
after it yields the "co-operate" control value, unless it either it *has* a 
stack of generators, or the Python core somehow maintains a stack of the 
executing generators.

So, my point was that since this can already be done in user-level code 
with a stack of generators, I don't see the point to adding a facility to 
Python to create hidden stacks of generators.

Instead, it would be more useful to get rid of the one piece that really is 
"magic", by providing a way to pass values or exceptions into a running 
generator.  My comment about "coroutines" was more that Guido previously 
expressed distaste for adding such a communication mechanism to generators 
as abusing the concept of a generator just being a way to implement complex 
iterators.  Therefore, I thought a coroutine proposal (backed by a suitable 
syntax and an implementation plan) might have more success.

From mcherm at mcherm.com  Mon Aug 23 21:04:09 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Aug 23 21:01:03 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <1093287849.412a3fa9ef447@mcherm.com>

amk writes:
> I disagree.  I think there's a pool of volunteer labour in the Python
> community that we aren't tapping yet.  For example [...]

Well, if there's ANY decent chance that you're right, then it's worth
sending the email. So I'll enthustiastically back the idea of sending
the bug/patch summary to c.l.py.

-- Michael Chermside

From bac at OCF.Berkeley.EDU  Mon Aug 23 21:04:19 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Aug 23 21:04:31 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <1093011363.8761.57.camel@geddy.wooz.org>
References: <002401c48645$bdf482c0$e841fea9@oemcomputer>	
	<41255DA4.1070002@ocf.berkeley.edu>
	<1093011363.8761.57.camel@geddy.wooz.org>
Message-ID: <412A3FB3.4060900@ocf.berkeley.edu>

Barry Warsaw wrote:

> On Thu, 2004-08-19 at 22:10, Brett C. wrote:
> 
> 
>>Don't let the photos from PyCON fool you; I never have a good look on my 
>>face and I tend to look stoned.
> 
> 
> Just "look"?  Dang, I really thought Tim, Fred, and I had a new recruit
> for our CrackPython project.  The first one's free, y'know.
> 

Ooh.  Tempting.  But then I would have to give up my Opium addiction to 
go back and I don't know if I can do that.  That communal feel of 
hanging out in the local Opium den at my cousin's place would be such a 
loss.  =)

> 
>>Basically I can live with having a single string module, but I would 
>>like to see some real deprecation happen.  The nice thing about the 
>>pacakge separation is it made it clear in the code what would be going 
>>away.  If we at least at PendingDeprecation to what is supposed to be 
>>taken out I will be happy.
> 
> 
> At the very least, my rewrite of libstring.tex will make it clear which
> inhabitants of the string module are going to be deprecated.  Besides,
> since it's clear that Python 3000 will take a broader look at standard
> library packagization, I'll drop this if we can agree that the PEP 292
> classes should go in the existing string module.
> 

OK, that works for me.

-Brett
From bac at OCF.Berkeley.EDU  Mon Aug 23 21:08:54 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Aug 23 21:09:04 2004
Subject: [Python-Dev] python-dev Summary for 2004-08-01
	through	2004-08-15[draft]
In-Reply-To: <3A81C87DC164034AA4E2DDFE11D258E3022E1D@exchange.hqamor.amorhq.net>
References: <3A81C87DC164034AA4E2DDFE11D258E3022E1D@exchange.hqamor.amorhq.net>
Message-ID: <412A40C6.7020405@ocf.berkeley.edu>

Robert Brewer wrote:

> Brett C. wrote:
> 
>>Option two out of all of this is people just say, "summarize what you
>>want, Brett."  Then I just cover what I find interesting and just
> 
> don't
> 
>>worry about covering a specific area.
> 
> 
> +1. Bring back "Skipped Threads" if you can, just for the sake of
> indexing completeness.
> 

Do people miss the "Skipped Threads" feature of the summaries?  Part of 
  the point of cutting down is so I can just delete the threads that 
won't be covered and cut down on my inbox.  But if people do like the 
feature then I could  stand to leave the emails in my box and have a 
list of threads I just didn't touch (sans linking, that is just not 
worth the hassle until I write a tool to generate the links automatically).

-Brett
From cce at clarkevans.com  Mon Aug 23 21:09:53 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Mon Aug 23 21:09:56 2004
Subject: [Python-Dev] Multiple decorators per line
In-Reply-To: <200408230327.i7N3R7717007@guido.python.org>
References: <3A81C87DC164034AA4E2DDFE11D258E3022E32@exchange.hqamor.amorhq.net>
	<200408230327.i7N3R7717007@guido.python.org>
Message-ID: <20040823190953.GA62618@prometheusresearch.com>

It seems that decorators solve two problems:
  - a way to move decoration close-to (or ideally on) the 'def' line
  - a way to specify N decorators together as a whole

While J2 seems nice, I'd rather see someone build a 'composite'
decorator as the function it really is, rather than introducing 
a new keyword:

    def composite(instance):
        return decorator_a(decorator_b(instance))
        
    def @composite func(args):
        """ function body """

By splitting these problems, the definition of the decorator can
be on the same line as the 'def' line, and the composite decorator
can be applied to more than one function.  Also, the operation of the
decorator is very obvious ... far less "deep magic".  

One could even use function arguments...

    def makeDecorator(use_a = False, use_b = True):
        retval = lambda instance: instance
        if use_a:
            retval = lambda instance: retval(decorator_a(instance))
        if use_b:
            retval = lambda instance: retval(decorator_b(instance))
        return retval
    
    composite = makeDecorator(use_a = True)
    def composite func(args):
        """ function body """

I suppose if one wanted more magic...


    def @makeDecorator(use_a = True) func(args):
        """ function body """

In short, I don't see a reason why this solution needs to solve #2, 
as regular functions should suffice.  Also, The problem is just #1

Cheers,

Clark
From cce at clarkevans.com  Mon Aug 23 21:18:49 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Mon Aug 23 21:18:51 2004
Subject: [Python-Dev] Minimal 'stackless' PEP using generators?
In-Reply-To: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com>
References: <5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<20040823153923.GA50408@prometheusresearch.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>
	<5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com>
Message-ID: <20040823191849.GA95532@prometheusresearch.com>

On Mon, Aug 23, 2004 at 02:59:18PM -0400, Phillip J. Eby wrote:
| >    def generator():
| >        yield 'one value'
|     value, = generator()
Oh, cool.  ;)

| > Full-blown corountines arn't necessary.  A small tweak to
| > to generators will do.
| 
| I don't think this is true.  Your hypothetical example can't resume 
| 'top()' after it yields the "co-operate" control value, unless it either 
| it *has* a stack of generators, or the Python core somehow maintains a 
| stack of the executing generators.

Yes.  The idea is for Python to maintain a stack of generators, and when
ever the top-most generator yields <magic> this yield skips the
intermediate generators on the stack, and goes immediately to the
calling function.

| So, my point was that since this can already be done in user-level code 
| with a stack of generators, I don't see the point to adding a facility to 
| Python to create hidden stacks of generators.

The problem is, if you maintain the stack of generators in userland,
then each generator on the stack must do special stuff to handle
'cooperate' (in one form or another).  You demonstrated an equivalent
problem with your peek.event code presented earlier.  While, if this is
built into the core, intermediate generators can be happily oblivious to
the cooperative magic.

| Instead, it would be more useful to get rid of the one piece that really 
| is "magic", by providing a way to pass values or exceptions into a 
| running generator.

I don't like this either.  

Clark
From python at rcn.com  Mon Aug 23 21:18:29 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug 23 21:19:50 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <412A3FB3.4060900@ocf.berkeley.edu>
Message-ID: <001101c48946$19591100$e841fea9@oemcomputer>

[Barry]
> > ... I'll drop this if we can agree that the PEP 292
> > classes should go in the existing string module.

[Brett] 
> OK, that works for me.

Does it?  Apply the patch and try "import re".  


Raymond

From python at rcn.com  Mon Aug 23 21:19:24 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug 23 21:19:58 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <412A3FB3.4060900@ocf.berkeley.edu>
Message-ID: <001201c48946$19b49e80$e841fea9@oemcomputer>

[Barry]
> > ... I'll drop this if we can agree that the PEP 292
> > classes should go in the existing string module.

[Brett] 
> OK, that works for me.

Does it?  Apply the patch and try "import re".  


Raymond

From pje at telecommunity.com  Mon Aug 23 21:34:13 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug 23 21:33:57 2004
Subject: [Python-Dev] Minimal 'stackless' PEP using generators?
In-Reply-To: <20040823191849.GA95532@prometheusresearch.com>
References: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com>
	<5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<20040823153923.GA50408@prometheusresearch.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>
	<5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040823152302.036b88b0@mail.telecommunity.com>

At 03:18 PM 8/23/04 -0400, Clark C. Evans wrote:
>On Mon, Aug 23, 2004 at 02:59:18PM -0400, Phillip J. Eby wrote:
>| >    def generator():
>| >        yield 'one value'
>|     value, = generator()
>Oh, cool.  ;)
>
>| > Full-blown corountines arn't necessary.  A small tweak to
>| > to generators will do.
>|
>| I don't think this is true.  Your hypothetical example can't resume
>| 'top()' after it yields the "co-operate" control value, unless it either
>| it *has* a stack of generators, or the Python core somehow maintains a
>| stack of the executing generators.
>
>Yes.  The idea is for Python to maintain a stack of generators, and when
>ever the top-most generator yields <magic> this yield skips the
>intermediate generators on the stack, and goes immediately to the
>calling function.

That's not the part I mean.  I'm talking about *after* that happens, when 
you *resume* the generators, and you'll need to reconstruct the stack as it 
stood when the generators were interrupted.

(Your original proposal doesn't actually mention this part at all, but I 
assume it's necessary.  After all, if all you needed was a way to stop a 
bunch of generators in their tracks, raising an exception would suffice.)


>| So, my point was that since this can already be done in user-level code
>| with a stack of generators, I don't see the point to adding a facility to
>| Python to create hidden stacks of generators.
>
>The problem is, if you maintain the stack of generators in userland,
>then each generator on the stack must do special stuff to handle
>'cooperate' (in one form or another).  You demonstrated an equivalent
>problem with your peek.event code presented earlier.  While, if this is
>built into the core, intermediate generators can be happily oblivious to
>the cooperative magic.

Though the problems are "equivalent" in complexity, the difficulty of their 
respective solutions are not at all comparable, because your approach 
either requires a technique for dynamically reconstructing the stack, or 
some similar mechanism for generators to keep track of the state of *other* 
generators when the containing generator is interrupted.

From skip at pobox.com  Mon Aug 23 21:41:43 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 23 21:42:55 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <1093278123.412a19ab99d0b@mcherm.com>
References: <1093278123.412a19ab99d0b@mcherm.com>
Message-ID: <16682.18551.966353.315826@montanaro.dyndns.org>


    Michael> I worry that posting this to c.l.py would be unlikely to
    Michael> generate anything other than occasional bellyaching about how
    Michael> slow we are about addressing bug reports.

Well:

    1. They get what they pay for.

    2. My guess is that most of the bellyaching will come from people who
       don't currently contribute.  If they want things done more quickly,
       they know where to start.

    Michael> Actually, figuring out how to recruit volunteers is a difficult
    Michael> problem. But I'm not sure the weekly bug/patch summary is the
    Michael> solution.

Until something better comes along it might pick up one or two extra pairs
of eyeballs.

Skip


From pje at telecommunity.com  Mon Aug 23 22:16:08 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Aug 23 22:15:55 2004
Subject: [Python-Dev] Minimal 'stackless' PEP using generators?
In-Reply-To: <20040823194155.GA19785@prometheusresearch.com>
References: <5.1.1.6.0.20040823152302.036b88b0@mail.telecommunity.com>
	<5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com>
	<5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<20040823153923.GA50408@prometheusresearch.com>
	<5.1.1.6.0.20040823114546.03bf44e0@mail.telecommunity.com>
	<5.1.1.6.0.20040823124135.02f0d450@mail.telecommunity.com>
	<5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com>
	<5.1.1.6.0.20040823152302.036b88b0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040823155203.022af320@mail.telecommunity.com>

At 03:41 PM 8/23/04 -0400, Clark C. Evans wrote:
>(off list)
>
>On Mon, Aug 23, 2004 at 03:34:13PM -0400, Phillip J. Eby wrote:
>| >Yes.  The idea is for Python to maintain a stack of generators, and when
>| >ever the top-most generator yields <magic> this yield skips the
>| >intermediate generators on the stack, and goes immediately to the
>| >calling function.
>|
>| That's not the part I mean.  I'm talking about *after* that happens, when
>| you *resume* the generators, and you'll need to reconstruct the stack as
>| it stood when the generators were interrupted.
>
>Yes.  The entire generator stack would have to be put back into play.

Right, but you haven't offered any proposal for where this stack will come 
from, how it will actually work, etc.

I personally don't see any way to implement your approach in CPython that 
doesn't essentially require re-implementing a portion of 'peak.events' in 
C, while adding the overhead of the control value check to *every* 
generator .next() operation, even in programs that don't use the feature.

From martin at v.loewis.de  Mon Aug 23 22:15:54 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug 23 22:16:01 2004
Subject: [Python-Dev] #ifdeffery
In-Reply-To: <2mpt5h7upy.fsf@starship.python.net>
References: <Pine.LNX.4.58.0408230806520.770@bagira>
	<2mpt5h7upy.fsf@starship.python.net>
Message-ID: <412A507A.7050203@v.loewis.de>

Michael Hudson wrote:
> No, this definitely makes sense to me.  I'd prefer to call the macro
> something more transparent, though.  One of the other things I didn't
> do when porting this stuff to PPC was changing the names so that it's
> less Pentium specific.  Patches welcome :-)

I'd like to stress that last point. Ilya, please do consider
contributing a patch. As you say, it is technically trivial, but
probably too much effort for any by-standing volunteer. I only committed
the original patch because I considered the feature "mostly harmless",
and because it had been sitting on SF for several years. I certainly
won't touch this unless
a) somebody reports a REAL bug, and Jeremy doesn't fix that, or
b) somebody contributes another patch.

Actually writing a patch, and testing it (including the two
recompilations that will take) is too much effort, so I rather
write a longish email seeking volunteers...

Regards,
Martin
From martin at v.loewis.de  Mon Aug 23 22:23:14 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug 23 22:23:26 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <1093278123.412a19ab99d0b@mcherm.com>
References: <1093278123.412a19ab99d0b@mcherm.com>
Message-ID: <412A5232.6060202@v.loewis.de>

Michael Chermside wrote:
> I don't want to exercise a veto here or anything, but I don't think
> it would be helpful. The barrier to more participation isn't that people
> don't know there's work to be done. I suspect that anyone who was going
> to consider helping with development would either be scratching their
> own itch (ie, contributing a patch and wouldn't care about this report),
> or would already be reading python-dev (after all, it's not hard to
> qualify to receive it!). 

I completely disagree (and I have to, because I was supporting more
wide distribution :-). The posting lists bug reports and patches that
some people might consider interesting, and just try out in order to
find out whether it scratches their own itches. If it does, they might
comment on the patch.

If a patch scratches nobody's itches, it is best rejected.

> I worry that posting this to c.l.py would be
> unlikely to generate anything other than occasional bellyaching about
> how slow we are about addressing bug reports.

And that would be a good thing! People *need* to remember that this is
free software, and that we are all volunteers. For some, this might
cause them to go away, with so little support from the developers.
Farewell to them! Others might consider taking over the project.
Let them come! The rest will have an occasional bellyaching, which
will remind them of how precisely free software works. They'll either
get over it, or do something about it.

> What might be more helpful would be a call for volunteers. We could
> form a "patch reviewers" group that would review patches on SF. Or
> something. Actually, figuring out how to recruit volunteers is a
> difficult problem. But I'm not sure the weekly bug/patch summary is the
> solution.

Perhaps no. However, I do think I have personally exhausted all other
possibilities, with regular postings on all available channels. It
might not help, but it won't harm, and it is easy to implement.

Regards,
Martin
From skip at pobox.com  Mon Aug 23 22:38:28 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 23 22:38:41 2004
Subject: [Python-Dev] #ifdeffery
In-Reply-To: <412A507A.7050203@v.loewis.de>
References: <Pine.LNX.4.58.0408230806520.770@bagira>
	<2mpt5h7upy.fsf@starship.python.net> <412A507A.7050203@v.loewis.de>
Message-ID: <16682.21956.821020.924013@montanaro.dyndns.org>


    Martin> Actually writing a patch, and testing it (including the two
    Martin> recompilations that will take) is too much effort, so I rather
    Martin> write a longish email seeking volunteers...

Teach them to fish...

Skip
From walter at livinglogic.de  Mon Aug 23 22:41:55 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon Aug 23 22:42:01 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <4124FA07.1070701@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>
	<4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de>
	<4124FA07.1070701@v.loewis.de>
Message-ID: <412A5693.4050008@livinglogic.de>

Martin v. L?wis wrote:

> Walter D?rwald wrote:
> 
>> Both approaches have one problem: Error handling won't
>> work for them. If the error handling is "replace", the decoder
>> should return U+FFFD for the final trailing incomplete sequence
>> in read(). This won't happen when read() never reads those
>> bytes from the input stream.
> 
> Ok. So it really looks like a final flag on read is necessary.
> 
>> Well, I had to choose a subject. ;)
> 
> I still would prefer if the two issues were discussed separately.

OK, let's come up with a patch that fixes the incomplete byte
sequences problem and then discuss non-stream APIs.

So, what should the next step be?

Bye,
    Walter D?rwald


From bac at OCF.Berkeley.EDU  Mon Aug 23 22:41:53 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Aug 23 22:42:17 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <001101c48946$19591100$e841fea9@oemcomputer>
References: <001101c48946$19591100$e841fea9@oemcomputer>
Message-ID: <412A5691.5080703@ocf.berkeley.edu>

Raymond Hettinger wrote:

> [Barry]
> 
>>>... I'll drop this if we can agree that the PEP 292
>>>classes should go in the existing string module.
> 
> 
> [Brett] 
> 
>>OK, that works for me.
> 
> 
> Does it?  Apply the patch and try "import re".  
> 

Sorry, should have been clearer.  I meant that I didn't have an issue 
with not making 'string' a package anymore with Barry strongly wording 
what is deprecated.  I have not looked at the import issue.

-Brett
From mal at egenix.com  Mon Aug 23 23:56:37 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon Aug 23 23:56:42 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <200408191838.i7JIcmm21008@guido.python.org>
References: <200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>
	<4122304E.1070907@egenix.com>	<Pine.LNX.4.58.0408172141110.4375@rnd.onego.ru>	<41227F83.4060403@egenix.com>	<Pine.LNX.4.58.0408180821240.1555@rnd.onego.ru>	<41231830.2090903@egenix.com>	<200408190139.i7J1dX819100@guido.python.org>
	<41246FA5.1010804@egenix.com>
	<200408191838.i7JIcmm21008@guido.python.org>
Message-ID: <412A6815.2050307@egenix.com>

Guido van Rossum wrote:
>>Ok, so I suppose that we can learn from Jython and IronPython in
>>this respect...
>>
>>How do they handle binary data and the interfacing between various
>>I/O facilities, e.g. files, sockets, pipes, user input, etc.
> 
> 
> I'm not sure, but I expect that in most cases they use Unicode strings
> in order to be compatibly with Python's standard library.  That's not
> the outcome I'd like to see though.  I believe Jython at least also
> has a bytes-like type (probably a thin wrapper around Java's byte
> array) that's used for interfacing to java classes.

I've had a discussion with Jack Janssen about using bytes as
default return value for I/O operations where no encoding is
specified (or unknown).

He raised the issue of bytes not being usable as dictionary
keys due to their mutability. He was also concerned about the
increase in complexity when writing programs that work with
non-text data or mixed text/data I/O.

If we want to make the move from Python 2.x to 3.0 feasable
for large code bases, then we have to do something about these
issues. It seems that the simple solution of going with Unicode
+ bytes type is not going to be a suitable approach.

Anyway, we still have 4-5 years to think about this :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 23 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From pm_mon at yahoo.com  Mon Aug 23 01:49:11 2004
From: pm_mon at yahoo.com (Paul Morrow)
Date: Tue Aug 24 00:12:58 2004
Subject: [Python-Dev] Re: __metaclass__ and __author__ are already decorators
In-Reply-To: <200408222110.i7MLAZJ16423@guido.python.org>
References: <cg7t9r$r8r$4@sea.gmane.org>	<5.1.1.6.0.20040821124536.02787270@mail.telecommunity.com>	<cg842n$r8r$5@sea.gmane.org>
	<41284983.3000303@v.loewis.de>	<cga9pf$qgp$1@sea.gmane.org>
	<4128B778.1080503@v.loewis.de>	<cgaghp$qgp$2@sea.gmane.org>	<200408221902.i7MJ2sd02547@guido.python.org>
	<cgb19j$qgp$3@sea.gmane.org>
	<200408222110.i7MLAZJ16423@guido.python.org>
Message-ID: <412930F7.9080605@yahoo.com>

Guido van Rossum wrote:

> 
> Your recent posts seemed out of line to me only because you were being
> very argumentative and at the same time exposed serious
> misunderstanding (*).  That's usually a clue to back off...  High
> frequency posters are only okay if their posts have a high information
> content.
> 
> (*) Which you acknowledged, but in messages showing more poor form by
> containing many dozens of lines of quoted material followed by a
> single "I see".
> 

My apologies to all.  I'll be more careful in the future.

Paul
From revol at free.fr  Mon Aug 23 14:26:07 2004
From: revol at free.fr (=?windows-1252?q?Fran=E7ois?= Revol)
Date: Tue Aug 24 00:13:04 2004
Subject: [Python-Dev] problem with pymalloc on the BeOS port.
Message-ID: <2246153618-BeMail@taz>

Hi there,
I'm updating the BeOS port of python to include the latest version in 
Zeta, the next incarnation of BeOS.

I'm having some problem when enabling pymalloc:

case $MAKEFLAGS in \
*-s*) LIBRARY_PATH=/boot/home/Python-2.3.4:%A/lib:/boot/home/config/
lib:/boot/beos/system/lib CC='gcc' LDSHARED='./Modules/ld_so_beos 
libpython2.3.so' OPT='-O' ./python -E ./setup.py -q build;; \
*) LIBRARY_PATH=/boot/home/Python-2.3.4:%A/lib:/boot/home/config/lib:/
boot/beos/system/lib CC='gcc' LDSHARED='./Modules/ld_so_beos 
libpython2.3.so' OPT='-O' ./python -E ./setup.py build;; \
esac
running build
running build_ext
building 'struct' extension
gcc -O -fno-strict-aliasing -I. -I/boot/home/Python-2.3.4/./Include -I/
boot/home/config/include -I/boot/home/Python-2.3.4/Include -I/boot/home
/Python-2.3.4 -c /boot/home/Python-2.3.4/Modules/structmodule.c -o 
build/temp.beos-5.1-BePC-2.3/structmodule.o
Debug memory block at address p=0x80010fb8:
    0 bytes originally requested
    The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected.
    The 4 pad bytes at tail=0x80010fb8 are not all FORBIDDENBYTE 
(0xfb):
        at tail+0: 0xdb *** OUCH
        at tail+1: 0xdb *** OUCH
        at tail+2: 0xdb *** OUCH
        at tail+3: 0xdb *** OUCH
    The block was made by call #3688627195 to debug malloc/realloc.
Fatal Python error: bad trailing pad byte


indeed, there seem to be 4 deadblock marks between the forbidden ones, 
while the len is supposed to be null:
python:dm 0x80010fb8-8 32
80010fb0  00000000 fbfbfbfb  dbdbdbdb fbfbdbdb                 
................
80010fc0  0100fbfb 507686ef  04000000 fbfbfbfb                 
......vP........
80010fd0  8013cbc8 fbfbfbfb  44ee0100 ffed0100                 
...........D....

Any clue ?

Fran?ois.
From janusfury at citlink.net  Mon Aug 23 17:30:47 2004
From: janusfury at citlink.net (Kevin Gadd)
Date: Tue Aug 24 00:13:10 2004
Subject: [Python-Dev] Python icons
Message-ID: <412A0DA7.9070306@citlink.net>

I was bored the other day so I played around with an image editor and 
redid most of the windows python icons.

Image of most of the icons (I bet this is going to murder my bandwidth):
http://luminance.org/py/screenshot.png

I did the basic one in vector. Let me know what you guys think, I figure 
since they came out pretty decent that somebody might be able to make 
something of 'em!

And, just to get in a bit of sucking up, thanks to all you devs for the 
great work you've done on the language. Keep it up!
-- 
Kevin Gadd
E-Mail <mailto:kevin@luminance.org> | Website <http://www.luminance.org/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040823/37d30992/attachment.htm
From ncoghlan at iinet.net.au  Tue Aug 24 01:49:52 2004
From: ncoghlan at iinet.net.au (ncoghlan@iinet.net.au)
Date: Tue Aug 24 01:49:57 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <002401c48931$dda69060$e841fea9@oemcomputer>
References: <002401c48931$dda69060$e841fea9@oemcomputer>
Message-ID: <1093304992.412a82a0a5a88@mail.iinet.net.au>

Quoting Raymond Hettinger <python@rcn.com>:

> > This code-snippet is littered everwhere in my applications:
> > 
> >     string.join([str(x) for x in iterable])
> > 
> > Its tedious and makes code hard to read.  Do we need a PEP to fix
> this?
> 
> A PEP would be overkill.
> 
> Still, it would be helpful to do PEP-like things such as reference
> implementation, soliticing comments, keep an issue list, etc.
> 
> A minor issue is that the implementation automatically shifts to Unicode
> upon encountering a Unicode string.  So you would need to test for this
> before coercing to a string.

Perhaps have string join coerce to string, and Unicode join coerce to the
separator's encoding. If we do that, the existing string->Unicode promotion code
should handle the switch between the two join types.

> Also, join works in multiple passes.  The proposal should be specific
> about where stringizing occurs.  IIRC, you need the object length on the
> first pass, but the error handling and switchover to Unicode occur on
> the second.

Having been digging in the guts of string join last week, I'm pretty sure the
handover to the Unicode join happens on the first 'how much space do we need'
pass (essentially, all of the work done so far is thrown away, and the Unicode
join starts from scratch. If you know you have Unicode, you're better off using
a Unicode separator to avoid this unnecessary work </tangent>).

We then have special casing of zero length and single item sequences, before
dropping into the 'build the new string' loop.

By flagging the need for a 'stringisation' operation in the failed side of the
'PyUnicode_Check' that occurs during the first pass (to see if we should hand
over to the Unicode join), we could avoid unnecessarily slowing the pure string
cases.

To keep the speed of the pure-string case, we would need to guarantee that the
sequence consists of only strings when we run through the final pass to build
the new string. So we would need an optional second pass that constructs a new
sequence, containing any strings from the original sequence, plus 'stringised'
versions of the non-strings.

The final pass could remain as-is. The only possible difference is that it may
be operating on the new 'stringised' sequence rather than the old one.

The alternative implementation (checking each item's type as it is added to the
new string in the final pass) has the significant downside of slowing down the
existing case of joining only strings.

Either implementation should still be a lot faster than ''.join([str(x) for x in
seq]) though.

Time to go knock out some code, I think. . .

Cheers,
Nick.

P.S. "'\n'.join(locals().items())" sure would be pretty, though

From tjreedy at udel.edu  Tue Aug 24 02:03:38 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue Aug 24 02:03:42 2004
Subject: [Python-Dev] Re: Weekly Python Bug/Patch Summary
References: <1093278123.412a19ab99d0b@mcherm.com>
	<412A5232.6060202@v.loewis.de>
Message-ID: <cge0kq$lqb$1@sea.gmane.org>


"Martin v. Löwis" <martin@v.loewis.de> wrote in message
news:412A5232.6060202@v.loewis.de...
> I completely disagree (and I have to, because I was supporting more
> wide distribution :-).

I also support clp posting because the little bit of help I have been able
to give (on bug reports) has mostly been prompted by the weekly reminder.
I can imagine that there are people even more competant to contribute who
only read or skim c.l.p.  If the consequences are net negative, it will be
easy to cease the additional posting.

Terry J. Reedy



From tjreedy at udel.edu  Tue Aug 24 02:33:09 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue Aug 24 02:33:14 2004
Subject: [Python-Dev] Re: Minimal 'stackless' PEP using generators?
References: <20040823153923.GA50408@prometheusresearch.com>
Message-ID: <cge2c6$p6t$1@sea.gmane.org>


"Clark C. Evans" <cce@clarkevans.com> wrote in message
news:20040823153923.GA50408@prometheusresearch.com...
> I just read the thread 'Stackless Python' in June 2004 on python-dev and
> was wondering if you'd comment

Sure

> on a simpler cooperative mechanism, via a small hack to generators:

If it is possible, as proposed, maybe at least a medium hack?

> 1.  The PEP would introduce a new 'builtin' class called 'Cooperate'
>
> 2.  Generator semantics would be altered so that 'yield X', where X
>     is an instance of Cooperate, would automagically propigate to
>     the outer-most non-generator.

-1

Skipping over the generic objections of 'hard to teach to beginners' and
'opens to door to a hundred other such specialized features':

1. This alters and complexifies the current generator concept.  At present,
'generator' (the return type of any generator function) is an instance of
the iterator concept, as is 'iterator' (the return type of iter()) and any
class with appropriate .__iter__ and .next methods.  A generator .next
method is a resumable function; yield is like return except not permanent.
The local namespace is not destroyed and instruction pointer not reset for
the next call.

>From the outside, a generator .next method is, I believe, indistinguishable
in Python (excluding possible implementation-specific voodoo) from an
interator-type .next method and behaviorally indistingueshable from any
other iterator-protocol parameterless .next method.  As far as I know, the
difference between a generator and other iterators is ease of writing
(sometimes *much* easier) and speed of execution, not external behavior.

2. This imposes an isinstance(x, magic.Collaborate) function call cost on
every yield in every generator.  (I do not believe the alternative of
imposing the same cost on every function call in every generator will work
since I do not believe that callers can dependably know whether objects are
returned or yielded.  See comments on example.)

3. 'automagic' it is.  The pertinent runtime call stack contains not
generator objects but .next methods (or in CPython, method-wrappers).  So I
presume you are proposing that yielded Collaborates magically plunk
themselves into the first frame up the stack not associated such objects.

4. If the returned to frame does not re-call the passed over suspended
frames, they and anything they hold onto are leaked until gc'ed.

>     >>> def MyCooperate(magic.Cooperate):
>     >>>     __str__(self): return "cooperate"
>     >>>
>     >>> def top():
>     >>>     yield "one"
>     >>>     yield MyCooperate()
>     >>>     yield "two"
>     >>>
>     >>> def middle():
>     >>>     """ intermediate generator _only_ sees one and two """
>     >>>     for x in top():
>     >>>         print "middle", x
>     >>>         yield x

Here, the effect you seek could be accomplished by an explicit 'if not
isinstance(x, Collaborate):' before the print statement (or, in general,
the block of code that uses x).  But what if top is rewritten as
def top(): return ("one", MyCooperate(), "two")
Do you really want the behavior of middle to change, as with your proposal?

>     >>> def lower():
>     >>>     """ this is not a generator, so it sees cooperate """
>     >>>     for x in middle():
>     >>>         print "lower", x

Now add 'if isinstance(x, Cooperate): return' and the suspended middle will
be left in limbo.  Not very tidy ;-)

Terry J. Reedy



From kbk at shore.net  Tue Aug 24 03:21:40 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue Aug 24 03:21:44 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
In-Reply-To: <1093287849.412a3fa9ef447@mcherm.com> (Michael Chermside's
	message of "Mon, 23 Aug 2004 12:04:09 -0700")
References: <1093287849.412a3fa9ef447@mcherm.com>
Message-ID: <87pt5h49gb.fsf@hydra.localdomain>

Michael Chermside <mcherm@mcherm.com> writes:

> Well, if there's ANY decent chance that you're right, then it's worth
> sending the email. So I'll enthustiastically back the idea of sending
> the bug/patch summary to c.l.py.

OK, and we can always stop if there are unforseen consequences.

Anyone else?

-- 
KBK
From cce at clarkevans.com  Tue Aug 24 03:54:53 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Tue Aug 24 03:54:56 2004
Subject: [Python-Dev] Re: Minimal 'stackless' PEP using generators?
In-Reply-To: <cge2c6$p6t$1@sea.gmane.org>
References: <20040823153923.GA50408@prometheusresearch.com>
	<cge2c6$p6t$1@sea.gmane.org>
Message-ID: <20040824015453.GA10615@prometheusresearch.com>

Thanks for responding Terry.   I'm not sure if my exposition was clear.

On Mon, Aug 23, 2004 at 08:33:09PM -0400, Terry Reedy wrote:
| > 1.  The PEP would introduce a new 'builtin' class called 'Cooperate'
| >
| > 2.  Generator semantics would be altered so that 'yield X', where X
| >     is an instance of Cooperate, would automagically propigate to
| >     the outer-most non-generator.

Suppose I have three 'frames', two generators and one function, the
call stack looks like:

     g2
     g1
     fn
     
For this to occur, fn has called .next() on g1, which has called .next()
on g2.  The proposed behavior is that "yield magic.Collaborate" provides
the value returned by g1.next() within fn.  In particular, it would be
equivalent as if every "val = g2.next()" in g1 was rewritten:

    val = g2.next()
    if isinstance(val, magic.Cooperate):
        yield val

One could do this in three ways: (a) by a convention above, this is done
in twisted.flow, (b) by a bytecode hack, or (c) by modifying eval.c
to have equivalent behavior.

| 1. This alters and complexifies the current generator concept.

Correct; it introduces something similar to a corountine.

| 2. This imposes an isinstance(x, magic.Collaborate) function call cost on
| every yield in every generator.

Ok.  So, in addition to causing unexpected results (one would have to
check to see what sort of object to debug), it would also cause a 
performance hit.

| 3. 'automagic' it is.  The pertinent runtime call stack contains not
| generator objects but .next methods (or in CPython, method-wrappers).  So I
| presume you are proposing that yielded Collaborates magically plunk
| themselves into the first frame up the stack not associated such objects.

Right, the first frame that is not a generator, aka, not having a yield
statement.  And yes, it is different semantics.

| 4. If the returned to frame does not re-call the passed over suspended
| frames, they and anything they hold onto are leaked until gc'ed.

No.  See the above equivalence.  It might take a bit of thinking
to make the "C" implementation correct, but the semantics are 
farily straight-forward and should not create leaks.

| Here, the effect you seek could be accomplished by an explicit 'if not
| isinstance(x, Collaborate):' before the print statement (or, in general,
| the block of code that uses x).  But what if top is rewritten as
| def top(): return ("one", MyCooperate(), "two")
| Do you really want the behavior of middle to change, as with your proposal?

Yes.

| >     >>> def lower():
| >     >>>     """ this is not a generator, so it sees cooperate """
| >     >>>     for x in middle():
| >     >>>         print "lower", x
| 
| Now add 'if isinstance(x, Cooperate): return' and the suspended middle will
| be left in limbo.  Not very tidy ;-)

No, it would be just like a generator had 'if 1: return'; the suspended
middle would be cleaned up just as it is today.

...

However, you've convinced me on the semantic differences and the
performance hit.  I guess the proposal would have to present a
new keyword, say 'collaborate' and have to have its semantics
clearly specified.

Thank you _so_ much for your kind review,

Clark
From tim.peters at gmail.com  Tue Aug 24 04:05:00 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 24 04:05:03 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <20040823164109.GB94018@prometheusresearch.com>
References: <1093276486.412a1346e30e9@mcherm.com>
	<20040823161233.GB71531@prometheusresearch.com>
	<1f7befae040823093264275843@mail.gmail.com>
	<20040823164109.GB94018@prometheusresearch.com>
Message-ID: <1f7befae040823190560339fff@mail.gmail.com>

[Clark C. Evans]
> This code-snippet is littered everwhere in my applications:
>
>    string.join([str(x) for x in iterable])
>
> Its tedious and makes code hard to read.  Do we need a PEP to fix
> this?

You won't need a PEP to replace it with the similar code-snippet from my code:

    string.join(map(str, iterable))

Same thing in the end, but map reads quite well (perhaps even better
than a listcomp) in applications as simple as this.
From tim.peters at gmail.com  Tue Aug 24 04:27:33 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 24 04:27:39 2004
Subject: [Python-Dev] problem with pymalloc on the BeOS port.
In-Reply-To: <2246153618-BeMail@taz>
References: <2246153618-BeMail@taz>
Message-ID: <1f7befae04082319274df5de61@mail.gmail.com>

[Fran?ois Revol]
> I'm updating the BeOS port of python to include the latest version in
> Zeta, the next incarnation of BeOS.
>
> I'm having some problem when enabling pymalloc:
> 
> case $MAKEFLAGS in \
> *-s*) LIBRARY_PATH=/boot/home/Python-2.3.4:%A/lib:/boot/home/config/
> lib:/boot/beos/system/lib CC='gcc' LDSHARED='./Modules/ld_so_beos
> libpython2.3.so' OPT='-O' ./python -E ./setup.py -q build;; \
> *) LIBRARY_PATH=/boot/home/Python-2.3.4:%A/lib:/boot/home/config/lib:/
> boot/beos/system/lib CC='gcc' LDSHARED='./Modules/ld_so_beos
> libpython2.3.so' OPT='-O' ./python -E ./setup.py build;; \
> esac
> running build
> running build_ext
> building 'struct' extension
> gcc -O -fno-strict-aliasing -I. -I/boot/home/Python-2.3.4/./Include -I/
> boot/home/config/include -I/boot/home/Python-2.3.4/Include -I/boot/home
> /Python-2.3.4 -c /boot/home/Python-2.3.4/Modules/structmodule.c -o
> build/temp.beos-5.1-BePC-2.3/structmodule.o
> Debug memory block at address p=0x80010fb8:
>    0 bytes originally requested
>    The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected.
>    The 4 pad bytes at tail=0x80010fb8 are not all FORBIDDENBYTE
> (0xfb):
>        at tail+0: 0xdb *** OUCH
>        at tail+1: 0xdb *** OUCH
>        at tail+2: 0xdb *** OUCH
>        at tail+3: 0xdb *** OUCH
>    The block was made by call #3688627195 to debug malloc/realloc.

No it wasn't.  The four bytes following the 4 trailing 0xfb hold the
call number, and they're apparently corrupt too.


> Fatal Python error: bad trailing pad byte
>
> indeed, there seem to be 4 deadblock marks between the forbidden ones,
> while the len is supposed to be null:

That's reliable.  If there actually was a request for 0 bytes (that
is, assuming this pointer isn't just pointing at a random memory
address), the debug pymalloc allocates 16 bytes for it, filled with

    00000000 fbfbfbfb fbfbfbfb serial

where "serial" is a big-endian 4-byte "serial number" uniquely
identifying which call to malloc (or realloc) this was.  The address
of the second block of fb's is returned.

> python:dm 0x80010fb8-8 32
> 80010fb0  00000000 fbfbfbfb  dbdbdbdb fbfbdbdb
> .................


> 80010fc0  0100fbfb 507686ef  04000000 fbfbfbfb
> .......vP........
> 80010fd0  8013cbc8 fbfbfbfb  44ee0100 ffed0100
> ............D....
> 
> Any clue ?

Try gcc without -O.  Nobody has reported anything like this before --
you're in for a lot of fun <wink>.
From greg at cosc.canterbury.ac.nz  Tue Aug 24 04:50:19 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug 24 04:50:24 2004
Subject: [Python-Dev] Re: Minimal 'stackless' PEP using generators?
In-Reply-To: <20040824015453.GA10615@prometheusresearch.com>
Message-ID: <200408240250.i7O2oJCI002327@cosc353.cosc.canterbury.ac.nz>

> it would be
> equivalent as if every "val = g2.next()" in g1 was rewritten:
> 
>     val = g2.next()
>     if isinstance(val, magic.Cooperate):
>         yield val

Something bothers me about that proposal, and I think it's this: it
would mean that generators are no longer simply a particular technique
for implementing iterators, but would have special powers that other
iterators don't have. Unless recognising the magic Cooperate value and
doing the right thing were made part of the iterator protocol, and all
iterators were expected to do the right thing with it, which would be
a big imposition.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From djc at object-craft.com.au  Tue Aug 24 04:53:06 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Tue Aug 24 04:53:10 2004
Subject: [Python-Dev] Submitted patch #1014930 to exposes the current parse
 location in pyexpat
Message-ID: <412AAD92.9010009@object-craft.com.au>

I would like someone to review the patch I just submitted - it exposes 
more the the expat API to the xml.parsers.expat.XMLParser object.

Expat supplies the XML_GetCurrentLineNumber, XML_GetCurrentColumnNumber, 
and XML_GetCurrentByteIndex functions to obtain the current parse 
location.  The patch exposes these as the following respective members 
of the xml.parsers.expat.XMLParser object; CurrentLineNumber, 
CurrentColumnNumber, CurrentByteIndex.

I have checkin permission, I just want someone to cast their eye over 
the patch before I commit.

- Dave

-- 
http://www.object-craft.com.au
From tim.peters at gmail.com  Tue Aug 24 05:02:34 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 24 05:02:41 2004
Subject: [Python-Dev] Missing peice of PEP 237 (int unification)
Message-ID: <1f7befae040823200212c31262@mail.gmail.com>

We're still generating OverflowWarnings in CVS:

>>> import warnings
>>> warnings.filterwarnings("error", "", OverflowWarning)
>>> import sys
>>> sys.maxint + 1
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
OverflowError: integer addition
>>>

The PEP says OverflowWarning was for "transition phase A":

     Here are the rules that guide warnings generated in situations
     that currently raise OverflowError.  This applies to transition
     phase A.

Phase A was done in 2.2.  Phase B0 was done in 2.3, and 2.4 is
supposed to be phase B1.  The OverflowWarnings are still there, but
should have gone away in 2.3 already.

It would be a lot of fun to rip out the bletcherous code raising
OverflowWarning, and finding the infinite loop that will then occur in
the test suite (you think I'm kidding -- heh).

One question the PEP doesn't answer:  does PyExc_OverflowWarning
vanish from the C API at the same time?  Does OverflowWarning vanish
from Python too?  Or do we need to add a PendingDeprecationWarning for
OverflowWarning in 2.4, a DeprecationWarning for OverflowWarning in
2.5, and finally nuke the silly bastard tin 2.6?
From guido at python.org  Tue Aug 24 06:02:01 2004
From: guido at python.org (Guido van Rossum)
Date: Tue Aug 24 06:02:15 2004
Subject: [Python-Dev] Missing piece of PEP 237 (int unification)
In-Reply-To: Your message of "Mon, 23 Aug 2004 23:02:34 EDT."
	<1f7befae040823200212c31262@mail.gmail.com> 
References: <1f7befae040823200212c31262@mail.gmail.com> 
Message-ID: <200408240402.i7O421b19106@guido.python.org>

> The PEP says OverflowWarning was for "transition phase A":
> 
>      Here are the rules that guide warnings generated in situations
>      that currently raise OverflowError.  This applies to transition
>      phase A.
> 
> Phase A was done in 2.2.  Phase B0 was done in 2.3, and 2.4 is
> supposed to be phase B1.  The OverflowWarnings are still there, but
> should have gone away in 2.3 already.
> 
> It would be a lot of fun to rip out the bletcherous code raising
> OverflowWarning, and finding the infinite loop that will then occur in
> the test suite (you think I'm kidding -- heh).

Sounds like a good idea.  I totally forgot about this part, because
it's so nearyly invisible. :-)

> One question the PEP doesn't answer:  does PyExc_OverflowWarning
> vanish from the C API at the same time?  Does OverflowWarning vanish
> from Python too?  Or do we need to add a PendingDeprecationWarning for
> OverflowWarning in 2.4, a DeprecationWarning for OverflowWarning in
> 2.5, and finally nuke the silly bastard tin 2.6?

I would keep the APIs for one release and then nuke them, but not add
any deprecation code.  Just document them as on the way out.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From bac at OCF.Berkeley.EDU  Tue Aug 24 06:56:13 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Aug 24 06:56:21 2004
Subject: [Python-Dev] Re: Minimal 'stackless' PEP using generators?
In-Reply-To: <200408240250.i7O2oJCI002327@cosc353.cosc.canterbury.ac.nz>
References: <200408240250.i7O2oJCI002327@cosc353.cosc.canterbury.ac.nz>
Message-ID: <412ACA6D.1090908@ocf.berkeley.edu>

Greg Ewing wrote:

>>it would be
>>equivalent as if every "val = g2.next()" in g1 was rewritten:
>>
>>    val = g2.next()
>>    if isinstance(val, magic.Cooperate):
>>        yield val
> 
> 
> Something bothers me about that proposal, and I think it's this: it
> would mean that generators are no longer simply a particular technique
> for implementing iterators, but would have special powers that other
> iterators don't have. Unless recognising the magic Cooperate value and
> doing the right thing were made part of the iterator protocol, and all
> iterators were expected to do the right thing with it, which would be
> a big imposition.
> 

I agree with Greg and the other comments on wanting to keep generators 
simple.  Part of the beauty of generators is the are brain-dead simple 
to use while still being handy as hell.  Playing with them to add the 
ability to play asynchronous stuff doesn't seem right.

I would rather see full-blown coroutines for Python 3000 to deal with 
this somehow.

-Brett
From martin at v.loewis.de  Tue Aug 24 07:13:29 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 24 07:13:34 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412A5693.4050008@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>
	<4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de>
	<4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de>
Message-ID: <412ACE79.4060605@v.loewis.de>

Walter D?rwald wrote:
> OK, let's come up with a patch that fixes the incomplete byte
> sequences problem and then discuss non-stream APIs.
> 
> So, what should the next step be?

I think your first patch should be taken as a basis for that.
Add the state-supporting decoding C functions, and change
the stream readers to use them. That still leaves the issue
of the last read operation, which I'm tempted to leave unresolved
for Python 2.4. No matter what the solution is, it would likely
require changes to all codecs, which is not good.

Regards,
Martin
From martin at v.loewis.de  Tue Aug 24 07:50:16 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 24 07:50:17 2004
Subject: [Python-Dev] problem with pymalloc on the BeOS port.
In-Reply-To: <1f7befae04082319274df5de61@mail.gmail.com>
References: <2246153618-BeMail@taz> <1f7befae04082319274df5de61@mail.gmail.com>
Message-ID: <412AD718.7020602@v.loewis.de>

Tim Peters wrote:

>>Any clue ?
> 
> 
> Try gcc without -O.  Nobody has reported anything like this before --
> you're in for a lot of fun <wink>.

In addition, try to find out whether BeOS' malloc is somehow "strange".
Does the system have the notion of a linear address space? Does it do
pages? and so on. If you find a feature that you wouldn't expect in
Windows NT, Linux, or HP-UX (*), that might give a clue.

Regards,
Martin

(*) I did not expect the Spanish Inquisition.
From greg at cosc.canterbury.ac.nz  Tue Aug 24 07:57:18 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Aug 24 07:57:30 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com>
Message-ID: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz>

> My comment about "coroutines" was more that Guido previously
> expressed distaste for adding such a communication mechanism to
> generators as abusing the concept of a generator just being a way to
> implement complex iterators.

This makes me think that maybe we want another kind of object, similar
to a generator, but designed to be used for side effects rather than
to produce values.

For want of a better term, let's call it a "cooperator" (so it ends in
"ator" like "generator" :-). Actually there will be two objects, a
cooperator-function (analogous to a generator-function) and a
cooperator-instance (analogous to a generator-iterator).

Calling a cooperator-function will return a cooperator-instance, which
will obey the cooperator protocol. This protocol consists of a single
function run(), which causes the cooperator-instance to perform some
amount of work and then stop. If there is more work remaining to be
done, run() returns a true value, otherwise it returns a false value.

There will be a new statement:

    suspend

for use inside a cooperator-function. The presence of this statement
is what distinguishes a cooperator-function from an ordinary function
(just as the presence of 'yield' distinguishes a generator from an
ordinary function). Execution of 'suspend' will cause the run() method
to return with a true value. The next time run() is called, execution
will resume after the 'suspend'.

This is really all that's needed, but one further piece of syntax may
be desirable to make it easier for one cooperator to invoke another:

    do another_coop()

would be equivalent to

    co = another_coop()
    while co.run():
        suspend

(Because it implies a 'suspend', the presence of 'do' would also mark
a function as a cooperator-function).

Something to note is that there's no requirement for a cooperator-
instance to be implemented by a cooperator-function, just as an
iterator can be implemented in ways other than a generator.  This
sidesteps some of the difficulties of mixing Python and C calls, since
a Python cooperator-function could invoke a cooperator implemented in
C which in turn invokes another Python cooperator- function, etc. The
only requirement is that all the objects on the way down obey the
cooperator protocol.

The drawback is that you always have to know when you're calling
another cooperator and use 'do'. But that's no worse than the current
situation with generators, where you always have to know you're using
an iterator and use 'for... yield'.

What think ye all?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From martin at v.loewis.de  Tue Aug 24 07:57:45 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 24 07:57:46 2004
Subject: [Python-Dev] Python icons
In-Reply-To: <412A0DA7.9070306@citlink.net>
References: <412A0DA7.9070306@citlink.net>
Message-ID: <412AD8D9.3050509@v.loewis.de>

Kevin Gadd wrote:
> I was bored the other day so I played around with an image editor and 
> redid most of the windows python icons.

Here is my man! (sorry if this is not the proper English phrase)

I have finally moved the Windows installer generator into the Python
source tree proper (Tools/msi). In the process, I had to delete two
icons taken from Microsoft sample code, for licensing issues, and
had to replace them with plain text, because I have virtually no
graphic skills.

They are both 16x16 icons, one indicating "go up one directory",
and the other "create a new folder". The original Microsoft bitmaps
are at

http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/nondist/sandbox/msi/Attic/Up.bin?rev=1.1.1.1

http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/nondist/sandbox/msi/Attic/New.bin?rev=1.1.1.1

Can you help creating a pair of such icons which are similar
in appearance but not quite the same?

Regards,
Martin
From martin at v.loewis.de  Tue Aug 24 08:00:49 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 24 08:00:50 2004
Subject: [Python-Dev] Python icons
In-Reply-To: <412A0DA7.9070306@citlink.net>
References: <412A0DA7.9070306@citlink.net>
Message-ID: <412AD991.3080404@v.loewis.de>

Kevin Gadd wrote:
> I did the basic one in vector. Let me know what you guys think, I figure 
> since they came out pretty decent that somebody might be able to make 
> something of 'em!

Responding to your actual question: They look nice to me, indeed;
assuming there are no down-sides, I'd be +1 for including them.
Of course, as apparently always the case with artwork, we would
need to know the licensing conditions that you want to associate
with using them.

Regards,
Martin
From fdrake at acm.org  Tue Aug 24 09:07:38 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Aug 24 09:07:50 2004
Subject: [Python-Dev] Submitted patch #1014930 to exposes the current
	parse location in pyexpat
In-Reply-To: <412AAD92.9010009@object-craft.com.au>
References: <412AAD92.9010009@object-craft.com.au>
Message-ID: <200408240307.38486.fdrake@acm.org>

On Monday 23 August 2004 10:53 pm, Dave Cole wrote:
 > Expat supplies the XML_GetCurrentLineNumber, XML_GetCurrentColumnNumber,
 > and XML_GetCurrentByteIndex functions to obtain the current parse
 > location.  The patch exposes these as the following respective members
 > of the xml.parsers.expat.XMLParser object; CurrentLineNumber,
 > CurrentColumnNumber, CurrentByteIndex.

This sounds like a good idea to me.  Please post the patch on SF and assign to 
me.

Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From mal at egenix.com  Tue Aug 24 10:39:32 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue Aug 24 10:39:36 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <1093304992.412a82a0a5a88@mail.iinet.net.au>
References: <002401c48931$dda69060$e841fea9@oemcomputer>
	<1093304992.412a82a0a5a88@mail.iinet.net.au>
Message-ID: <412AFEC4.6020007@egenix.com>

ncoghlan@iinet.net.au wrote:
> Quoting Raymond Hettinger <python@rcn.com>:
> 
> 
>>>This code-snippet is littered everwhere in my applications:
>>>
>>>    string.join([str(x) for x in iterable])
>>>
>>>Its tedious and makes code hard to read.  Do we need a PEP to fix
>>
>>this?
>>
>>A PEP would be overkill.
>>
>>Still, it would be helpful to do PEP-like things such as reference
>>implementation, soliticing comments, keep an issue list, etc.
>>
>>A minor issue is that the implementation automatically shifts to Unicode
>>upon encountering a Unicode string.  So you would need to test for this
>>before coercing to a string.
> 
> Perhaps have string join coerce to string, and Unicode join coerce to the
> separator's encoding. If we do that, the existing string->Unicode promotion code
> should handle the switch between the two join types.

The general approach is always to coerce to Unicode if strings
and Unicode meet; very much like coercion to floats is done when
integers and floats meet.

Your suggestion would break this logic and make coercion depend
on an argument.

>>Also, join works in multiple passes.  The proposal should be specific
>>about where stringizing occurs.  IIRC, you need the object length on the
>>first pass, but the error handling and switchover to Unicode occur on
>>the second.
> 
> 
> Having been digging in the guts of string join last week, I'm pretty sure the
> handover to the Unicode join happens on the first 'how much space do we need'
> pass (essentially, all of the work done so far is thrown away, and the Unicode
> join starts from scratch. If you know you have Unicode, you're better off using
> a Unicode separator to avoid this unnecessary work </tangent>).

It's just a simple length querying loop; there's no storage allocation
or anything expensive happening there, so the "throw-away" operation
is not expensive. Aside: ''.join() currently only works for true
sequences - not iterators.

OTOH, the %-format operation is which is why PyString_Format goes
through some extra hoops to make sure the work already is not
dropped (indeed, it may not even be possible to reevaluate the
arguments; think iterators here).

We could probably add similar logic to ''.join() to have
it also support iterators.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 24 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Tue Aug 24 11:36:45 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue Aug 24 11:36:47 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412ACE79.4060605@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>
	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>
	<412A5693.4050008@livinglogic.de> <412ACE79.4060605@v.loewis.de>
Message-ID: <412B0C2D.7030507@egenix.com>

Martin v. L?wis wrote:
> Walter D?rwald wrote:
> 
>> OK, let's come up with a patch that fixes the incomplete byte
>> sequences problem and then discuss non-stream APIs.
>>
>> So, what should the next step be?
> 
> I think your first patch should be taken as a basis for that.

We do need a way to communicate state between the codec
and Python.

However, I don't like the way that the patch
implements this state handling: I think we should use a
generic "state" object here which is passed to the stateful
codec and returned together with the standard return values
on output:

def decode_stateful(data, state=None):
     ... decode and modify state ...
     return (decoded_data, length_consumed, state)

where the object type and contents of the state variable
is defined per codec (e.g. could be a tuple, just a single
integer or some other special object).

Otherwise we'll end up having different interface
signatures for all codecs and extending them to accomodate
for future enhancement will become unfeasable without
introducing yet another set of APIs.

Let's discuss this some more and implement it for Python 2.5.
For Python 2.4, I think we can get away with what we already
have:

If we leave out the UTF-7 codec changes in the
patch, the only state that the UTF-8 and UTF-16
codecs create is the number of bytes consumed. We already
have the required state parameter for this in the
standard decode API, so no extra APIs are needed for
these two codecs.

So the patch boils down to adding a few new C APIs
and using the consumed parameter in the standard
_codecs module APIs instead of just defaulting to the
input size (we don't need any new APIs in _codecs).

> Add the state-supporting decoding C functions, and change
> the stream readers to use them.

The buffer logic should only be used for streams
that do not support the interface to push back already
read bytes (e.g. .unread()).

 From a design perspective, keeping read data inside the
codec is the wrong thing to do, simply because it leaves
the input stream in an undefined state in case of an error
and there's no way to correlate the stream's read position
to the location of the error.

With a pushback method on the stream, all the stream
data will be stored on the stream, not the codec, so
the above would no longer be a problem.

However, we can always add the .unread() support to the
stream codecs at a later stage, so it's probably ok
to default to the buffer logic for Python 2.4.

> That still leaves the issue
> of the last read operation, which I'm tempted to leave unresolved
> for Python 2.4. No matter what the solution is, it would likely
> require changes to all codecs, which is not good.

We could have a method on the codec which checks whether
the codec buffer or the stream still has pending data
left. Using this method is an application scope consideration,
not a codec issue.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 24 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From walter at livinglogic.de  Tue Aug 24 11:38:12 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Aug 24 11:38:16 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test
 test_string.py, 1.25, 1.26
In-Reply-To: <E1BzOAO-0005Hx-Qc@sc8-pr-cvs1.sourceforge.net>
References: <E1BzOAO-0005Hx-Qc@sc8-pr-cvs1.sourceforge.net>
Message-ID: <412B0C84.30204@livinglogic.de>

rhettinger@users.sourceforge.net wrote:

> Update of /cvsroot/python/python/dist/src/Lib/test
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv20193/Lib/test
> 
> Modified Files:
> 	test_string.py 
> Log Message:
> SF Patch #1007087:  Return new string for single subclass joins (Bug #1001011)
> (Patch contributed by Nick Coghlan.)
> 
> Now joining string subtypes will always return a string.
> Formerly, if there were only one item, it was returned unchanged.

test_string.py is supposed to test the string module. This new test
should be moved to test_str.py (or even better to string_test.py,
so that it can be reused for doing the same test with unicode objects).

Bye,
    Walter D?rwald


From jtk at yahoo.com  Tue Aug 24 13:06:06 2004
From: jtk at yahoo.com (Jeff Kowalczyk)
Date: Tue Aug 24 13:03:52 2004
Subject: [Python-Dev] Re: Python icons
References: <412A0DA7.9070306@citlink.net>
Message-ID: <pan.2004.08.24.11.06.04.789962@yahoo.com>

Kevin Gadd wrote:
> I was bored the other day so I played around with an image editor and
> redid most of the windows python icons... Let me know what you guys
> think.

Very nice to have a quality vector rendition. May I suggest a slight tweak
to +smirk/grin and +top eyelid? He looks just a bit nervous in
comparison to the original.

http://www.python.org/wiki/python/img/odi.jpg
http://luminance.org/py/screenshot.png

From dbickett at gmail.com  Tue Aug 24 13:30:50 2004
From: dbickett at gmail.com (Daniel Bickett)
Date: Tue Aug 24 13:31:03 2004
Subject: [Python-Dev] Re: Python icons
In-Reply-To: <pan.2004.08.24.11.06.04.789962@yahoo.com>
References: <412A0DA7.9070306@citlink.net>
	<pan.2004.08.24.11.06.04.789962@yahoo.com>
Message-ID: <1d6cdae304082404305fb5b553@mail.gmail.com>

+1, those really look great :) Nice job.

BTW, my name's Daniel Bickett, I've been lurking for a week or two
with the intention of learning more. Finally something that isn't too
far over my head to respond to!
From pje at telecommunity.com  Tue Aug 24 13:34:10 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug 24 13:33:55 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz>
References: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040824071549.02d407c0@mail.telecommunity.com>

At 05:57 PM 8/24/04 +1200, Greg Ewing wrote:
>What think ye all?

I don't think it's in the right direction to achieve the effects desired by 
Clark or me.  Here's a metaphor for what would be ideal: resumable 
exceptions.  If you could throw a special kind of exception (or even a 
regular exception), and call traceback.resume() to pick up execution at the 
point where it was thrown, whether thrown by a generator or a regular 
function.  If you had that capability, you could implement any sort of 
coroutine or task-switching facilities you wanted.

Without such a facility, one needs a stack of *ators to emulate it, to 
provide the effects desired by Clark's Flow or by peak.events.  However, 
co-operators' "suspend" doesn't provide a way to communicate between 
co-operators.  At least 'yield' lets you yield a value.  What's really 
needed (IMO) is to add a way to communicate *into* a co-operator, passing 
it a value to "accept" or a traceback to raise.  E.g.:

     input = suspend output

Where 'output' is returned from the current 'run()', and 'input' is a value 
passed to the next 'run()'.  Or, if there's a type/value/traceback passed 
to a 'throw()' method, then the 'suspend' statement should raise an error.

With that facility, 'peak.events' could drop the 'events.resume()' magic 
function.  It'd still need a stack of co-operators, and there'd still be 
ugliness when iterating over a generator that needed to be 
suspendable.  But at least it would be able to have clean syntax.  (Though 
I don't think that 'input=suspend output' is actually clean syntax, it's 
just better than the yield/resume thing.)

Anyway, as I said, what would be *most* useful for async programming is a 
way to resume a traceback, because then you wouldn't need for every 
intervening frame to have special suspension capability.


From janusfury at citlink.net  Tue Aug 24 12:54:14 2004
From: janusfury at citlink.net (Kevin Gadd)
Date: Tue Aug 24 13:36:58 2004
Subject: [Python-Dev] Python icons
In-Reply-To: <412AD991.3080404@v.loewis.de>
References: <412A0DA7.9070306@citlink.net> <412AD991.3080404@v.loewis.de>
Message-ID: <412B1E56.1060502@citlink.net>

I'm not really worried about licensing... basically, I'd probably just 
want to keep my copyright on the actual vector art. I'm not terribly 
concerned about anything else, since they are based on the original 
python icons. I don't know much about how licensing artwork for open 
source projects works, so if you or someone else can let me know what 
steps would be necessary to make it possible for these icons to be used, 
I'd be glad to help.

In response to your previous post, as well, I will try and put together 
replacements for those two icons.

Martin v. L?wis wrote:

> Kevin Gadd wrote:
>
>> I did the basic one in vector. Let me know what you guys think, I 
>> figure since they came out pretty decent that somebody might be able 
>> to make something of 'em!
>
>
> Responding to your actual question: They look nice to me, indeed;
> assuming there are no down-sides, I'd be +1 for including them.
> Of course, as apparently always the case with artwork, we would
> need to know the licensing conditions that you want to associate
> with using them.
>
> Regards,
> Martin
>
>

-- 
Kevin Gadd
E-Mail <mailto:kevin@luminance.org> | Website <http://www.luminance.org/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040824/b2a7b731/attachment.htm
From ajm at flonidan.dk  Tue Aug 24 13:42:06 2004
From: ajm at flonidan.dk (Anders J. Munch)
Date: Tue Aug 24 13:42:10 2004
Subject: [Python-Dev] Re: Update PEP 292
Message-ID: <6D9E824FA10BD411BE95000629EE2EC3C6DE1D@FLONIDAN-MAIL>

>From Brett C. [mailto:bac@OCF.Berkeley.EDU]
> 
> That is coming down to a question of convenience compared to 
> explicitness.  Do we want "Give me $1,000,000 now!" to raise 
> ValueError, 
> or do we want to let it pass through?

This particular problem has been considered and decided on years ago,
in the context of %-interpolation:

    >>> "Give me %1,000,000 now!" % {}
    ValueError: unsupported format character ',' (0x2c) at index 10

String interpolation with named parameters given as a dictionary is a
problem already solved in Python.  I am puzzled at this urge to
re-solve the same problem, only differently.

If remembering to type the trailing s in "%(varname)s" is so much of a
problem, why not extend the current interpolation syntax with a
variation that is more convenient, yet still backwards-compatible?  I
would suggest a formatting-character-less version of "%(varname)s"
that uses brackets instead: "%[varname]".

dollar-signs?-we-don't-need-no-steenkin-dollar-signs-ly y'rs, Anders
From dbickett at gmail.com  Tue Aug 24 13:45:13 2004
From: dbickett at gmail.com (Daniel Bickett)
Date: Tue Aug 24 13:45:21 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz>
References: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz>
Message-ID: <1d6cdae304082404452c15617b@mail.gmail.com>

On Tue, 24 Aug 2004 17:57:18 +1200, Greg Ewing
<greg@cosc.canterbury.ac.nz> wrote:
> > My comment about "coroutines" was more that Guido previously
> > expressed distaste for adding such a communication mechanism to
> > generators as abusing the concept of a generator just being a way to
> > implement complex iterators.
> 
> This makes me think that maybe we want another kind of object, similar
> to a generator, but designed to be used for side effects rather than
> to produce values.
> 
> For want of a better term, let's call it a "cooperator" (so it ends in
> "ator" like "generator" :-). Actually there will be two objects, a
> cooperator-function (analogous to a generator-function) and a
> cooperator-instance (analogous to a generator-iterator).
> 
> Calling a cooperator-function will return a cooperator-instance, which
> will obey the cooperator protocol. This protocol consists of a single
> function run(), which causes the cooperator-instance to perform some
> amount of work and then stop. If there is more work remaining to be
> done, run() returns a true value, otherwise it returns a false value.

Tell me if I'm missing something here, but with the exception of your
two introductory keywords (a subject I thought was supposed to be
sacred, with the intention of keeping the amount of keywords as small
as possible,) how much difference is there to generators? Are you
saying the run() method, after having been called initially, will
return a value indicative of the cooperator's business when called
again?

It seems like a similar feat could be achieved with custom generators
and boolean yield statements.

Daniel Bickett
From anthony at interlink.com.au  Tue Aug 24 14:19:31 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug 24 14:19:58 2004
Subject: [Python-Dev] SF Priorities, 2.4 release
In-Reply-To: <1f7befae040819094825bfacf0@mail.gmail.com>
References: <4124CC59.80007@interlink.com.au>
	<1f7befae040819094825bfacf0@mail.gmail.com>
Message-ID: <412B3253.9080106@interlink.com.au>

Working originally from Tim's list, I propose to start using the
priorities in the SF tracker to try and manage the outstanding work
for 2.4. Here's the proposed list:

Priority 9:
   - MUST be done before the next release (whether it's a3, b1, b2,
     or whatever)
Priority 8:
   - MUST be done before b1. It's a functionality change, and it's
     needed for 2.4. If it's not done before b1, it will have to wait
     until 2.5, whenever that might be.
Priority 7:
   - SHOULD be done before 2.4 final.

I'm going to go through and start updating various open bugs and
patches with priorities.

How can you help? I'd _really_ prefer that people not run around
setting priorities themselves - instead, if you could let me know
of bugs/patches that you think should be changed, let me know,
either via email, or via IRC - #python-dev on irc.freenode.net.

Once this is done, I plan to start actively nagging relevant people
for feedback on patches/bugs that only they can provide. I'm happy
to do the work of actually checking the patches in, if you're too
busy, but in many cases, I'm not going to be able to provide the
level of local knowledge about a particular patch or bug report.

I'll also be updating the 2.4 release PEP, and I'll also create a
2.5 release PEP, and move stuff that misses 2.4 to it.

Anyway, I'm open to feedback on this. I plan to start frobbing
priorities shortly.

Anthony
From FBatista at uniFON.com.ar  Tue Aug 24 15:27:21 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Aug 24 15:31:42 2004
Subject: [Python-Dev] SF Priorities, 2.4 release
Message-ID: <A128D751272CD411BC9200508BC2194D053C783B@escpl.tcp.com.ar>

[Anthony Baxter]

#- Priority 9:
#-    - MUST be done before the next release (whether it's a3, b1, b2,
#-      or whatever)
#- Priority 8:
#-    - MUST be done before b1. It's a functionality change, and it's
#-      needed for 2.4. If it's not done before b1, it will have to wait
#-      until 2.5, whenever that might be.
#- Priority 7:
#-    - SHOULD be done before 2.4 final.

+1

.	Facundo
From Paul.Moore at atosorigin.com  Tue Aug 24 15:58:42 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Tue Aug 24 15:58:44 2004
Subject: [Python-Dev] Simple coroutines?
Message-ID: <16E1010E4581B049ABC51D4975CEDB88052CB3BE@UKDCX001.uk.int.atosorigin.com>

From: Greg Ewing
> There will be a new statement:
>
>     suspend
>
[...]
>
>    do another_coop()
>
> would be equivalent to
>
>    co = another_coop()
>    while co.run():
>        suspend


I'm not sure I see the difference between suspend/do and
yield True/for _ in co: pass, other than possibly that
co-operators and generators are intended to be mixed (which
strikes me as implausible).

I'm likely to be missing something here, but I don't follow
the semantics you are suggesting.

If there was a simple, realistic use case for this, it might
help clarify the semantics. (For extra credit, mix co-operators
and generators in your use case so that the semantics of
interaction are shown as well :-))

Paul.


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________
From kbk at shore.net  Tue Aug 24 16:09:33 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue Aug 24 16:09:35 2004
Subject: [Python-Dev] SF Priorities, 2.4 release
In-Reply-To: <412B3253.9080106@interlink.com.au> (Anthony Baxter's message
	of "Tue, 24 Aug 2004 22:19:31 +1000")
References: <4124CC59.80007@interlink.com.au>
	<1f7befae040819094825bfacf0@mail.gmail.com>
	<412B3253.9080106@interlink.com.au>
Message-ID: <87fz6c4ogy.fsf@hydra.localdomain>

Anthony Baxter <anthony@interlink.com.au> writes:

> Working originally from Tim's list, I propose to start using the
> priorities in the SF tracker to try and manage the outstanding work
> for 2.4. Here's the proposed list:
>
> Priority 9:
>    - MUST be done before the next release (whether it's a3, b1, b2,
>      or whatever)
> Priority 8:
>    - MUST be done before b1. It's a functionality change, and it's
>      needed for 2.4. If it's not done before b1, it will have to wait
>      until 2.5, whenever that might be.
> Priority 7:
>    - SHOULD be done before 2.4 final.
>
> I'm going to go through and start updating various open bugs and
> patches with priorities.
>
> How can you help? I'd _really_ prefer that people not run around
> setting priorities themselves - instead, if you could let me know
> of bugs/patches that you think should be changed, let me know,
> either via email, or via IRC - #python-dev on irc.freenode.net.

OK, but how about a further tweak (I use priorities to manage IDLE):

Priorities 7 - 9 are reserved and modifiable only by the Release Manager.

Priorities 6 and below are not reserved and may be modified as desired
within the range 1 - 6.

-- 
KBK
From revol at free.fr  Tue Aug 24 14:37:47 2004
From: revol at free.fr (=?windows-1252?q?Fran=E7ois?= Revol)
Date: Tue Aug 24 16:13:32 2004
Subject: [Python-Dev] problem with pymalloc on the BeOS port.
In-Reply-To: <412AD718.7020602@v.loewis.de>
Message-ID: <2456435963-BeMail@taz>

> Tim Peters wrote:
> 
> >>Any clue ?
> > 
> > 
> > Try gcc without -O.  Nobody has reported anything like this before 
> > --
> > you're in for a lot of fun <wink>.
> 
> In addition, try to find out whether BeOS' malloc is somehow 
> "strange".
> 
It's plain GNU malloc (with some locking for multithreading I think)
libroot is actually a glibc +libm with some goodies.
Though the glibc is getting quite old.

> Does the system have the notion of a linear address space? Does it do
> pages? and so on. If you find a feature that you wouldn't expect in
> Windows NT, Linux, or HP-UX (*), that might give a clue.
Sure, BeOS is a modern OS, mind you :D And it has a fairly complete 
(complete enough) POSIX subsystem.
The only thing that I miss sometimes is mmap() but most of the time it 
can be worked around easily with areas.

Fran?ois.
From revol at free.fr  Tue Aug 24 15:25:28 2004
From: revol at free.fr (=?windows-1252?q?Fran=E7ois?= Revol)
Date: Tue Aug 24 16:13:38 2004
Subject: [Python-Dev] problem with pymalloc on the BeOS port.
Message-ID: <5317443691-BeMail@taz>

Tim Peters <tim.peters@gmail.com>:
> [Fran?ois Revol]
> > I'm updating the BeOS port of python to include the latest version 
> > in
> > Zeta, the next incarnation of BeOS.
> >
> > I'm having some problem when enabling pymalloc:

> > [zapped]

> >    0 bytes originally requested
> >    The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected.
> >    The 4 pad bytes at tail=0x80010fb8 are not all FORBIDDENBYTE
> > (0xfb):
> >        at tail+0: 0xdb *** OUCH
> >        at tail+1: 0xdb *** OUCH
> >        at tail+2: 0xdb *** OUCH
> >        at tail+3: 0xdb *** OUCH
> >    The block was made by call #3688627195 to debug malloc/realloc.
> 
> No it wasn't.  The four bytes following the 4 trailing 0xfb hold the
> call number, and they're apparently corrupt too.
Eh...

> 
> > Fatal Python error: bad trailing pad byte
> >
> > indeed, there seem to be 4 deadblock marks between the forbidden 
> > ones,
> > while the len is supposed to be null:
> 
> That's reliable.  If there actually was a request for 0 bytes (that
> is, assuming this pointer isn't just pointing at a random memory
> address), the debug pymalloc allocates 16 bytes for it, filled with
> 
>     00000000 fbfbfbfb fbfbfbfb serial
> 
> where "serial" is a big-endian 4-byte "serial number" uniquely
> identifying which call to malloc (or realloc) this was.  The address
> of the second block of fb's is returned.
Yes that's what I deduced from the code of pymalloc.

> 
> > python:dm 0x80010fb8-8 32
> > 80010fb0  00000000 fbfbfbfb  dbdbdbdb fbfbdbdb
> > .................
> 
> 
> > 80010fc0  0100fbfb 507686ef  04000000 fbfbfbfb
> > .......vP........
> > 80010fd0  8013cbc8 fbfbfbfb  44ee0100 ffed0100
> > ............D....
> > 
> > Any clue ?
> 
> Try gcc without -O.  Nobody has reported anything like this before --
> you're in for a lot of fun <wink>.
> 
OK, tried -O0 -g but same result.
I suspect it might be a bad interaction with fork(), as it crashes in a 
child, quite badly, as no images are repported as loaded in the team (= 
no binary are mapped in the process), however the areas are there (= 
pages).

Now, I don't see why malloc itself would give such a result, it's 
pyMalloc which places those marks, so the thing malloc does wouldn't 
place them 4 bytes of each other for no reason, or repport 0 bytes 
where 4 are allocated.

Fran?ois.
From mcherm at mcherm.com  Tue Aug 24 16:57:22 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Aug 24 16:53:54 2004
Subject: [Python-Dev] Python icons
Message-ID: <1093359442.412b5752c3558@mcherm.com>

Kevin Gadd writes:
> I'm not really worried about licensing... basically, I'd probably just want
> to keep my copyright on the actual vector art. I'm not terribly concerned
> about anything else, since they are based on the original python icons. I
> don't know much about how licensing artwork for open source projects works,
> so if you or someone else can let me know what steps would be necessary to
> make it possible for these icons to be used, I'd be glad to help.

While I can't speak authoritatively, I have an idea of what folks would
want. My guess is that the PSF (Python Software Foundation) would want
the right to use the icons any way they deem necessary for promoting
Python. I'm sure they wouldn't mind your keeping the copyright, just so
long as the PSF can authorize people to use them (eg: in the installer).
A simple note to the PSF stating that you own the copyright and that
you're willing to allow the use MIGHT be sufficient, unfortunately it
would take a real lawyer to say for sure what's sufficient. In the past
other Icons have come with a request by the artist that they be
credited... and I'm sure no one would begrudge you a mention in Python's
credits (see [1]).

And let me add my own thanks! It's nice to have good artwork -- I'm sure
there are a few other places you could help also.

-- Michael Chermside
   (speaking only for myself, of course)

[1]
http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Misc/ACKS?view=markup
From mwh at python.net  Tue Aug 24 16:57:25 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug 24 16:57:27 2004
Subject: [Python-Dev] Python icons
In-Reply-To: <1093359442.412b5752c3558@mcherm.com> (Michael Chermside's
	message of "Tue, 24 Aug 2004 07:57:22 -0700")
References: <1093359442.412b5752c3558@mcherm.com>
Message-ID: <2macwk7fe2.fsf@starship.python.net>

Michael Chermside <mcherm@mcherm.com> writes:

> Kevin Gadd writes:
>> I'm not really worried about licensing... basically, I'd probably just want
>> to keep my copyright on the actual vector art. I'm not terribly concerned
>> about anything else, since they are based on the original python icons. I
>> don't know much about how licensing artwork for open source projects works,
>> so if you or someone else can let me know what steps would be necessary to
>> make it possible for these icons to be used, I'd be glad to help.
>
> While I can't speak authoritatively, I have an idea of what folks would
> want. My guess is that the PSF (Python Software Foundation) would want
> the right to use the icons any way they deem necessary for promoting
> Python.

I think the Python Powered License is inspirational in this respect
(read the bottom of http://starship.python.net/~just/pythonpowered/).

Well, maybe not.  But it's always good for a chuckle :-)

Cheers,
mwh

-- 
  <exarkun> today's lesson
  <exarkun> don't strace X in an xterm
                                                -- from Twisted.Quotes
From theller at python.net  Tue Aug 24 17:07:09 2004
From: theller at python.net (Thomas Heller)
Date: Tue Aug 24 17:07:13 2004
Subject: [Python-Dev] SF Priorities, 2.4 release
In-Reply-To: <412B3253.9080106@interlink.com.au> (Anthony Baxter's message
	of "Tue, 24 Aug 2004 22:19:31 +1000")
References: <4124CC59.80007@interlink.com.au>
	<1f7befae040819094825bfacf0@mail.gmail.com>
	<412B3253.9080106@interlink.com.au>
Message-ID: <fz6cwp5u.fsf@python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

> Working originally from Tim's list, I propose to start using the
> priorities in the SF tracker to try and manage the outstanding work
> for 2.4. Here's the proposed list:
>
> Priority 9:
>    - MUST be done before the next release (whether it's a3, b1, b2,
>      or whatever)
> Priority 8:
>    - MUST be done before b1. It's a functionality change, and it's
>      needed for 2.4. If it's not done before b1, it will have to wait
>      until 2.5, whenever that might be.
> Priority 7:
>    - SHOULD be done before 2.4 final.

What about bugs that MUST be fixed before 2.4 final, when I don't care
about the exact release they are fixed?  http://python.org/sf/1014215
is such an example, imo.

Thomas

From vinay_sajip at yahoo.co.uk  Tue Aug 24 15:57:18 2004
From: vinay_sajip at yahoo.co.uk (=?iso-8859-1?q?Vinay=20Sajip?=)
Date: Tue Aug 24 18:06:40 2004
Subject: [Python-Dev] Proposed change to logging
Message-ID: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com>

An improvement to logging has been proposed by Miki
Tebeka, which involves the creation of file-like
wrapper objects for loggers. This makes it possible to
use the print idiom for logging:
 
import logging
 
logging.basicConfig()
logger = logging.getLogger("my.app")
err = logger.getFileObject(logging.ERROR)
dbg = logger.getFileObject(logging.DEBUG)
...
 
print >> err, "My logging message with %s" %
"arguments"
print >> dbg, "A message at %s level" % "debug"
 
Miki has helpfully sent in a patch (#1001864), and I
propose to check it in by the 27th of August. I just
wanted to get comments from python-dev first. There's
one wart in the implementation - to avoid printing out
blank lines caused by the "\n" appended by print, we
ignore messages which are == "\n". This works only
because print does a separate write for the "\n". Can
anyone see a problem with this approach?
 
class LoggerFileObject:
    """
    Simulate a file object for a given logging level.
    """
    def __init__(self, logger, level):
        """
        Initialize file object with logger and level
        """
        self.logger = logger
        self.level = level
        self.closed = 0
 
    def write(self, msg):
        """
        Write message to log.
        """
        if self.closed:
            raise ValueError("I/O operation on closed
file")
        # Avoid printing blank lines.
        if msg != "\n":
            self.logger.log(self.level, msg)
 
    def flush(self):
        """
        Flush the file object. This flushes the
logger's handlers.
        """
        if self.closed:
            return
        for handler in self.logger.handlers:
            handler.flush()
 
    def close(self):
        """
        Close file object.
 
        Calling "write" after "close" will raise a
ValueError.
        """
        self.closed = 1
 
Regards,
 
 
Vinay



	
	
		
___________________________________________________________ALL-NEW Yahoo! Messenger - all new features - even more fun!  http://uk.messenger.yahoo.com
From python at rcn.com  Tue Aug 24 18:25:06 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Aug 24 18:25:35 2004
Subject: [Python-Dev] SF Priorities, 2.4 release
In-Reply-To: <412B3253.9080106@interlink.com.au>
Message-ID: <003901c489f6$eb012f20$04f9cc97@oemcomputer>

> I'd _really_ prefer that people not run around
> setting priorities themselves - instead, if you could let me know
> of bugs/patches that you think should be changed, let me know,
> either via email, or via IRC - #python-dev on irc.freenode.net.

Why are other developers suddenly no longer qualified to raise or lower
the priority of a bug they've looked at?


Raymond

From anthony at interlink.com.au  Tue Aug 24 18:29:31 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug 24 18:29:40 2004
Subject: [Python-Dev] SF Priorities, 2.4 release
In-Reply-To: <003901c489f6$eb012f20$04f9cc97@oemcomputer>
References: <003901c489f6$eb012f20$04f9cc97@oemcomputer>
Message-ID: <412B6CEB.30304@interlink.com.au>

Raymond Hettinger wrote:
> Why are other developers suddenly no longer qualified to raise or lower
> the priority of a bug they've looked at?

That's not the issue - the issue is that I want to use the top three 
priorities for tracking release management critical bugs. Perhaps we
should then flag the other priorities for regular use.

The SF tracker is too damn limited, but I'm trying to figure out the
best approach to this.

Anthony
From tim.peters at gmail.com  Tue Aug 24 18:31:08 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 24 18:31:21 2004
Subject: [Python-Dev] SF Priorities, 2.4 release
In-Reply-To: <412B3253.9080106@interlink.com.au>
References: <4124CC59.80007@interlink.com.au>
	<1f7befae040819094825bfacf0@mail.gmail.com>
	<412B3253.9080106@interlink.com.au>
Message-ID: <1f7befae0408240931353fcda2@mail.gmail.com>

BTW, priority 6 is a good way to identify issues that ought to be
assigned a 7, 8 or 9.  For example, I set the "asyncore breaks Zope"
bug to 6.  6 is one above the default, so is the least intrusive way
to say "I'm pretty sure this needs to be addressed one way or another
'soon'".
From tim.peters at gmail.com  Tue Aug 24 18:43:11 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Aug 24 18:43:17 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <412AFEC4.6020007@egenix.com>
References: <002401c48931$dda69060$e841fea9@oemcomputer>
	<1093304992.412a82a0a5a88@mail.iinet.net.au>
	<412AFEC4.6020007@egenix.com>
Message-ID: <1f7befae04082409436abe3ded@mail.gmail.com>

[M.-A. Lemburg]
> ...
> Aside: ''.join() currently only works for true sequences - not iterators.

>>> def gen():
...     for s in "sure", "it", "does":
...         yield s
...
>>> ' '.join(gen())
'sure it does'
>>> u' '.join(gen())
u'sure it does'
>>>

Every function implemented with PySequence_Fast() works with any
iterable, although it's fastest if the input argument is a builtin
list or tuple.  For anything else (including list or tuple subclasses,
and other "true sequences"), it materializes a temp tuple, via the
iterator protocol.
From barry at python.org  Tue Aug 24 18:49:54 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 24 18:49:28 2004
Subject: [Python-Dev] Proposed change to logging
In-Reply-To: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com>
References: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com>
Message-ID: <1093366081.18857.42.camel@geddy.wooz.org>

On Tue, 2004-08-24 at 09:57, Vinay Sajip wrote:
> An improvement to logging has been proposed by Miki
> Tebeka, which involves the creation of file-like
> wrapper objects for loggers. 

Yay! +1, especially if I can use them interchangeably.  In my code, I
usually like the log.debug() calls, but once in a while I have output
that comes from something expecting a file-like API, and in that case,
it would be nice to be able to pass in log.getFileObject().  Currently,
I'm StringIO'ing it.

> There's
> one wart in the implementation - to avoid printing out
> blank lines caused by the "\n" appended by print, we
> ignore messages which are == "\n". This works only
> because print does a separate write for the "\n". Can
> anyone see a problem with this approach?

What if the .log() method on loggers took an extra flag to suppress the
newline or not, and the wrapper's .write() method passed the appropriate
value?  The default would be to include the extra newline so the .log()
interface would be backward compatible.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/e08fbcaa/attachment-0001.pgp
From barry at python.org  Tue Aug 24 19:00:37 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 24 19:00:23 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <1f7befae040823093264275843@mail.gmail.com>
References: <1093276486.412a1346e30e9@mcherm.com>
	<20040823161233.GB71531@prometheusresearch.com>
	<1f7befae040823093264275843@mail.gmail.com>
Message-ID: <1093366837.18858.51.camel@geddy.wooz.org>

On Mon, 2004-08-23 at 12:32, Tim Peters wrote:

> Apologies to Barry, but I'm +1 on auto-str() too.  It's a string
> interpolation -- the user is explicitly asking for a string.  If they
> made a mistake, it was in asking for a string to begin with, not in
> feeding it a non-string.

Should it be auto-unicode(), given that Template is derived from
unicode?  And if so, should we entertain the possibility of insanities
like giving the user the ability to pass optional arguments to the
unicode() call?  If the answers to that are yes and no, that's fine with
me.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/e2907160/attachment.pgp
From barry at python.org  Tue Aug 24 19:05:31 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 24 19:05:18 2004
Subject: [Python-Dev] Update PEP 292
In-Reply-To: <001201c48946$19b49e80$e841fea9@oemcomputer>
References: <001201c48946$19b49e80$e841fea9@oemcomputer>
Message-ID: <1093367131.18858.54.camel@geddy.wooz.org>

On Mon, 2004-08-23 at 15:19, Raymond Hettinger wrote:

> Does it?  Apply the patch and try "import re".  

% cd Lib
% find -name \*.pyc
% cd ..
% ./python
Python 2.4a2 (#5, Aug 22 2004, 19:41:48)
[GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
[25352 refs]
>>> import string
[25356 refs]
>>> string.Template
<class 'string.Template'>
[25356 refs]
>>>

isn't-anybody-else-going-to-try-the-patch-or-should-i-just-check-it-in-ly
y'rs, -Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/3744a324/attachment.pgp
From barry at python.org  Tue Aug 24 19:14:07 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 24 19:13:41 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <20040823132043.GA4501@titan.progiciels-bpi.ca>
References: <1093232832.28573.16.camel@geddy.wooz.org>
	<20040823132043.GA4501@titan.progiciels-bpi.ca>
Message-ID: <1093367647.18858.62.camel@geddy.wooz.org>

On Mon, 2004-08-23 at 09:20, Fran?ois Pinard wrote:

> These two quotes let me think that people will often forget to stringify
> numbers and various other value types, or to escape a few isolated `$'.

I'm convinced that we should auto-convert the values.

> I have the fuzzy feeling that one unstated, but important goal of the
> PEP is to find some way to push `$' forward for stringification in
> Python.  The whole PEP might even be a stunt or justification for this
> unstated goal.  However, as this goal is well reached, it might somehow
> have contributed to the PEP acceptation and counterweight the deficiency
> stated in the preceding paragraph.  If my feeling is right, then the PEP
> should clearly explicit this goal, it will make the PEP stronger.

I will neither confirm nor deny whether the PSU is bankrolling the PEP
292 initiative, nor the actual existence of any 527 organization
claiming to be called the "PSU", nor whether if they did exist, they
were or weren't acting in coordination with the campaign organizations
of any 2004 US presidential nominee.

> I witnessed a serious effort in this developers' group to "empty" the
> `string' module in favour of string methods.  That effort has been
> abandoned?  I also personally think the word `string' should be left
> free, in the long run, for Python programmers to use, on the ground that
> modules should not be named after common words programmers would like to
> keep to themselves as variable names.  So, I think _nothing_ should be
> added to the `string' module, with the effect of nailing it deeper in
> the library, even if there is a need to rush some solution in.

My own personal opinion is that function that were traditionally
available in the string module, but which have been available as string
methods for a long time now, should be deprecated and eventually
removed.  But the ground for a string module is staked out, and to the
extent that useful things like Template need a home, I'd rather see them
end up in the existing string module (which everyone already deals with)
than to contrive some name like 'stringtools' or 'stringlib'.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/04170b80/attachment.pgp
From bac at OCF.Berkeley.EDU  Tue Aug 24 19:39:25 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Aug 24 19:39:41 2004
Subject: [Python-Dev] Re: Update PEP 292
In-Reply-To: <6D9E824FA10BD411BE95000629EE2EC3C6DE1D@FLONIDAN-MAIL>
References: <6D9E824FA10BD411BE95000629EE2EC3C6DE1D@FLONIDAN-MAIL>
Message-ID: <412B7D4D.6010702@ocf.berkeley.edu>

Anders J. Munch wrote:

>>From Brett C. [mailto:bac@OCF.Berkeley.EDU]
> 
>>That is coming down to a question of convenience compared to 
>>explicitness.  Do we want "Give me $1,000,000 now!" to raise 
>>ValueError, 
>>or do we want to let it pass through?
> 
> 
> This particular problem has been considered and decided on years ago,
> in the context of %-interpolation:
> 
>     >>> "Give me %1,000,000 now!" % {}
>     ValueError: unsupported format character ',' (0x2c) at index 10
> 

That is not a fair comparison.  First off, to make it fair, you would 
need to make it ``%(1,000,000)s`` to compare to what $-interpolation is 
doing.

Second, the whole point of this discussion is ease of use and thus 
%-interpolation is not even worth comparing to.  If it was we wouldn't 
be bothering with this.

> String interpolation with named parameters given as a dictionary is a
> problem already solved in Python.  I am puzzled at this urge to
> re-solve the same problem, only differently.
>

It is not solving the idea of using a dictionary to key to values that 
are to interpolated.  It is to use a simpler syntax to represent the 
text to be interpolated.

> If remembering to type the trailing s in "%(varname)s" is so much of a
> problem, why not extend the current interpolation syntax with a
> variation that is more convenient, yet still backwards-compatible?  I
> would suggest a formatting-character-less version of "%(varname)s"
> that uses brackets instead: "%[varname]".
> 

This was all discussed and rejected.  $ is used in so many other 
languages that there is the perk of familiarity to other languages.

And in terms of backwards-compatibility, that is not an issue since this 
is all being put into a self-contained class and not into the str type.

All of this has been discussed long ago and settled on.  At this point 
the only thing left to do is to tell Barry his patch is fine and to just 
apply the sucker.  =)

-Brett
From aahz at pythoncraft.com  Tue Aug 24 20:45:28 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Aug 24 20:45:38 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <1093366837.18858.51.camel@geddy.wooz.org>
References: <1093276486.412a1346e30e9@mcherm.com>
	<20040823161233.GB71531@prometheusresearch.com>
	<1f7befae040823093264275843@mail.gmail.com>
	<1093366837.18858.51.camel@geddy.wooz.org>
Message-ID: <20040824184528.GB2562@panix.com>

On Tue, Aug 24, 2004, Barry Warsaw wrote:
> On Mon, 2004-08-23 at 12:32, Tim Peters wrote:
>> 
>> Apologies to Barry, but I'm +1 on auto-str() too.  It's a string
>> interpolation -- the user is explicitly asking for a string.  If they
>> made a mistake, it was in asking for a string to begin with, not in
>> feeding it a non-string.
> 
> Should it be auto-unicode(), given that Template is derived from
> unicode?  And if so, should we entertain the possibility of insanities
> like giving the user the ability to pass optional arguments to the
> unicode() call?  If the answers to that are yes and no, that's fine with
> me.

Here you go: yes and no
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From cce at clarkevans.com  Tue Aug 24 21:04:22 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Tue Aug 24 21:04:24 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz>
References: <5.1.1.6.0.20040823144545.03c84c50@mail.telecommunity.com>
	<200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040824190422.GA48456@prometheusresearch.com>

A few definitions:

  iterator: >
    An iterator is an object returned by an __iter__ with
    a next() method which produces values.  An iterator's
    state is kept on the heap (not on the stack), thus
    one could consider it 'resumable'.  An iterator ends when
    it raises an exception, the very special exception,
    'StopIteration' is used to signal that the iterator is
    done producing values, but is not really an error condition.
    
  generator: >
    A syntax for making an iterator, it appears in python
    source code as a function or method with a 'yield'
    statement.  An implementation is free to optimize this
    function, but it acts as if it had created an iterator.
   
  iterlink: >
    An iterlink is an association of one iterator, the consumer,
    to another iterator, the producer.  The iterlink contains
    information about both iterators, and a marker to indicate
    the state of the consumer when it asked for the next value
    of the producer.

  iterstack: > 
    A stack of iterlinks, where the producer of one iterlink is
    the consumer of another.  The top iterator in the iterstack
    is an iterator which is behaving like a producer, but not
    a consumer.  The bottom iterator in the iterstack is one
    which is a producer to a non-iterator.  That is, the caller
    of the bottom iterator is a function or method which is
    not an iterator.

Ok.  With this in mind, I like your idea of a new keyword, 'suspend'.
This keyword could be used by the top iterator in an iterstack.  When
'suspend' happens, the entire iterstack, and its runstate, is setaside.
An a signal is sent to the caller of the bottom iterator; the signal
could be viewed as a 'resumable exception'.   For example, it could
be pictured as a 'SuspendIteration' exception.  More concretely, using a
very simple case.

   class TopIterator:
       """  
            def TopIterator():
                yield "one"
                suspend
                yield "two"
       """
       def _one(self):
           self.next = self._suspend
           return "one"
       def _suspend(self):
           self.next = self._two
           raise SuspendIteration()
       def _two(self):
           self.next = self._stop
           return "two"
       def _stop(self):
           raise StopIteration
       def __iter__():
           self.next = self._one
           return self

    class BottomIterator():
        """
            def BottomIterator():
                producer = iter(TopIterator())
                saved_one = producer.next()
                saved_two = producer.next()
                yield (saved_one,saved_two)
        """
        def _one(self):
            self.saved_one = self.producer.next()
            self.next = self._two
            return self.next()
        def _two(self):
            self.saved_two = self.producer.next()
            self.next = self._stop
            return (self.saved_one, self.saved_two)
       def _stop(self):
            raise StopIteration
        def __iter__(self):
            self.producer = iter(TopIterator())
            self.next = self._one
            return self

    def caller():
        it = iter(BottomIterator())
        while True:
            try:
                 print it.next()
            except StopIteration:
                break
            except SuspendIteration:
                sleep(10)
                continue

Anyway, the behavior probably equivalent to 'resumable exceptions',
however, I don't think it is _different_ item from an iterator, just a
new set of behaviors triggered by the 'suspend' keyword.  As long as the
'iterator construction' is automated, the 'caller()' function can be
delegated to Twisted's reactor, or asynccore or any other cooperative
multitasking base.  The superclasses for SuspendIteration could be
'BlockForSocket', for example.

On Tue, Aug 24, 2004 at 05:57:18PM +1200, Greg Ewing wrote:
| This makes me think that maybe we want another kind of object, similar
| to a generator, but designed to be used for side effects rather than
| to produce values.

  (a) the generation of 'top-level' iterators using 'suspend'
      would be quite easy, see above

  (b) the generation logic for "pass-through" generators (aka
      BottomIterator in the example above) would have to happen
      for the mechanism to be truly useful; SuspendIteration
      would have to be a pass-through exception

  (c) updating itertools to allow SuspendIteration to 'passthrough'

| For want of a better term, let's call it a "cooperator" (so it ends in
| "ator" like "generator" :-). Actually there will be two objects, a
| cooperator-function (analogous to a generator-function) and a
| cooperator-instance (analogous to a generator-iterator).
| 
| Calling a cooperator-function will return a cooperator-instance, which
| will obey the cooperator protocol. This protocol consists of a single
| function run(), which causes the cooperator-instance to perform some
| amount of work and then stop. If there is more work remaining to be
| done, run() returns a true value, otherwise it returns a false value.

Well, run() need not be specified.  It would be implemented by the
lower-level 'reactor' code, aka the "caller" in the example

| There will be a new statement:
| 
|     suspend
| 
| for use inside a cooperator-function. The presence of this statement
| is what distinguishes a cooperator-function from an ordinary function
| (just as the presence of 'yield' distinguishes a generator from an
| ordinary function). Execution of 'suspend' will cause the run() method
| to return with a true value. The next time run() is called, execution
| will resume after the 'suspend'.

Right.  Only that I'd allow both 'yield' and 'suspend' in the same
function, or it really isn't that useful.

| This is really all that's needed, but one further piece of syntax may
| be desirable to make it easier for one cooperator to invoke another:
| 
|     do another_coop()
| 
| would be equivalent to
| 
|     co = another_coop()
|     while co.run():
|         suspend

Exactly; this is where the tough part is, the 'intermediate' cooperators.
Ideally, all generators (since they keep their state on the stack) would
make good 'intermediate' cooperators.

| Something to note is that there's no requirement for a cooperator-
| instance to be implemented by a cooperator-function, just as an
| iterator can be implemented in ways other than a generator.  This
| sidesteps some of the difficulties of mixing Python and C calls, since
| a Python cooperator-function could invoke a cooperator implemented in
| C which in turn invokes another Python cooperator- function, etc. The
| only requirement is that all the objects on the way down obey the
| cooperator protocol.

Yes.  This is the key, you _only_ attempt to suspend iterators, that is
objects that already keep their state in the heap.

| The drawback is that you always have to know when you're calling
| another cooperator and use 'do'. But that's no worse than the current
| situation with generators, where you always have to know you're using
| an iterator and use 'for... yield'.

Er, well, if the interface was an 'SuspendIteration' then it could be
generalized to all iterators.  Afterall, while an exception invalidates
the stack-frame for the current call to next(), it doesn't prevent the
caller from invoking next() once again.

If someone calls next() manually, then SuspendIteration would act as
a regular exception... and be uncaught.

Clark
From cce at clarkevans.com  Tue Aug 24 21:12:08 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Tue Aug 24 21:12:11 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <1d6cdae304082404452c15617b@mail.gmail.com>
References: <200408240557.i7O5vIaH002592@cosc353.cosc.canterbury.ac.nz>
	<1d6cdae304082404452c15617b@mail.gmail.com>
Message-ID: <20040824191208.GB48456@prometheusresearch.com>

On Tue, Aug 24, 2004 at 07:45:13AM -0400, Daniel Bickett wrote:
| Tell me if I'm missing something here, but with the exception of your
| two introductory keywords (a subject I thought was supposed to be
| sacred, with the intention of keeping the amount of keywords as small
| as possible,) how much difference is there to generators? Are you
| saying the run() method, after having been called initially, will
| return a value indicative of the cooperator's business when called
| again?

In that such a 'cooperator' would keep its state on the heap (in an
iterator object to be more precice), it is very much like a generator.
In fact, one could implement it using a SuspendIteration exception,
in addition to a StopIteration.  SuspendIteration would signal that
the caller of the iterator _may_ call next() again.  In other words,
it says: 

  "I didn't produce a value this time, but call me again, and
   I may have a value for you".

Specializations of SuspendIteration could signal that the process
is blocked on a socket read, or some other low-level process.  So
that the caller (which would be a generic run-loop in most cases)
would only attempt next() once the block is resolved.

| It seems like a similar feat could be achieved with custom generators
| and boolean yield statements.

  I currently do this in twisted.flow, however, it requires that
  intermediate generators "check" for the special value.   This
  intermediate work is slow (since it is not in C) and tedious
  since it is mostly boilerplate catch-check-yield kinda stuff

On Tue, Aug 24, 2004 at 02:58:42PM +0100, Moore, Paul wrote:
| From: Greg Ewing
| > There will be a new statement:
| >
| >     suspend
| >
| [...]
| >
| >    do another_coop()
| >
| > would be equivalent to
| >
| >    co = another_coop()
| >    while co.run():
| >        suspend
| 
| 
| I'm not sure I see the difference between suspend/do and
| yield True/for _ in co: pass, other than possibly that
| co-operators and generators are intended to be mixed (which
| strikes me as implausible).

You'd definately want to mix them.  For example, a 'cooperator'
would be reading from a socket, it could do one of three things:

  yield the next line from the sender
  raise StopIteration if the socket closed
  raise SuspendIteration if the read() is blocked

The 'suspend' statement is analogous to raising SuspendIteration.

| I'm likely to be missing something here, but I don't follow
| the semantics you are suggesting.
| 
| If there was a simple, realistic use case for this, it might
| help clarify the semantics. (For extra credit, mix co-operators
| and generators in your use case so that the semantics of
| interaction are shown as well :-))

There ya go. ;)

Clark
From nas at arctrix.com  Tue Aug 24 21:33:36 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue Aug 24 21:33:44 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26
In-Reply-To: <E1BzgSr-000338-LB@sc8-pr-cvs1.sourceforge.net>
References: <E1BzgSr-000338-LB@sc8-pr-cvs1.sourceforge.net>
Message-ID: <20040824193336.GA8484@mems-exchange.org>

On Tue, Aug 24, 2004 at 11:56:13AM -0700, montanaro@users.sourceforge.net wrote:
> Modified Files:
> 	pep-0318.txt 
> Log Message:
> List some possible reasons why arriving at consensus about
> decorators has been so hard (or impossible) to acheive.  There are
> certainly more.

Perhaps you could add my reservation (objection is too strong a
word).  I think decorators are not powerful enough given their high
syntactic profile.  This could be rephrased as "if we are going the
use the @ sign then it should be able to do really cool things".

One idea is to have the compiler pass the AST for the function body
to the decorator function.  The decorator could create new nodes in
the AST or modify existing ones.  That would allow decorators to do
things like adding a try/except without introducing another function
call.  The output of the decorator would be passed to the code
generator.

  Neil
From barry at python.org  Tue Aug 24 21:42:58 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 24 21:42:33 2004
Subject: [Python-Dev] sre.py backward compatibility and PEP 291
Message-ID: <1093376578.18850.107.camel@geddy.wooz.org>

PEP 291 says that sre.py needs to be backward compatible with Python
2.1.  sre.py says:

# this module works under 1.5.2 and later.  don't use string methods
import string

Which is it?  Raymond has seen some circular reference problems between
my PEP 292 string.py patch and re.py (really, by way of sre.py and
friends).  Even though I haven't seen the same problems, I'd like to
avoid any direct circular references by changing the few string module
function calls in sre*.py to string methods.  But the comment above
stopped me, until I read PEP 291.

The PEP should be the definitive answer, and I'm willing to
string-meth-ify sre*.py, if there are no objections.

out-posting-phillip-ly y'rs,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/b392f5ba/attachment.pgp
From bac at OCF.Berkeley.EDU  Tue Aug 24 21:57:03 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Aug 24 21:57:12 2004
Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps
	pep-0318.txt, 1.25, 1.26
In-Reply-To: <20040824193336.GA8484@mems-exchange.org>
References: <E1BzgSr-000338-LB@sc8-pr-cvs1.sourceforge.net>
	<20040824193336.GA8484@mems-exchange.org>
Message-ID: <412B9D8F.2060906@ocf.berkeley.edu>

Neil Schemenauer wrote:

> On Tue, Aug 24, 2004 at 11:56:13AM -0700, montanaro@users.sourceforge.net wrote:
> 
>>Modified Files:
>>	pep-0318.txt 
>>Log Message:
>>List some possible reasons why arriving at consensus about
>>decorators has been so hard (or impossible) to acheive.  There are
>>certainly more.
> 
> 
> Perhaps you could add my reservation (objection is too strong a
> word).  I think decorators are not powerful enough given their high
> syntactic profile.  This could be rephrased as "if we are going the
> use the @ sign then it should be able to do really cool things".
> 
> One idea is to have the compiler pass the AST for the function body
> to the decorator function.  The decorator could create new nodes in
> the AST or modify existing ones.  That would allow decorators to do
> things like adding a try/except without introducing another function
> call.  The output of the decorator would be passed to the code
> generator.
> 

That kind of stuff is my dream use of the AST; modifying it before final 
compilation to a .pyc file.  Although that could also just be set up in 
a list that gets called on *all* compilations.

We could also just keep the AST around in the code object, although that 
would be space-consuming.

-Brett
From skip at pobox.com  Tue Aug 24 21:59:33 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 24 21:59:48 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26
In-Reply-To: <20040824193336.GA8484@mems-exchange.org>
References: <E1BzgSr-000338-LB@sc8-pr-cvs1.sourceforge.net>
	<20040824193336.GA8484@mems-exchange.org>
Message-ID: <16683.40485.877453.285028@montanaro.dyndns.org>


    >> List some possible reasons why arriving at consensus about decorators
    >> has been so hard (or impossible) to acheive.  There are certainly
    >> more.

    Neil> Perhaps you could add my reservation (objection is too strong a
    Neil> word).  I think decorators are not powerful enough given their
    Neil> high syntactic profile.  This could be rephrased as "if we are
    Neil> going the use the @ sign then it should be able to do really cool
    Neil> things".

I was trying to elaborate general barriers to concluding this process, not
issues related to selecting one proposal over another.

    Neil> One idea is to have the compiler pass the AST for the function
    Neil> body to the decorator function.  The decorator could create new
    Neil> nodes in the AST or modify existing ones.  That would allow
    Neil> decorators to do things like adding a try/except without
    Neil> introducing another function call.  The output of the decorator
    Neil> would be passed to the code generator.

I suppose you could do that, although if you did I suspect metaclasses would
seem tame by comparison.  Couldn't you do something like this?

    @ast_to_func
    @neils_cool_ast_transform
    @func_to_ast
    def f(a):
        pass

After all, there's no restriction on what a decorator can return.

How about we save true AST decorators for "$"? <wink>

Skip
From walter at livinglogic.de  Tue Aug 24 22:04:25 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Aug 24 22:04:29 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412ACE79.4060605@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com> <4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com> <4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>
	<4123D0C2.4080505@v.loewis.de> <4124DA28.50106@livinglogic.de>
	<4124FA07.1070701@v.loewis.de> <412A5693.4050008@livinglogic.de>
	<412ACE79.4060605@v.loewis.de>
Message-ID: <412B9F49.700@livinglogic.de>

Martin v. L?wis wrote:

> Walter D?rwald wrote:
> 
>> OK, let's come up with a patch that fixes the incomplete byte
>> sequences problem and then discuss non-stream APIs.
>>
>> So, what should the next step be?
> 
> I think your first patch should be taken as a basis for that.
> Add the state-supporting decoding C functions, and change
> the stream readers to use them.

OK, I've updated the patch.

> That still leaves the issue
> of the last read operation, which I'm tempted to leave unresolved
> for Python 2.4.

Agreed! This shouldn't be done for Python 2.4.

> No matter what the solution is, it would likely
> require changes to all codecs, which is not good.

True, but the changes should be rather trivial for most.

Bye,
    Walter D?rwald


From fumanchu at amor.org  Tue Aug 24 22:08:21 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Tue Aug 24 22:13:48 2004
Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps
	pep-0318.txt, 1.25, 1.26
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E54@exchange.hqamor.amorhq.net>

Skip Montanaro (I think) wrote:
> List some possible reasons why arriving at consensus about decorators
> has been so hard (or impossible) to acheive.  There are certainly
more.

First reaction: Each proposal, by its very syntax, suggested a range of
non-decorator use cases and semantics. Unfortunately, we have dozens of
syntax proposals with "ranges of implication" which do not overlap, and
therefore cannot be evaluated fairly (very little "apples to apples"
comparison). Most of the remaining discussions addressed Beauty, Fear,
Magic, or Intuition, which take more effort and space to analyze
properly than your typical Usenet post allows.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From walter at livinglogic.de  Tue Aug 24 22:15:01 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Aug 24 22:16:04 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412B0C2D.7030507@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>
	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>
	<412A5693.4050008@livinglogic.de>
	<412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com>
Message-ID: <412BA1C5.5050808@livinglogic.de>

M.-A. Lemburg wrote:

> Martin v. L?wis wrote:
> 
>> Walter D?rwald wrote:
>>
>>> OK, let's come up with a patch that fixes the incomplete byte
>>> sequences problem and then discuss non-stream APIs.
>>>
>>> So, what should the next step be?
>>
>> I think your first patch should be taken as a basis for that.
> 
> We do need a way to communicate state between the codec
> and Python.
> 
> However, I don't like the way that the patch
> implements this state handling: I think we should use a
> generic "state" object here which is passed to the stateful
> codec and returned together with the standard return values
> on output:
> 
> def decode_stateful(data, state=None):
>     ... decode and modify state ...
>     return (decoded_data, length_consumed, state)

Another option might be that the decode function changes
the state object in place.

> where the object type and contents of the state variable
> is defined per codec (e.g. could be a tuple, just a single
> integer or some other special object).

If a tuple is passed and returned this makes it possible
from Python code to mangle the state. IMHO this should be
avoided if possible.

> Otherwise we'll end up having different interface
> signatures for all codecs and extending them to accomodate
> for future enhancement will become unfeasable without
> introducing yet another set of APIs.

We already have slightly different decoding functions:
utf_16_ex_decode() takes additional arguments.

> Let's discuss this some more and implement it for Python 2.5.
> For Python 2.4, I think we can get away with what we already
> have:
 > [...]

OK, I've updated the patch.

> [...]
> The buffer logic should only be used for streams
> that do not support the interface to push back already
> read bytes (e.g. .unread()).
> 
>  From a design perspective, keeping read data inside the
> codec is the wrong thing to do, simply because it leaves
> the input stream in an undefined state in case of an error
> and there's no way to correlate the stream's read position
> to the location of the error.
> 
> With a pushback method on the stream, all the stream
> data will be stored on the stream, not the codec, so
> the above would no longer be a problem.

On the other hand this requires special stream. Data
already read is part of the codec state, so why not
put it into the codec?

> However, we can always add the .unread() support to the
> stream codecs at a later stage, so it's probably ok
> to default to the buffer logic for Python 2.4.

OK.

>> That still leaves the issue
>> of the last read operation, which I'm tempted to leave unresolved
>> for Python 2.4. No matter what the solution is, it would likely
>> require changes to all codecs, which is not good.
> 
> We could have a method on the codec which checks whether
> the codec buffer or the stream still has pending data
> left. Using this method is an application scope consideration,
> not a codec issue.

But this mean that the normal error handling can't be used
for those trailing bytes.

Bye,
    Walter D?rwald


From martin at v.loewis.de  Tue Aug 24 22:38:39 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 24 22:38:59 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412B0C2D.7030507@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>
	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>
	<412A5693.4050008@livinglogic.de>
	<412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com>
Message-ID: <412BA74F.6030701@v.loewis.de>

M.-A. Lemburg wrote:
> However, I don't like the way that the patch
> implements this state handling: I think we should use a
> generic "state" object here which is passed to the stateful
> codec and returned together with the standard return values
> on output:

Why is that better? Practicality beats purity.
This is useless over-generalization.

> Otherwise we'll end up having different interface
> signatures for all codecs and extending them to accomodate
> for future enhancement will become unfeasable without
> introducing yet another set of APIs.

What is "a codec" here? A class implementing the StreamReader
and/or Codec interface? Walter's patch does not change the
API of any of these. It just adds a few functions to some
module, which are not meant to be called directly.

> If we leave out the UTF-7 codec changes in the
> patch, the only state that the UTF-8 and UTF-16
> codecs create is the number of bytes consumed. We already
> have the required state parameter for this in the
> standard decode API, so no extra APIs are needed for
> these two codecs.

Where precisely is the number of decoded bytes in the API?

Regards,
Martin

From nas at arctrix.com  Tue Aug 24 22:58:58 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue Aug 24 22:59:03 2004
Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps
	pep-0318.txt, 1.25, 1.26
In-Reply-To: <16683.40485.877453.285028@montanaro.dyndns.org>
References: <E1BzgSr-000338-LB@sc8-pr-cvs1.sourceforge.net>
	<20040824193336.GA8484@mems-exchange.org>
	<16683.40485.877453.285028@montanaro.dyndns.org>
Message-ID: <20040824205858.GA8823@mems-exchange.org>

On Tue, Aug 24, 2004 at 02:59:33PM -0500, Skip Montanaro wrote:
> Couldn't you do something like this?
> 
>     @ast_to_func
>     @neils_cool_ast_transform
>     @func_to_ast
>     def f(a):
>         pass

That doesn't work as there is, AFAIK, no way to write func_to_ast().
The necessary information has already been thrown away when it gets
called.

  Neil
From fumanchu at amor.org  Tue Aug 24 22:56:04 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Tue Aug 24 23:01:31 2004
Subject: [Python-Dev] Important decorator proposal on c.l.p.
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E58@exchange.hqamor.amorhq.net>

In case it wasn't made clear earlier, there's an effort being made to
propose an alternative to the current @decorator syntax before 2.4 hits
beta. For everyone's sanity, that discussion is going on wholly on
comp.lang.python. I figured I'd mention it to those of you who don't
read clp on a regular basis--I should have done so earlier. You can find
the initial announcement at
http://groups.google.com/groups?th=c62eba24231bc2fb if you are
interested in contributing/commenting. But hurry! Deadlines loom.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From pje at telecommunity.com  Tue Aug 24 23:02:43 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Aug 24 23:02:29 2004
Subject: [Python-Dev] Re: [Python-checkins] python/nondist/peps
	pep-0318.txt, 1.25, 1.26
In-Reply-To: <20040824205858.GA8823@mems-exchange.org>
References: <16683.40485.877453.285028@montanaro.dyndns.org>
	<E1BzgSr-000338-LB@sc8-pr-cvs1.sourceforge.net>
	<20040824193336.GA8484@mems-exchange.org>
	<16683.40485.877453.285028@montanaro.dyndns.org>
Message-ID: <5.1.1.6.0.20040824170201.023a33c0@mail.telecommunity.com>

At 04:58 PM 8/24/04 -0400, Neil Schemenauer wrote:
>On Tue, Aug 24, 2004 at 02:59:33PM -0500, Skip Montanaro wrote:
> > Couldn't you do something like this?
> >
> >     @ast_to_func
> >     @neils_cool_ast_transform
> >     @func_to_ast
> >     def f(a):
> >         pass
>
>That doesn't work as there is, AFAIK, no way to write func_to_ast().

There is one, but it's ugly and involves inspect.get_source.  :)

From fredrik at pythonware.com  Tue Aug 24 23:05:24 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Aug 24 23:03:42 2004
Subject: [Python-Dev] Re: sre.py backward compatibility and PEP 291
References: <1093376578.18850.107.camel@geddy.wooz.org>
Message-ID: <cggaf9$2cm$1@sea.gmane.org>

Barry wrote:

> PEP 291 says that sre.py needs to be backward compatible with Python
> 2.1.  sre.py says:
>
> # this module works under 1.5.2 and later.  don't use string methods
> import string
>
> Which is it?

Martin changed the PEP last week (or so), but forgot to remove the
comment.

Note that you can remove some code from _sre.c as well (see join_list,
and perhaps other places that checks for >= 1.6).

cheers /F 



From barry at python.org  Tue Aug 24 23:37:36 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 24 23:37:10 2004
Subject: [Python-Dev] Re: sre.py backward compatibility and PEP 291
In-Reply-To: <cggaf9$2cm$1@sea.gmane.org>
References: <1093376578.18850.107.camel@geddy.wooz.org>
	<cggaf9$2cm$1@sea.gmane.org>
Message-ID: <1093383456.18852.125.camel@geddy.wooz.org>

On Tue, 2004-08-24 at 17:05, Fredrik Lundh wrote:

> Martin changed the PEP last week (or so), but forgot to remove the
> comment.
> 
> Note that you can remove some code from _sre.c as well (see join_list,
> and perhaps other places that checks for >= 1.6).

Awesome, thanks.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/81c5ca69/attachment.pgp
From martin at v.loewis.de  Tue Aug 24 23:42:01 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 24 23:42:00 2004
Subject: [Python-Dev] Re: sre.py backward compatibility and PEP 291
In-Reply-To: <cggaf9$2cm$1@sea.gmane.org>
References: <1093376578.18850.107.camel@geddy.wooz.org>
	<cggaf9$2cm$1@sea.gmane.org>
Message-ID: <412BB629.2090104@v.loewis.de>

Fredrik Lundh wrote:
> Barry wrote:

>>PEP 291 says that sre.py needs to be backward compatible with Python
>>2.1.  sre.py says:
>>
>># this module works under 1.5.2 and later.  don't use string methods
>>import string
>>
>>Which is it?
> 
> 
> Martin changed the PEP last week (or so), but forgot to remove the
> comment.

I only changed it for xmlrpclib, and changed the comment in that module.

Just made the change for sre in pep-0291.txt 1.5, on 02-Jul-03. The
commit message there says

Bumped sre's "baseline" to 2.1, per /F's recommendation in bug #764548

Regards,
Martin
From bac at OCF.Berkeley.EDU  Tue Aug 24 23:53:26 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Aug 24 23:53:34 2004
Subject: [Python-Dev] Need double-check on patch for bug #991962
 (--disable-toolbox-glue compile on OS X)
Message-ID: <412BB8D6.5020008@ocf.berkeley.edu>

Finally got around to writing up a quick patch for setup.py to not 
compile OS X-specific modules if the build is configured for 
--disable-toolbox-glue (would always have the Mac modules fail to build 
thanks to link errors if the toolbox glue was not compiled).  But since 
I hardly ever play with the build process and the last thing I want to 
do is break it I would appreciate someone taking a quick look at this 
(changed two 'if' statements so it's small).

And this is an indirect request to Anthony to bump the priority.  =)

-Brett
From ncoghlan at iinet.net.au  Wed Aug 25 00:20:39 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Aug 25 00:22:29 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <412AFEC4.6020007@egenix.com>
References: <002401c48931$dda69060$e841fea9@oemcomputer>	<1093304992.412a82a0a5a88@mail.iinet.net.au>
	<412AFEC4.6020007@egenix.com>
Message-ID: <412BBF37.4010402@iinet.net.au>

M.-A. Lemburg wrote:
> ncoghlan@iinet.net.au wrote:
> 
>> pass (essentially, all of the work done so far is thrown away, and the 
>> Unicode
>> join starts from scratch. If you know you have Unicode, you're better 
>> off using
>> a Unicode separator to avoid this unnecessary work </tangent>).
> 
> 
> It's just a simple length querying loop; there's no storage allocation
> or anything expensive happening there, so the "throw-away" operation
> is not expensive.

Yes, my mistake. The tuple that string join creates when the argument is 
an iterable rather than a list or tuple isn't thrown away - it is given 
to PyUnicode_Join to work with (since the original iterator may not be 
able to be iterated a second time).

The only work that is lost is that all of the type checks that have been 
done on prior elements will get repeated in the Unicode join code. So 
it's not as bad as I first thought - but it will still cost a few cycles.

Regards,
Nick.
From python at rcn.com  Wed Aug 25 00:24:49 2004
From: python at rcn.com (Raymond Hettinger)
Date: Wed Aug 25 00:25:19 2004
Subject: [Python-Dev] 
	FW: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26
Message-ID: <004b01c48a29$2b6d6ba0$04f9cc97@oemcomputer>

> List some possible reasons why arriving at consensus about 
> decorators has been so hard (or impossible) to achieve

Thanks.  I think that was an important contribution to the discussion.
At this point, looking at the meta-discussion is likely to best way to
help cut through the current morass.



+There is no one clear reason why this should be so, but a
+few problems seem to be most problematic.

A possible social issue is that decorators can be used in a tremendous
variety of ways, each of which is only needed or appreciated by small,
disjoint groups of users.  For instance, applications to ctypes have
unique needs that not understood or shared by others. Some users want to
use decorators for metaclass magic that scares the socks off of the
rest.


> +  Almost everyone agrees that decorating/transforming a function at
> +  the end of its definition is suboptimal.  

One other point of agreement is that no one like having to write the
function name three times:
   def f():
       ...
   f = deco(f)



> +* Syntactic constraints.

There is an existing (though clumsy) syntax.   Most of the proposals are
less flexible but more elegant.  This trade-off has created much more
consternation than if there were no existing ways to apply decorations.



+* Overall unfamiliarity with the concept.  For people who have a
+  passing acquaintance with algebra (or even basic arithmetic) or have
+  used at least one other programming language, much of Python is
+  intuitive.  Very few people will have had any experience with the
+  decorator concept before encountering it in Python.  There's just no
+  strong preexisting meme that captures the concept.

My take on this one is that there are some existing memes from C# and
Java and that in the future they will be more widely known.

However, there are not good existing, thought-out concepts of what
exactly decorators should do in terms of preserving docstrings,
attributes, using parameters, etc.  Most have only a single application
in mind and the full implications (and possible applications) of a given
syntax are shrouded in the mists of the future.

Like metaclasses, everyone knew they were powerful when they were put
in, but no one knew how they would be used or whether they were
necessary.  In fact, two versions later, we still don't know those
answers.



Raymond

From pje at telecommunity.com  Wed Aug 25 00:54:49 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Aug 25 00:54:41 2004
Subject: [Python-Dev]  FW: [Python-checkins] python/nondist/peps
	pep-0318.txt, 1.25, 1.26
In-Reply-To: <004b01c48a29$2b6d6ba0$04f9cc97@oemcomputer>
Message-ID: <5.1.1.6.0.20040824184328.0297bd40@mail.telecommunity.com>

At 06:24 PM 8/24/04 -0400, Raymond Hettinger wrote:
>Like metaclasses, everyone knew they were powerful when they were put
>in, but no one knew how they would be used or whether they were
>necessary.  In fact, two versions later, we still don't know those
>answers.

The people who've been using metaclasses since Python 1.5 (including e.g., 
the Zope developers) would probably disagree.  (Metaclasses got an upgrade 
in capabilities due to type/class unification, plus a more convenient 
syntax in 2.2, but they were around long before that.)

Similarly, there are plenty of "pyoneers" with use cases today for 
decorators, and they know how they're being used today, and that an 
improved syntax is necessary.

I'd point to the fact that there has been *very* little argument among 
decorator proponents regarding the desired semantics, as an indication that 
we *do* know what capabilities are necessary and/or desirable.

(By decorator proponents, I mean people who have developed and use their 
own decorators, who have spoken in favor of having PEP 318 implemented for 
2.4.  That is, people interested enough in the functionality to have used 
the current syntax to get them.)

From trentm at ActiveState.com  Wed Aug 25 02:16:34 2004
From: trentm at ActiveState.com (Trent Mick)
Date: Wed Aug 25 02:17:15 2004
Subject: [Python-Dev] socketmodule build failures on Un*xs that don't
	HAVE_GETADDRINFO
Message-ID: <20040824171634.A8031@ActiveState.com>


Building the socket module with the current Python CVS (or at least with
the Python 2.4a2 source) on Solaris 2.6, HP-UX 11, and AIX 5.1 fails in
Modules/getaddrinfo.c with complaints about

    cc: ".../python/Modules/getaddrinfo.c", line 204: error 1588: "EAI_MAX" undefined.

    ".../python/Modules/getaddrinfo.c", line 282.25: 1506-045 (S) Undeclared identifier EAI_BADHINTS.
    ".../python/Modules/getaddrinfo.c", line 283.40: 1506-045 (S) Undeclared identifier AI_MASK.
    ".../python/Modules/getaddrinfo.c", line 373.41: 1506-045 (S) Undeclared identifier EAI_PROTOCOL.

and the like.

The problem seems to stem from this check-in which resulted in
Modules/addrinfo.h *not* getting included for Unix flavours that do not
HAVE_GETADDRINFO.

    http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.288&r2=1.289
    Wed May 26 17:06:31 2004 UTC (2 months, 4 weeks ago) by mwh 
    > Band-aid type fix for
    > 
    > [ 728330 ] Don't define _SGAPI on IRIX
    > 
    > The Right Thing would be nice, for now this'll do.  At least it
    > isn't going to break anything *other* than IRIX...

Relevant bug:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=728330&group_id=5470

Subseqent checkins have gone in to fix the lack of addrinfo.h import on
some platforms:

    http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.292&r2=1.293
    http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.297&r2=1.298

So the question: was that original change removing the 
    #include "addrinfo.h"
an accident? I.e. should it have been something like this instead?

    #if defined(__sgi)
    # if _COMPILER_VERSION>700 \
      && !defined(_SS_ALIGNSIZE) /* defined in sys/socket.h            */
                                 /* by some newer versions of IRIX     */
                                 /* (e.g. not by 6.5.10 but by 6.5.21) */
    #  include "addrinfo.h"
    # endif
    #else
    # include "addrinfo.h"
    #endif

Cheers,
Trent

-- 
Trent Mick
TrentM@ActiveState.com
From barry at python.org  Wed Aug 25 03:54:22 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 25 03:53:58 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <20040824184528.GB2562@panix.com>
References: <1093276486.412a1346e30e9@mcherm.com>
	<20040823161233.GB71531@prometheusresearch.com>
	<1f7befae040823093264275843@mail.gmail.com>
	<1093366837.18858.51.camel@geddy.wooz.org>
	<20040824184528.GB2562@panix.com>
Message-ID: <1093398861.4452.19.camel@geddy.wooz.org>

On Tue, 2004-08-24 at 14:45, Aahz wrote:

> Here you go: yes and no

Cool.  Well Aahz, I'm taking that as the green light to check this stuff
in.  It doesn't seem like anybody else wants to test the patch before
hand, so in it goes.  I'll also be checking in a de-string-ified sre
module, docs, test cases, and an updated PEP 292.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/4261adfe/attachment.pgp
From greg at cosc.canterbury.ac.nz  Wed Aug 25 04:51:22 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug 25 04:51:27 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <5.1.1.6.0.20040824071549.02d407c0@mail.telecommunity.com>
Message-ID: <200408250251.i7P2pMGI004207@cosc353.cosc.canterbury.ac.nz>

"Phillip J. Eby" <pje@telecommunity.com>:

> If you could throw a special kind of exception (or even a regular
> exception), and call traceback.resume() to pick up execution at the
> point where it was thrown, whether thrown by a generator or a
> regular function.

Actually, it probably wouldn't be too hard to make exceptions
resumable -- provided all the frames in between are Python. If the
exception gets thrown through any C calls, though, resumption becomes
impossible.  Some other structure is needed to hold the state of a
resumable C call.

I agree that maintaining a stack of *ators automatically somehow
would be desirable, but I haven't figured out yet exactly where
and how that stack would be maintained.

> What's really needed (IMO) is to add a way to communicate *into* a
> co-operator, passing it a value to "accept" or a traceback to raise.
> E.g.:
> 
>      input = suspend output

There have been many suggestions in the past for 'yield' to be
extended to allow values to be passed in as well as out. They all
suffer from a problem of asymmetry. However you arrange it, you always
end up having to discard the first value passed out or pass a dummy
value in the first time, or something like that.  I deliberately left
communication out of the suspend to avoid those problems. If you want
communication, you have to arrange it some other way.

> Anyway, as I said, what would be *most* useful for async programming is a 
> way to resume a traceback, because then you wouldn't need for every 
> intervening frame to have special suspension capability.

Certainly, but that way, ultimately, lies Stackless...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Wed Aug 25 04:59:43 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug 25 04:59:55 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <1d6cdae304082404452c15617b@mail.gmail.com>
Message-ID: <200408250259.i7P2xhX6004230@cosc353.cosc.canterbury.ac.nz>

> how much difference is there to generators?

Very little! Probably too little, in fact, to justify having a
different kind of entity at all. I only suggested it because there
seems to be a reluctance to extend generators in any way that might
suggest using them for purposes other than generation.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From tim.peters at gmail.com  Wed Aug 25 05:00:50 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Aug 25 05:00:52 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <1093366837.18858.51.camel@geddy.wooz.org>
References: <1093276486.412a1346e30e9@mcherm.com>
	<20040823161233.GB71531@prometheusresearch.com>
	<1f7befae040823093264275843@mail.gmail.com>
	<1093366837.18858.51.camel@geddy.wooz.org>
Message-ID: <1f7befae04082420007ec812b@mail.gmail.com>

[Barry Warsaw, on auto-str()]
> Should it be auto-unicode(), given that Template is derived from
> unicode?  And if so, should we entertain the possibility of insanities
> like giving the user the ability to pass optional arguments to the
> unicode() call?  If the answers to that are yes and no, that's fine with
> me.

unicode ... heh, I heard something about that in Java once.  Did
Python grow one of those too?  I sure hope it doesn't get in the way
of using God-given American strings!

no-no-no-no-no-but-i'll-settle-for-yes-no-ly y'rs  - tim
From tim.peters at gmail.com  Wed Aug 25 05:10:15 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Aug 25 05:10:17 2004
Subject: [Python-Dev] problem with pymalloc on the BeOS port.
In-Reply-To: <5317443691-BeMail@taz>
References: <5317443691-BeMail@taz>
Message-ID: <1f7befae04082420104f9158df@mail.gmail.com>

[Fran?ois Revol]
> Now, I don't see why malloc itself would give such a result, it's
> pyMalloc which places those marks, so the thing malloc does wouldn't
> place them 4 bytes of each other for no reason, or repport 0 bytes
> where 4 are allocated.

I think you're fooling yourself if you believe 4 *were* allocated. 
The memory dump shows nothing but gibberish, with 4 blocks of fbfbfbfb
not a one of which makes sense in context (the numbers before and
after them make no sense as "# of bytes allocated" or as "serial
number" values, so these forbidden-byte blocks don't make sense as
either end of an active pymalloc block).

You should at least try to get a C traceback at this point, on the
chance that the routine passing the pointer is a clue.  We don't even
know here yet whether the complaint came from a free() or realloc()
call.

This isn't going to be easy.
From kbk at shore.net  Wed Aug 25 05:23:15 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Aug 25 05:23:19 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200408250323.i7P3NFF7006771@h006008a7bda6.ne.client2.attbi.com>


Patch / Bug Summary
___________________

Patches :  259 open ( -5) /  2573 closed (+17) /  2832 total (+12)
Bugs    :  745 open ( +0) /  4405 closed (+21) /  5150 total (+21)
RFE     :  150 open ( +1) /   130 closed ( +0) /   280 total ( +1)

New / Reopened Patches
______________________

Patch for bug 933795. term.h and curses on Solaris  (2004-08-19)
       http://python.org/sf/1012280  opened by  Ariel Arjona

PEP 292 implementation  (2004-08-22)
       http://python.org/sf/1014055  opened by  Barry A. Warsaw

Rational.py various bugfixes  (2004-08-19)
       http://python.org/sf/1012468  opened by  Russell Blau

cStringIO's truncate doesn't  (2004-08-20)
CLOSED http://python.org/sf/1012740  opened by  Dima Dorfman

Fix for duplicate attributes in generated HTML  (2004-08-20)
       http://python.org/sf/1013055  opened by  Richard Brodie

Cleanup Peepholer Output  (2004-08-22)
CLOSED http://python.org/sf/1013667  opened by  Raymond Hettinger

Implementation for PEP 318 using syntax J2  (2004-08-22)
       http://python.org/sf/1013835  opened by  Michael

RobotFileParser.can_fetch return value  (2004-08-23)
CLOSED http://python.org/sf/1014237  opened by  George Yoshida

Expose current parse location to XMLParser  (2004-08-24)
       http://python.org/sf/1014930  opened by  Dave Cole

bug in tarfile.ExFileObject.readline  (2004-08-24)
       http://python.org/sf/1014992  opened by  GaryD

Improve markup and punctuation in libsocket.tex  (2004-08-24)
       http://python.org/sf/1015012  opened by  Dima Dorfman

Docs claim that coerce can return None  (2004-08-24)
       http://python.org/sf/1015021  opened by  Dima Dorfman

Patches Closed
______________

baseinteger: abstract superclass for int and long  (2004-08-11)
       http://python.org/sf/1007068  closed by  nnorwitz

Improve error reporting when Python opens source code  (2004-08-19)
       http://python.org/sf/1011822  closed by  loewis

Add Py_InitializeEx to allow embedding without signals.  (2004-02-19)
       http://python.org/sf/900727  closed by  loewis

implementation of Text.dump method  (2002-04-19)
       http://python.org/sf/546244  closed by  loewis

Return new string for single subclass joins (Bug #1001011)  (2004-08-11)
       http://python.org/sf/1007087  closed by  rhettinger

cStringIO's truncate doesn't  (2004-08-20)
       http://python.org/sf/1012740  closed by  tim_one

Bad URL encoding in SimpleHTTPServer directory listing  (2004-08-18)
       http://python.org/sf/1011123  closed by  jlgijsbers

doctest: add a special (dedented) marker for blank lines  (2004-04-11)
       http://python.org/sf/933238  closed by  jlgijsbers

incorrect end-of-message handling in mailbox.BabylMailbox  (2004-01-20)
       http://python.org/sf/880621  closed by  jlgijsbers

(bug 1005936) textunderscore python.sty fix  (2004-08-14)
       http://python.org/sf/1009373  closed by  nnorwitz

Cleanup Peepholer Output  (2004-08-22)
       http://python.org/sf/1013667  closed by  rhettinger

Non-ascii in non-unicode __credits__ in Lib/pydoc.py  (2004-08-15)
       http://python.org/sf/1009389  closed by  loewis

Allow pydoc to work with XP Themes (.manifest file)  (2004-07-25)
       http://python.org/sf/997284  closed by  loewis

LC_CTYPE locale and strings  (2004-07-26)
       http://python.org/sf/997768  closed by  loewis

Installation database patch  (2002-05-29)
       http://python.org/sf/562100  closed by  akuchling

RobotFileParser.can_fetch return value  (2004-08-23)
       http://python.org/sf/1014237  closed by  loewis

thread Module Breaks PyGILState_Ensure()  (2004-08-17)
       http://python.org/sf/1010677  closed by  mhammond

New / Reopened Bugs
___________________

dictionary referencing error  (2004-08-19)
CLOSED http://python.org/sf/1012249  opened by  AMIT Consulting LLC

weakref.WeakValueDictionary should override .has_key()  (2004-08-19)
CLOSED http://python.org/sf/1012315  opened by  Myers Carpenter

ctrl-left/-right works incorectly with diacritics  (2004-08-19)
       http://python.org/sf/1012435  opened by  Krzysztof Wilkosz

Can't pipe input to a python program  (2004-08-20)
CLOSED http://python.org/sf/1012692  opened by  Ronald L. Rivest

xmlrpclib, PEP291, "yield"  (2004-08-16)
CLOSED http://python.org/sf/1009803  reopened by  anthonybaxter

Python Tutorial 3.1.4 List example  (2004-08-20)
CLOSED http://python.org/sf/1013239  opened by  Elizibeth Thompson

Win XP DEP prevents Python call to 'C' DLL  (2004-08-21)
CLOSED http://python.org/sf/1013418  opened by  ajhewitt

xml.dom documentation omits createDocument, ...DocumentType  (2004-08-21)
       http://python.org/sf/1013525  opened by  Mike Brown

Error in Chapter 4 of Tutorial  (2004-08-22)
CLOSED http://python.org/sf/1013760  opened by  Alexey Peshehonov

No documentation for PyFunction_* (C-Api)  (2004-08-22)
       http://python.org/sf/1013800  opened by  Henning G?nther

tarfile.open with mode w|bz2  (2004-08-22)
CLOSED http://python.org/sf/1013882  opened by  Felix Wiemann

Error in representation of complex numbers  (2004-08-22)
CLOSED http://python.org/sf/1013908  opened by  Raymond Hettinger

mimetypes add_type has bogus self parameter  (2004-08-22)
       http://python.org/sf/1014022  opened by  Cliff Wells

Unspecific errors with metaclass  (2004-08-23)
       http://python.org/sf/1014215  opened by  Thomas Heller

optparse: parser.remove_option("-h") inconsistency  (2004-08-23)
       http://python.org/sf/1014230  opened by  strop

Missing urllib.urlretrieve docs  (2004-08-23)
       http://python.org/sf/1014761  opened by  David Abrahams

Misc/NEWS no valid reStructuredText  (2004-08-24)
       http://python.org/sf/1014770  opened by  Felix Wiemann

Misc/NEWS.help  (2004-08-24)
       http://python.org/sf/1014775  opened by  Felix Wiemann

"article id" in description of NNTP objects  (2004-08-24)
       http://python.org/sf/1015140  opened by  Felix Wiemann

cgi.FieldStorage.__len__ eventually throws TypeError  (2004-08-24)
       http://python.org/sf/1015249  opened by  Andreas Ames

Create cgi.FieldStorage._get_new_instance method as factory  (2004-08-24)
       http://python.org/sf/1015254  opened by  Andreas Ames

readline broken  (2004-08-24)
CLOSED http://python.org/sf/1015517  opened by  Walter D?rwald

Bugs Closed
___________

dictionary referencing error  (2004-08-19)
       http://python.org/sf/1012249  closed by  rhettinger

weakref.WeakValueDictionary should override .has_key()  (2004-08-19)
       http://python.org/sf/1012315  closed by  rhettinger

str.join([ str-subtype-instance ]) misbehaves  (2004-07-30)
       http://python.org/sf/1001011  closed by  rhettinger

compile errors on _codecs_iso2022.c  (2004-08-09)
       http://python.org/sf/1005737  closed by  perky

SGI (IRIX 6.5.24) Error building _codecs_iso2022.c  (2004-08-11)
       http://python.org/sf/1007249  closed by  perky

Warning in cjkcodecs/iso2022common.h  (2004-07-16)
       http://python.org/sf/991834  closed by  perky

test_queue fails occasionally  (2004-08-17)
       http://python.org/sf/1010777  closed by  mwh

Can't pipe input to a python program  (2004-08-20)
       http://python.org/sf/1012692  closed by  tim_one

xmlrpclib, PEP291, "yield"  (2004-08-16)
       http://python.org/sf/1009803  closed by  loewis

xmlrpclib, PEP291, "yield"  (2004-08-16)
       http://python.org/sf/1009803  closed by  loewis

logging.getLevelName returns a number  (2004-08-12)
       http://python.org/sf/1008295  closed by  vsajip

Python Tutorial 3.1.4 List example  (2004-08-21)
       http://python.org/sf/1013239  closed by  perky

optparse indents without respect to $COLUMNS  (2004-01-16)
       http://python.org/sf/878453  closed by  jlgijsbers

Win XP DEP prevents Python call to 'C' DLL  (2004-08-21)
       http://python.org/sf/1013418  closed by  loewis

Index entries with "_" are wrong in pdf file  (2004-08-09)
       http://python.org/sf/1005936  closed by  nnorwitz

xmlrpclib and backward compatibility  (2003-11-15)
       http://python.org/sf/842600  closed by  loewis

Error in Chapter 4 of Tutorial  (2004-08-22)
       http://python.org/sf/1013760  closed by  rhettinger

tarfile.open with mode w|bz2  (2004-08-22)
       http://python.org/sf/1013882  closed by  loewis

Error in representation of complex numbers  (2004-08-22)
       http://python.org/sf/1013908  closed by  loewis

Cmd in thread segfaults after Ctrl-C  (2004-06-06)
       http://python.org/sf/967334  closed by  mwh

2.3.4 fails build on solaris 10 - complexobject.c  (2004-06-10)
       http://python.org/sf/970334  closed by  mwh

readline broken  (2004-08-24)
       http://python.org/sf/1015517  closed by  nnorwitz

New / Reopened RFE
__________________

Standard exceptions are hostile to Unicode  (2004-08-20)
       http://python.org/sf/1012952  opened by  Marius Gedminas

From barry at python.org  Wed Aug 25 05:30:57 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 25 05:30:31 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <1f7befae04082420007ec812b@mail.gmail.com>
References: <1093276486.412a1346e30e9@mcherm.com>
	<20040823161233.GB71531@prometheusresearch.com>
	<1f7befae040823093264275843@mail.gmail.com>
	<1093366837.18858.51.camel@geddy.wooz.org>
	<1f7befae04082420007ec812b@mail.gmail.com>
Message-ID: <1093404657.4453.41.camel@geddy.wooz.org>

On Tue, 2004-08-24 at 23:00, Tim Peters wrote:

> unicode ... heh, I heard something about that in Java once.  Did
> Python grow one of those too?  I sure hope it doesn't get in the way
> of using God-given American strings!

That's "Freedom Strings" to you, buddy.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040824/664082b5/attachment.pgp
From greg at cosc.canterbury.ac.nz  Wed Aug 25 05:32:28 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug 25 05:32:34 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <20040824191208.GB48456@prometheusresearch.com>
Message-ID: <200408250332.i7P3WSkG004275@cosc353.cosc.canterbury.ac.nz>

"Clark C. Evans" <cce@clarkevans.com>:

> In fact, one could implement it using a SuspendIteration exception,
> in addition to a StopIteration.

I don't think this can be done quite right by raising an exception in
the normal way, because you don't want 'finally' clauses to be
triggered.

> You'd definately want to mix them.  For example, a 'cooperator'
> would be reading from a socket, it could do one of three things:
> 
>   yield the next line from the sender
>   raise StopIteration if the socket closed
>   raise SuspendIteration if the read() is blocked

Hmmm. It seems that generation and cooperation are really orthogonal
concepts -- you may want something that is both a generator *and* a
cooperator. My proposal doesn't allow for this. I will have to do some
more thinking...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From greg at cosc.canterbury.ac.nz  Wed Aug 25 05:36:48 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Aug 25 05:36:58 2004
Subject: [Python-Dev]	Re: [Python-checkins] python/nondist/peps
	pep-0318.txt, 1.25, 1.26
In-Reply-To: <16683.40485.877453.285028@montanaro.dyndns.org>
Message-ID: <200408250336.i7P3amJI004285@cosc353.cosc.canterbury.ac.nz>

> How about we save true AST decorators for "$"? <wink>

Nah, AST decorators should be spelled "@$~" (that's "AST"
written in funny symbols).

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From guido at python.org  Wed Aug 25 05:47:04 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug 25 05:47:16 2004
Subject: [Python-Dev] Proposed change to logging
In-Reply-To: Your message of "Tue, 24 Aug 2004 14:57:18 BST."
	<20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com> 
References: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com> 
Message-ID: <200408250347.i7P3l4O21288@guido.python.org>

> import logging
>  
> logging.basicConfig()
> logger = logging.getLogger("my.app")
> err = logger.getFileObject(logging.ERROR)
> dbg = logger.getFileObject(logging.DEBUG)
> ...
>  
> print >> err, "My logging message with %s" %
> "arguments"
> print >> dbg, "A message at %s level" % "debug"

Is this really a useful improvement?  It seems to save a few
keystrokes at most.  TOOWTDI etc.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From raymond.hettinger at verizon.net  Wed Aug 25 06:11:04 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed Aug 25 06:11:40 2004
Subject: [Python-Dev] Is something wrong with SF patch notifications?
Message-ID: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer>

I don't think Barry got notes showing that I reviewed his patch.
Also, I didn't get a note when Walter assigned a unittest patch to me.
That may also explain why I had a hard time soliciting a reviewer for
the peepholer patch.
 
IIRC, we always used to get notes whenever we were assigned a patch or
had participated in the patch discussions.
 
 
Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040825/692353c8/attachment.html
From tim.peters at gmail.com  Wed Aug 25 06:24:06 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Aug 25 06:24:09 2004
Subject: [Python-Dev] Is something wrong with SF patch notifications?
In-Reply-To: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer>
References: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer>
Message-ID: <1f7befae04082421246fa2a2ce@mail.gmail.com>

[Raymond Hettinger]
> I don't think Barry got notes showing that I reviewed his patch.
>
> so, I didn't get a note when Walter assigned a unittest patch to me.
>
> That may also explain why I had a hard time soliciting a reviewer for
> the peepholer patch.

I got SF email about all these things -- but I'm subscribed to the
patches list, and it doesn't look like you are.

> IIRC, we always used to get notes whenever we were assigned a
> patch or had participated in the patch discussions.

Never the latter.  It should generate email to the submitter, and to
the person currently assigned (if any).  But since I'm subscribed to
the patches list, I see all changes to all items regardless, and
wouldn't know whether the to-submitter and to-assignee gimmicks
stopped working.  I'm pretty sure that at least the to-submitter part
is working, since I've gotten quick replies from infrequent submitters
after adding a comment to their report.
From fumanchu at amor.org  Wed Aug 25 07:30:44 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Wed Aug 25 07:36:12 2004
Subject: [Python-Dev] possible CO_FUTURE_DECORATORS
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E65@exchange.hqamor.amorhq.net>

[Working on the J2 patch]...

If we use a keyword, we need a __future__ and AFAICT a new
CO_FUTURE_DECORATORS flag. Should it just be 0x4000? 

#define CO_NOFREE       0x0040
/* XXX Temporary hack.  Until generators are a permanent part of the
   language, we need a way for a code object to record that generators
   were *possible* when it was compiled.  This is so code dynamically
   compiled *by* a code object knows whether to allow yield stmts.  In
   effect, this passes on the "from __future__ import generators" state
   in effect when the code block was compiled. */
#define CO_GENERATOR_ALLOWED    0x1000 /* no longer used in an essential
way */
#define CO_FUTURE_DIVISION    	0x2000
#define CO_FUTURE_DECORATORS    	0x4000


I'm mostly wondering why there's a big gap between CO_NOFREE and
CO_GENERATOR_ALLOWED.

Or is it a non-issue, pick-what-you-want?


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From martin at v.loewis.de  Wed Aug 25 07:54:03 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 25 07:54:05 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412BA1C5.5050808@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>
	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>
	<412A5693.4050008@livinglogic.de>
	<412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com>
	<412BA1C5.5050808@livinglogic.de>
Message-ID: <412C297B.2040208@v.loewis.de>

Walter D?rwald wrote:
> If a tuple is passed and returned this makes it possible
> from Python code to mangle the state. IMHO this should be
> avoided if possible.

Not necessarily. We are all consenting adults. So if the
code checks whether the state is sane, from a typing point
of view, i.e. if you can't crash the interpreter, then there
is no need to hide the state from the user.

Regards,
Martin
From martin at v.loewis.de  Wed Aug 25 08:29:34 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 25 08:29:34 2004
Subject: [Python-Dev] socketmodule build failures on Un*xs that don't
	HAVE_GETADDRINFO
In-Reply-To: <20040824171634.A8031@ActiveState.com>
References: <20040824171634.A8031@ActiveState.com>
Message-ID: <412C31CE.5060306@v.loewis.de>

Trent Mick wrote:
> The problem seems to stem from this check-in which resulted in
> Modules/addrinfo.h *not* getting included for Unix flavours that do not
> HAVE_GETADDRINFO.

Thanks for discovering that. I have backed-out this change, and all
subsequent changes that have tried to work around, and re-opened the
patch.

Regards,
Martin
From martin at v.loewis.de  Wed Aug 25 08:33:06 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 25 08:33:04 2004
Subject: [Python-Dev] Is something wrong with SF patch notifications?
In-Reply-To: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer>
References: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer>
Message-ID: <412C32A2.6090602@v.loewis.de>

Raymond Hettinger wrote:
> IIRC, we always used to get notes whenever we were assigned a patch or 
> had participated in the patch discussions.

I just got mail when I reopened a patch, and added a comment.

Regards,
Martin
From theller at python.net  Wed Aug 25 09:12:04 2004
From: theller at python.net (Thomas Heller)
Date: Wed Aug 25 09:18:45 2004
Subject: [Python-Dev]  FW: [Python-checkins] python/nondist/peps
	pep-0318.txt, 1.25, 1.26
In-Reply-To: <004b01c48a29$2b6d6ba0$04f9cc97@oemcomputer> (Raymond
	Hettinger's message of "Tue, 24 Aug 2004 18:24:49 -0400")
References: <004b01c48a29$2b6d6ba0$04f9cc97@oemcomputer>
Message-ID: <wtznu1x7.fsf@python.net>

"Raymond Hettinger" <python@rcn.com> writes:

[decorators...]
> Like metaclasses, everyone knew they were powerful when they were put
> in, but no one knew how they would be used or whether they were
> necessary.  In fact, two versions later, we still don't know those
> answers.

Sorry I have to say this, but I don't think you know what you're talking
about in this paragraph.

Thomas

From anthony at interlink.com.au  Wed Aug 25 09:48:20 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Aug 25 09:48:31 2004
Subject: [Python-Dev] Is something wrong with SF patch notifications?
In-Reply-To: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer>
References: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer>
Message-ID: <412C4444.7080001@interlink.com.au>

Raymond Hettinger wrote:
> IIRC, we always used to get notes whenever we were assigned a patch or 
> had participated in the patch discussions.

I've found SF's behaviour to be inconsistent at best in this regard -
the only way to get the messages reliably is to subscribe to both the
python-bugs and python-patches lists. (As an aside: Why do we keep both
these lists going? Why not simply point all of the trackers at
python-bugs. I can see no reason why you'd want to know about only one
of the trackers...)
From mal at egenix.com  Wed Aug 25 10:32:41 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 25 10:32:44 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412C297B.2040208@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>
	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>
	<412A5693.4050008@livinglogic.de>	<412ACE79.4060605@v.loewis.de>
	<412B0C2D.7030507@egenix.com>	<412BA1C5.5050808@livinglogic.de>
	<412C297B.2040208@v.loewis.de>
Message-ID: <412C4EA9.8010003@egenix.com>

Martin v. L?wis wrote:
> Walter D?rwald wrote:
> 
>> If a tuple is passed and returned this makes it possible
>> from Python code to mangle the state. IMHO this should be
>> avoided if possible.
> 
> Not necessarily. We are all consenting adults. So if the
> code checks whether the state is sane, from a typing point
> of view, i.e. if you can't crash the interpreter, then there
> is no need to hide the state from the user.

Martin, there are two reasons for hiding away these details:

1. we need to be able to change the codec state without
    breaking the APIs

2. we don't want the state to be altered by the user

A single object serves this best and does not create
a whole plethora of new APIs in the _codecs module.
This is not over-design, but serves a reason.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 25 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Wed Aug 25 10:41:58 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Aug 25 10:42:01 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412BA1C5.5050808@livinglogic.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>
	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>
	<412A5693.4050008@livinglogic.de>	<412ACE79.4060605@v.loewis.de>
	<412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de>
Message-ID: <412C50D6.6010801@egenix.com>

Walter D?rwald wrote:
> M.-A. Lemburg wrote:
> 
>> Martin v. L?wis wrote:
>>
>>> Walter D?rwald wrote:
>>>
>>>> OK, let's come up with a patch that fixes the incomplete byte
>>>> sequences problem and then discuss non-stream APIs.
>>>>
>>>> So, what should the next step be?
>>>
>>>
>>> I think your first patch should be taken as a basis for that.
>>
>>
>> We do need a way to communicate state between the codec
>> and Python.
>>
>> However, I don't like the way that the patch
>> implements this state handling: I think we should use a
>> generic "state" object here which is passed to the stateful
>> codec and returned together with the standard return values
>> on output:
>>
>> def decode_stateful(data, state=None):
>>     ... decode and modify state ...
>>     return (decoded_data, length_consumed, state)
> 
> Another option might be that the decode function changes
> the state object in place.

Good idea.

>> where the object type and contents of the state variable
>> is defined per codec (e.g. could be a tuple, just a single
>> integer or some other special object).
> 
> If a tuple is passed and returned this makes it possible
> from Python code to mangle the state. IMHO this should be
> avoided if possible.
 >
>> Otherwise we'll end up having different interface
>> signatures for all codecs and extending them to accomodate
>> for future enhancement will become unfeasable without
>> introducing yet another set of APIs.
> 
> We already have slightly different decoding functions:
> utf_16_ex_decode() takes additional arguments.

Right - it was a step in the wrong direction. Let's not
use a different path for the future.

>> Let's discuss this some more and implement it for Python 2.5.
>> For Python 2.4, I think we can get away with what we already
>> have:
> 
>  > [...]
> 
> OK, I've updated the patch.
> 
>> [...]
>> The buffer logic should only be used for streams
>> that do not support the interface to push back already
>> read bytes (e.g. .unread()).
>>
>>  From a design perspective, keeping read data inside the
>> codec is the wrong thing to do, simply because it leaves
>> the input stream in an undefined state in case of an error
>> and there's no way to correlate the stream's read position
>> to the location of the error.
>>
>> With a pushback method on the stream, all the stream
>> data will be stored on the stream, not the codec, so
>> the above would no longer be a problem.
> 
> On the other hand this requires special stream. Data
> already read is part of the codec state, so why not
> put it into the codec?

Ideally, the codec should not store data, but only
reference it. It's better to keep things well
separated which is why I think we need the .unread()
interface and eventually a queue interface to support
the feeding operation.

>> However, we can always add the .unread() support to the
>> stream codecs at a later stage, so it's probably ok
>> to default to the buffer logic for Python 2.4.
> 
> OK.
> 
>>> That still leaves the issue
>>> of the last read operation, which I'm tempted to leave unresolved
>>> for Python 2.4. No matter what the solution is, it would likely
>>> require changes to all codecs, which is not good.
>>
>>
>> We could have a method on the codec which checks whether
>> the codec buffer or the stream still has pending data
>> left. Using this method is an application scope consideration,
>> not a codec issue.
> 
> But this mean that the normal error handling can't be used
> for those trailing bytes.

Right, but then: missing data (which usually causes the trailing
bytes) is really something for the application to deal with,
e.g. by requesting more data from the user, another application
or trying to work around the problem in some way. I don't think
that the codec error handler can practically cover these
possibilities.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 25 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From vinay_sajip at yahoo.co.uk  Wed Aug 25 11:43:16 2004
From: vinay_sajip at yahoo.co.uk (=?iso-8859-1?q?Vinay=20Sajip?=)
Date: Wed Aug 25 11:43:19 2004
Subject: [Python-Dev] Proposed change to logging
In-Reply-To: <200408250347.i7P3l4O21288@guido.python.org>
Message-ID: <20040825094316.59895.qmail@web25406.mail.ukl.yahoo.com>

 --- Guido van Rossum <guido@python.org> wrote: 
> > print >> err, "My logging message with %s" %
> > "arguments"
> > print >> dbg, "A message at %s level" % "debug"
> 
> Is this really a useful improvement?  It seems to
> save a few
> keystrokes at most.  TOOWTDI etc.

My personal vote would be +0, I posted the suggestion
only because some people have raised concerns that the
logging module is hard to use, and using "print" might
seem easier for newbies or casual users. Before
checking in, I want to see if I can remove that
implementation wart I mentioned. I feel the solution
Barry suggested is not as elegant as I'd like to have.

Vinay Sajip


	
	
		
___________________________________________________________ALL-NEW Yahoo! Messenger - all new features - even more fun!  http://uk.messenger.yahoo.com
From mwh at python.net  Wed Aug 25 12:02:01 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Aug 25 12:02:03 2004
Subject: [Python-Dev] Is something wrong with SF patch notifications?
In-Reply-To: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer> (Raymond
	Hettinger's message of "Wed, 25 Aug 2004 00:11:04 -0400")
References: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer>
Message-ID: <2m65777cyu.fsf@starship.python.net>

"Raymond Hettinger" <raymond.hettinger@verizon.net> writes:

> Link: File-List
>
> I don't think Barry got notes showing that I reviewed his patch.
>
> Also, I didn't get a note when Walter assigned a unittest patch to me.

I've failed to get the "personal" notifications (e.g. I don't think I
got a notification when you unassigned the peepholer patch from me).

OTOH, I'm not aware of the python.org lists missing anything.

Filing a SF support request may or may not help (it did last time
something went a bit wrong in this respect).

Cheers,
mwh

-- 
  All programs evolve until they can send email.      -- Richard Letts
  Except Microsoft Exchange.                                    -- Art
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
From mwh at python.net  Wed Aug 25 12:44:48 2004
From: mwh at python.net (Michael Hudson)
Date: Wed Aug 25 12:44:50 2004
Subject: [Python-Dev] socketmodule build failures on Un*xs that don't
	HAVE_GETADDRINFO
In-Reply-To: <20040824171634.A8031@ActiveState.com> (Trent Mick's message of
	"Tue, 24 Aug 2004 17:16:34 -0700")
References: <20040824171634.A8031@ActiveState.com>
Message-ID: <2m1xhv7azj.fsf@starship.python.net>

Trent Mick <trentm@ActiveState.com> writes:

>     http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.288&r2=1.289
>     Wed May 26 17:06:31 2004 UTC (2 months, 4 weeks ago) by mwh 
>     > Band-aid type fix for
>     > 
>     > [ 728330 ] Don't define _SGAPI on IRIX
>     > 
>     > The Right Thing would be nice, for now this'll do.  At least it
>     > isn't going to break anything *other* than IRIX...
>
> Relevant bug:
> https://sourceforge.net/tracker/?func=detail&atid=105470&aid=728330&group_id=5470
>
> Subseqent checkins have gone in to fix the lack of addrinfo.h import on
> some platforms:
>
>     http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.292&r2=1.293
>     http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/socketmodule.c?r1=1.297&r2=1.298
>
> So the question: was that original change removing the 
>     #include "addrinfo.h"
> an accident? I.e. should it have been something like this instead?
>
>     #if defined(__sgi)
>     # if _COMPILER_VERSION>700 \
>       && !defined(_SS_ALIGNSIZE) /* defined in sys/socket.h            */
>                                  /* by some newer versions of IRIX     */
>                                  /* (e.g. not by 6.5.10 but by 6.5.21) */
>     #  include "addrinfo.h"
>     # endif
>     #else
>     # include "addrinfo.h"
>     #endif

Well, it certainly seems I goofed with that checkin: as the message
implies, I thought the code I was touching only affected Irix.  I seem
to have mentally entirely paged out the details, though.

Sorry for all the pain.

This hilights what I said in the #ifdeffery thread, though: it's
difficult to check in code that affects systems you have no access to.
Thanks to Trent for taking the time to actually try to work out what's
going on!

Cheers,
mwh

-- 
  About the use of language: it is impossible to sharpen a
  pencil with a blunt axe.  It is equally vain to try to do
  it with ten blunt axes instead.
      -- E.W.Dijkstra, 18th June 1975. Perl did not exist at the time.
From skip at pobox.com  Wed Aug 25 13:38:48 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed Aug 25 13:38:58 2004
Subject: [Python-Dev] Proposed change to logging
In-Reply-To: <20040825094316.59895.qmail@web25406.mail.ukl.yahoo.com>
References: <200408250347.i7P3l4O21288@guido.python.org>
	<20040825094316.59895.qmail@web25406.mail.ukl.yahoo.com>
Message-ID: <16684.31304.985278.482389@montanaro.dyndns.org>


    Vinay> I posted the suggestion only because some people have raised
    Vinay> concerns that the logging module is hard to use, and using
    Vinay> "print" might seem easier for newbies or casual users.

My personal head scratching about the logging package has more to do with
initialization of the system than with emission of log info.  I wouldn't
worry about the function call vs. print thing, though if it was there I'd
probably use it.

Skip
From barry at python.org  Wed Aug 25 14:17:43 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 25 14:17:17 2004
Subject: [Python-Dev] Proposed change to logging
In-Reply-To: <200408250347.i7P3l4O21288@guido.python.org>
References: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com>
	<200408250347.i7P3l4O21288@guido.python.org>
Message-ID: <1093436263.4453.55.camel@geddy.wooz.org>

On Tue, 2004-08-24 at 23:47, Guido van Rossum wrote:

> Is this really a useful improvement?  It seems to save a few
> keystrokes at most.  TOOWTDI etc.

I'm really more interested in providing a compatible file-like (i.e.
write()) interface to logging objects.  There are a lot of APIs in the
world that are written to accept file-like objects and it would be nice
to be able to pass logging objects directly to them, without having to
first make a detour with StringIO.  I don't really care too much about
the actual implementation of it all.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040825/5a23affe/attachment.pgp
From barry at python.org  Wed Aug 25 14:33:06 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Aug 25 14:32:40 2004
Subject: [Python-Dev] Is something wrong with SF patch notifications?
In-Reply-To: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer>
References: <001a01c48a59$8aa1d2c0$04f9cc97@oemcomputer>
Message-ID: <1093437186.4453.63.camel@geddy.wooz.org>

On Wed, 2004-08-25 at 00:11, Raymond Hettinger wrote:
> I don?t think Barry got notes showing that I reviewed his patch.

I didn't, but I'm not sure it's SF's fault.  It's possible that such a
message got caught by my Spambayes filters, however I recently built out
a new mail server and my filters were pointing at imap folders on the
old server.  So any Unsures or Spam I got in the last week or so were
bit bucketed.  I definitely didn't see them in my regular inbox.

I also get patches in a digest and tend to ignore them when swamped.

But I will review your comments on the PEP 292 patch later today.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040825/ed2b5a96/attachment.pgp
From guido at python.org  Wed Aug 25 17:32:42 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug 25 17:32:53 2004
Subject: [Python-Dev] Proposed change to logging
In-Reply-To: Your message of "Wed, 25 Aug 2004 10:43:16 BST."
	<20040825094316.59895.qmail@web25406.mail.ukl.yahoo.com> 
References: <20040825094316.59895.qmail@web25406.mail.ukl.yahoo.com> 
Message-ID: <200408251532.i7PFWgX22320@guido.python.org>

> > > print >> err, "My logging message with %s" % "arguments"
> > > print >> dbg, "A message at %s level" % "debug"
> > 
> > Is this really a useful improvement?  It seems to
> > save a few keystrokes at most.  TOOWTDI etc.
> 
> My personal vote would be +0, I posted the suggestion
> only because some people have raised concerns that the
> logging module is hard to use, and using "print" might
> seem easier for newbies or casual users. Before
> checking in, I want to see if I can remove that
> implementation wart I mentioned. I feel the solution
> Barry suggested is not as elegant as I'd like to have.

But the complexity of the logging module doesn't stem from the message
logging interface, which is actually very simple.

It comes from the configuration interface, and from the complexity of
having separate concepts of loggers, handlers, and formatters, and of
the unintuitive filter rules (often setting the logger's level doesn't
work because the handler's level is still wrong, or the other way
around).

All of that complexity was inherited from Java, BTW (I've run in
exactly those same problems at work with the Java logging package).

I'm -0.5 on the file interface -- passing loggers around actually
becomes more complicated because you have to pass around separate info
and debug streams (etc.).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Aug 25 17:34:40 2004
From: guido at python.org (Guido van Rossum)
Date: Wed Aug 25 17:34:44 2004
Subject: [Python-Dev] Proposed change to logging
In-Reply-To: Your message of "Wed, 25 Aug 2004 08:17:43 EDT."
	<1093436263.4453.55.camel@geddy.wooz.org> 
References: <20040824135718.47678.qmail@web25406.mail.ukl.yahoo.com>
	<200408250347.i7P3l4O21288@guido.python.org> 
	<1093436263.4453.55.camel@geddy.wooz.org> 
Message-ID: <200408251534.i7PFYeI22352@guido.python.org>

> I'm really more interested in providing a compatible file-like (i.e.
> write()) interface to logging objects.  There are a lot of APIs in the
> world that are written to accept file-like objects and it would be nice
> to be able to pass logging objects directly to them, without having to
> first make a detour with StringIO.  I don't really care too much about
> the actual implementation of it all.

Hm.  So would you turn each separate line into a log message?  When
you write a partial line, it needs to be buffered somewhere until you
see the newline.

--Guido van Rossum (home page: http://www.python.org/~guido/)
From pje at telecommunity.com  Wed Aug 25 18:13:46 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Aug 25 18:13:46 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <200408250251.i7P2pMGI004207@cosc353.cosc.canterbury.ac.nz>
References: <5.1.1.6.0.20040824071549.02d407c0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040825004418.02a43d00@mail.telecommunity.com>

At 02:51 PM 8/25/04 +1200, Greg Ewing wrote:
>"Phillip J. Eby" <pje@telecommunity.com>:
>
> > If you could throw a special kind of exception (or even a regular
> > exception), and call traceback.resume() to pick up execution at the
> > point where it was thrown, whether thrown by a generator or a
> > regular function.
>
>Actually, it probably wouldn't be too hard to make exceptions
>resumable -- provided all the frames in between are Python. If the
>exception gets thrown through any C calls, though, resumption becomes
>impossible.  Some other structure is needed to hold the state of a
>resumable C call.

Unfortunately, as was discussed on the previous Stackless thread, nearly 
*all* Python bytecodes pass through C on their way to other Python code.

The trick is to be able to figure out whether the return value of a given 
frame is being put onto the value stack of its parent frame.  In principle, 
you could tell this from the operation being performed at the current 
opcode pointer in the parent frame, and the prior contents of the value 
stack.  In practice, the eval loop sometimes messes with the value stack 
prior to the call (e.g. to optimize method calls), and in the case of e.g 
try/finally and try/except, it's going to execute other stuff in the frame 
before the exception is passed up to the next frame.

So, to make it work,the interpreter loop would need to be particular about 
putting the stack back to the way it was before the current instruction 
began, when an exception occurs.  Also, it would need to *not* invoke 
except/finally clauses for one of these special exceptions, or else have a 
way of restoring the frame to its pre-exception state, which seems 
arbitrarily complex.  I guess it would have to treat the resumable 
exception as if a YIELD operation took place.

I'm going to have to think about this one some more, but it seems to me 
that it would be much easier to add bidirectional communication to 
generators to create a coroutine type, than to try to implement resumable 
exceptions.

From cce at clarkevans.com  Wed Aug 25 19:16:54 2004
From: cce at clarkevans.com (Clark C. Evans)
Date: Wed Aug 25 19:16:59 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <5.1.1.6.0.20040825004418.02a43d00@mail.telecommunity.com>
References: <5.1.1.6.0.20040824071549.02d407c0@mail.telecommunity.com>
	<5.1.1.6.0.20040825004418.02a43d00@mail.telecommunity.com>
Message-ID: <20040825171653.GA90924@prometheusresearch.com>

I'm not tied to my proposal, I'm suggesting that if the scope of
continuations is limited to only work with iterators, that a solution
may emerge.  This limitation, in pratice with generator syntax, is not
all that bothersome to an application developer.

On Wed, Aug 25, 2004 at 12:13:46PM -0400, Phillip J. Eby wrote:
| At 02:51 PM 8/25/04 +1200, Greg Ewing wrote:
| >> If you could throw a special kind of exception (or even a regular
| >> exception), and call traceback.resume() to pick up execution at the
| >> point where it was thrown, whether thrown by a generator or a
| >> regular function.
| >
| >Actually, it probably wouldn't be too hard to make exceptions
| >resumable -- provided all the frames in between are Python. If the
| >exception gets thrown through any C calls, though, resumption becomes
| >impossible.  Some other structure is needed to hold the state of a
| >resumable C call.
| 
| Unfortunately, as was discussed on the previous Stackless thread, nearly 
| *all* Python bytecodes pass through C on their way to other Python code.

However, the C code in the distribution could be scrubbed to make sure
that it works with the mechanism.  And the C API could be detailed so
that this case is managable; else, it is an "uncaught" exception.

| I'm going to have to think about this one some more, but it seems to me 
| that it would be much easier to add bidirectional communication to 
| generators to create a coroutine type, than to try to implement resumable 
| exceptions.

Sure; but resumable exceptions would only need to operate with generators,
and there are some simple rewrite tricks one could do.  If an iterator is
calling a generator it could be re-written in a form that maintains state
and _doesn't_ catch the exception.   Already for generators 'finally'
blocks are troublesome, so this doesn't change things for the worse.
StopIteration has many of these same issues.

On Wed, Aug 25, 2004 at 03:32:28PM +1200, Greg Ewing wrote:
| > You'd definately want to mix them.  For example, a 'cooperator'
| > would be reading from a socket, it could do one of three things:
| > 
| >   yield the next line from the sender
| >   raise StopIteration if the socket closed
| >   raise SuspendIteration if the read() is blocked
| 
| Hmmm. It seems that generation and cooperation are really orthogonal
| concepts -- you may want something that is both a generator *and* a
| cooperator. My proposal doesn't allow for this. I will have to do some
| more thinking...

Not only othogonal, but quite complementary.

Kind Regards,

Clark

-- 
Clark C. Evans                      Prometheus Research, LLC.
                                    http://www.prometheusresearch.com/
    o                               office: +1.203.777.2550 
  ~/ ,                              mobile: +1.203.444.0557 
 //
((   Prometheus Research: Transforming Data Into Knowledge
 \\  ,
   \/    - Research Exchange Database
   /\    - Survey & Assessment Technologies
   ` \   - Software Tools for Researchers
    ~ *
From martin at v.loewis.de  Wed Aug 25 21:56:25 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Aug 25 21:56:26 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412C4EA9.8010003@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>
	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>
	<412A5693.4050008@livinglogic.de>	<412ACE79.4060605@v.loewis.de>
	<412B0C2D.7030507@egenix.com>	<412BA1C5.5050808@livinglogic.de>
	<412C297B.2040208@v.loewis.de> <412C4EA9.8010003@egenix.com>
Message-ID: <412CEEE9.60207@v.loewis.de>

M.-A. Lemburg wrote:
> Martin, there are two reasons for hiding away these details:
> 
> 1. we need to be able to change the codec state without
>    breaking the APIs

That will be possible with the currently-proposed patch.
The _codecs methods are not public API, so changing them
would not be an API change.

> 2. we don't want the state to be altered by the user

We are all consenting adults, and we can't *really*
prevent it, anyway. For example, the user may pass an
old state, or a state originating from a different codec
(instance). We need to support this gracefully (i.e. with
a proper Python exception).

> A single object serves this best and does not create
> a whole plethora of new APIs in the _codecs module.
> This is not over-design, but serves a reason.

It does put a burden on codec developers, which need
to match the "official" state representation policy.
Of course, if they are allowed to return a tuple
representing their state, that would be fine with
me.

Regards,
Martin
From martin at v.loewis.de  Wed Aug 25 22:02:56 2004
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed Aug 25 22:02:53 2004
Subject: [Python-Dev] socketmodule build failures on Un*xs that don't
	HAVE_GETADDRINFO
In-Reply-To: <2m1xhv7azj.fsf@starship.python.net>
References: <20040824171634.A8031@ActiveState.com>
	<2m1xhv7azj.fsf@starship.python.net>
Message-ID: <412CF070.6040502@v.loewis.de>

Michael Hudson wrote:
> Well, it certainly seems I goofed with that checkin: as the message
> implies, I thought the code I was touching only affected Irix.  I seem
> to have mentally entirely paged out the details, though.
> 
> Sorry for all the pain.

No need to worry. This happens, and CVS is there for tracing things
back, and undoing/redoing them. Of course, anybody involved could
have done the analysis that Trent did (and yes, that includes yours
truly).

Regards,
Martin
From ncoghlan at iinet.net.au  Wed Aug 25 08:58:42 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Aug 26 00:20:24 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test
	test_string.py, 1.25, 1.26
In-Reply-To: <412B0C84.30204@livinglogic.de>
References: <E1BzOAO-0005Hx-Qc@sc8-pr-cvs1.sourceforge.net>
	<412B0C84.30204@livinglogic.de>
Message-ID: <412C38A2.9000909@iinet.net.au>

Walter D?rwald wrote:
> test_string.py is supposed to test the string module. This new test
> should be moved to test_str.py (or even better to string_test.py,
> so that it can be reused for doing the same test with unicode objects).

Ah, I think I see where the tests should have gone 
(MixinStrUnicodeUserStringTest.test_join in string_test.py).

I'll work up a patch to bring those tests in sync with the rest of the 
string tests (the structure of which I find harder to follow than the 
string code itself. . .). However, I probably won't get to it for a week 
or so.

Cheers,
Nick.

From djc at object-craft.com.au  Thu Aug 26 02:44:07 2004
From: djc at object-craft.com.au (Dave Cole)
Date: Thu Aug 26 02:44:13 2004
Subject: [Python-Dev] Submitted patch #1014930 to exposes the current
	parse location in pyexpat
In-Reply-To: <200408240307.38486.fdrake@acm.org>
References: <412AAD92.9010009@object-craft.com.au>
	<200408240307.38486.fdrake@acm.org>
Message-ID: <412D3257.4060707@object-craft.com.au>

Fred L. Drake, Jr. wrote:
> On Monday 23 August 2004 10:53 pm, Dave Cole wrote:
>  > Expat supplies the XML_GetCurrentLineNumber, XML_GetCurrentColumnNumber,
>  > and XML_GetCurrentByteIndex functions to obtain the current parse
>  > location.  The patch exposes these as the following respective members
>  > of the xml.parsers.expat.XMLParser object; CurrentLineNumber,
>  > CurrentColumnNumber, CurrentByteIndex.
> 
> This sounds like a good idea to me.  Please post the patch on SF and assign to 
> me.

I have committed the patch after adding the suggestions from the reviewer.

- Dave

-- 
http://www.object-craft.com.au
From pje at telecommunity.com  Thu Aug 26 03:01:47 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Aug 26 03:01:53 2004
Subject: [Python-Dev] Simple coroutines?
In-Reply-To: <20040825171653.GA90924@prometheusresearch.com>
References: <5.1.1.6.0.20040825004418.02a43d00@mail.telecommunity.com>
	<5.1.1.6.0.20040824071549.02d407c0@mail.telecommunity.com>
	<5.1.1.6.0.20040825004418.02a43d00@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040825205040.02a6cb10@mail.telecommunity.com>

At 01:16 PM 8/25/04 -0400, Clark C. Evans wrote:
>On Wed, Aug 25, 2004 at 12:13:46PM -0400, Phillip J. Eby wrote:
>| At 02:51 PM 8/25/04 +1200, Greg Ewing wrote:
>| >> If you could throw a special kind of exception (or even a regular
>| >> exception), and call traceback.resume() to pick up execution at the
>| >> point where it was thrown, whether thrown by a generator or a
>| >> regular function.
>| >
>| >Actually, it probably wouldn't be too hard to make exceptions
>| >resumable -- provided all the frames in between are Python. If the
>| >exception gets thrown through any C calls, though, resumption becomes
>| >impossible.  Some other structure is needed to hold the state of a
>| >resumable C call.
>|
>| Unfortunately, as was discussed on the previous Stackless thread, nearly
>| *all* Python bytecodes pass through C on their way to other Python code.
>
>However, the C code in the distribution could be scrubbed to make sure
>that it works with the mechanism.  And the C API could be detailed so
>that this case is managable; else, it is an "uncaught" exception.

Quick question: does Stackless' "greenlet" extension module do what you 
need?  I just built it and played around with it a bit.  One of the demos 
is a pseudo-generator implementation using a 'Yield' function that uses 
greenlet task switching to simulate the behavior of a normal generator 
function.

It looks to me that your "yield Cooperate" scenarios could perhaps be met 
by simply switching to a reactor/mainloop greenlet, when it's necessary to 
wait for an event, and by having your event callback switch back to the 
task's greenlet, returning a result if needed.

Interestingly, this greenlet thing looks like it would be perfect for 
peak.events, allowing me to get rid of all the generators and yield/resume 
stuff.  (In fairness, Bob Ippolito told me that before I ever implemented 
peak.events, but at the time greenlets didn't exist as an extension module 
that could be built outside of Stackless.)

From tim.peters at gmail.com  Thu Aug 26 04:07:27 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 26 04:07:29 2004
Subject: [Python-Dev] test_distutils failing in 3 ways today
Message-ID: <1f7befae0408251907b6d1a2f@mail.gmail.com>

This is on Windows.  I've heard that it fails on Linux too.  Who isn't
running the test suite?  A reminder:  with lots of people trying to
make changes at once, it's essential to run all tests before checking
in.

C:\Code\python\PCbuild>python ../lib/test/test_distutils.py
test_package_data (distutils.tests.test_build_py.BuildPyTestCase) ... ok
test_build (distutils.tests.test_build_scripts.BuildScriptsTestCase) ... ERROR
test_default_settings
(distutils.tests.test_build_scripts.BuildScriptsTestCase) ... ERROR
test_command_packages_cmdline
(distutils.tests.test_dist.DistributionTestCase) ... ok
test_command_packages_configfile
(distutils.tests.test_dist.DistributionTestCase) ... ok
test_command_packages_unspecified
(distutils.tests.test_dist.DistributionTestCase) ... ok
test_home_installation_scheme
(distutils.tests.test_install.InstallTestCase) ... ok
test_default_settings
(distutils.tests.test_install_scripts.InstallScriptsTestCase) ... ok
test_installation
(distutils.tests.test_install_scripts.InstallScriptsTestCase) ... ok

======================================================================
ERROR: test_build (distutils.tests.test_build_scripts.BuildScriptsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\python\lib\distutils\tests\test_build_scripts.py",
line 34, in test_build
    cmd.finalize_options()
  File "C:\Code\python\lib\distutils\command\build_scripts.py", line
44, in finalize_options
    ('executable', 'executable'))
  File "C:\Code\python\lib\distutils\cmd.py", line 309, in set_undefined_options
    getattr(src_cmd_obj, src_option))
AttributeError: DummyCommand instance has no attribute 'executable'

======================================================================
ERROR: test_default_settings
(distutils.tests.test_build_scripts.BuildScriptsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\python\lib\distutils\tests\test_build_scripts.py",
line 21, in test_default_settings
    cmd.finalize_options()
  File "C:\Code\python\lib\distutils\command\build_scripts.py", line
44, in finalize_options
    ('executable', 'executable'))
  File "C:\Code\python\lib\distutils\cmd.py", line 309, in set_undefined_options
    getattr(src_cmd_obj, src_option))
AttributeError: DummyCommand instance has no attribute 'executable'

----------------------------------------------------------------------
Ran 9 tests in 0.156s

FAILED (errors=2)
Traceback (most recent call last):
  File "../lib/test/test_distutils.py", line 17, in ?
    test_main()
  File "../lib/test/test_distutils.py", line 13, in test_main
    test.test_support.run_unittest(distutils.tests.test_suite())
  File "C:\Code\python\lib\test\test_support.py", line 290, in run_unittest
    run_suite(suite, testclass)
  File "C:\Code\python\lib\test\test_support.py", line 272, in run_suite
    msg = "errors occurred in %s.%s" \
AttributeError: 'TestSuite' object has no attribute '__name__'

C:\Code\python\PCbuild>
From alex_nanou at cs.msu.su  Thu Aug 26 12:58:30 2004
From: alex_nanou at cs.msu.su (alex_nanou@cs.msu.su)
Date: Thu Aug 26 12:58:33 2004
Subject: [Python-Dev] decorators: If you go for it, go all the way!!! :)
Message-ID: <3405.212.192.249.122.1093517910.squirrel@212.192.249.122>

the inability to decorate anything but a function, is quite a limitation,
though I must agree that decorating classes is not such a common task, but
using the syntax to *decorate* attributes would indeed prove useful (in
both the readability department and the uniformity of code)..

here is an example we use quite often:
---cut---
import time
import sys

def intonly(obj=0, doc=None):
    '''
    intonly(int) -> descriptor

    the resulting descriptor will prevent both the deletion of the
    attribute and assignments of any value other than ints.
    '''
    if type(obj) is not int:
        raise TypeError, 'value must be int (got: %s)' % obj

    f_locals = sys._getframe(1).f_locals

    # name the attr (I know this is ugly ^_^ )
    while True:
        name = '_priv_' + str(int(time.time()*10**8))
        if name not in f_locals:
            f_locals[name] = obj
            break

    # the descriptor reader function...
    def read(self):
        return getattr(self, name)

    # the descriptor writer...
    def write(self, val):
        if type(val) is not int:
            raise TypeError, 'value must be int (got: %s)' % val
        setattr(self, name, val)

    # now create the descriptor and return it...
    return property(read, write, doc=doc)

--uncut--


and here are a couple of usage examples:
the desired decorator syntax:
---cut---
class A(object):
    @intonly
    i = 123

--uncut--

the current usage syntax:
---cut---
class A(object):
    i = intonly(123)

--uncut--

// sorry if I bring this up agen! :)
thanks!

---
                                           Best Regards...
                                                      Alex.

From s.percivall at chello.se  Thu Aug 26 13:50:22 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Thu Aug 26 13:50:27 2004
Subject: [Python-Dev] decorators: If you go for it, go all the way!!! :)
In-Reply-To: <3405.212.192.249.122.1093517910.squirrel@212.192.249.122>
References: <3405.212.192.249.122.1093517910.squirrel@212.192.249.122>
Message-ID: <1CBF291C-F756-11D8-A1B4-0003934AD54A@chello.se>

On 2004-08-26, at 12.58, alex_nanou@cs.msu.su wrote:
> the inability to decorate anything but a function, is quite a 
> limitation,
> though I must agree that decorating classes is not such a common task, 
> but
> using the syntax to *decorate* attributes would indeed prove useful (in
> both the readability department and the uniformity of code)..
> [...]
>
> and here are a couple of usage examples:
> the desired decorator syntax:
> ---cut---
> class A(object):
>     @intonly
>     i = 123
>
> --uncut--
>
> the current usage syntax:
> ---cut---
> class A(object):
>     i = intonly(123)
>
> --uncut--

IMHO, this would be _bad_.

As I see it, the need for a special syntax for decoration, in this case 
a
prefix syntax, comes mainly from the fact that method/function and class
declaration is just that, declarations. You can only pass the entity to 
a
decorating function after it has been bound. Hence the irritating need 
to
type the entity name three times, once declaring, once re-binding and 
once
passing to the decorating function. This just isn't an issue here. If 
you
create an instance of some type, such as the integer literal in your
example, you can, just as you show, pass it to a function directly. 
This is
truly the best way, and the decoration is unnecessary.

Also, I don't agree it increases readability. The @ character marks
something special, something you might really need to see to understand
what is defined with a function/method or class. That's why re-binding
that entity afterwards isn't good enough: you might miss it. Again,
this isn't an issue here.

//Simon

From oliver.walczak at momatec.de  Thu Aug 26 14:00:40 2004
From: oliver.walczak at momatec.de (Oliver Walczak)
Date: Thu Aug 26 14:00:43 2004
Subject: [Python-Dev] exit in __builtins__
Message-ID: <FB26DAD89E43D411B25F0050DA517F382E45DB@MOMATEC01>

Dear list,

as I am already quite experienced in python programming I recently started
to dive into the C-depths of the language. So it was a logical implication
to register for this newsgroup as there is no one else I can talk about
this.

So here's one thing I tried to evolve out of the sources but couldn't find
out how it works internally:

If you call dir(__builtins__) after starting a python (2.3.4) session you
get a list all well known builtin functions. I noticed, that there is
already an exit function available as builtin (NOT sys.exit!!). But when you
call exit(0) it says

TypeError: 'str' object is not callable

So I called

print __builtins__.exit.__doc__

And surprisingly I got the doc string from str. So I started browsing the
sources, and in ./Python/bltinmodule.c there get most of the builtin
functions attached to __builtin__, but not the "exit" call.

Is it a bug or a feature? exit as builtin is not documented anywhere and I
cannot find out where it gets included or attached to the str object in the
sources. Any idea?

Best regards
Oliver

From rodrigobamboo at gmail.com  Thu Aug 26 14:07:02 2004
From: rodrigobamboo at gmail.com (Rodrigo B. de Oliveira)
Date: Thu Aug 26 14:07:16 2004
Subject: [Python-Dev] exit in __builtins__
In-Reply-To: <FB26DAD89E43D411B25F0050DA517F382E45DB@MOMATEC01>
References: <FB26DAD89E43D411B25F0050DA517F382E45DB@MOMATEC01>
Message-ID: <5917478b0408260507707bfc56@mail.gmail.com>

Hi!

Try to type 'exit' or 'quit' in the interactive shell and you see its
purpose. Pretty clever.

Cheers,
Rodrigo

On Thu, 26 Aug 2004 14:00:40 +0200, Oliver Walczak
<oliver.walczak@momatec.de> wrote:
> Dear list,
> ...
> 
> 
> Is it a bug or a feature? exit as builtin is not documented anywhere and I
> cannot find out where it gets included or attached to the str object in the
> sources. Any idea?
> 
> Best regards
> Oliver
From adurdin at gmail.com  Thu Aug 26 14:23:09 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Thu Aug 26 14:23:17 2004
Subject: [Python-Dev] exit in __builtins__
In-Reply-To: <5917478b0408260507707bfc56@mail.gmail.com>
References: <FB26DAD89E43D411B25F0050DA517F382E45DB@MOMATEC01>
	<5917478b0408260507707bfc56@mail.gmail.com>
Message-ID: <59e9fd3a04082605237676146b@mail.gmail.com>

On Thu, 26 Aug 2004 09:07:02 -0300, Rodrigo B. de Oliveira
<rodrigobamboo@gmail.com> wrote:
> Hi!
> 
> Try to type 'exit' or 'quit' in the interactive shell and you see its
> purpose. Pretty clever.

Yes, it's the string that tells you the proper way to exit the interpreter.

But does this really belong in __builtins__? Wouldn't it be better for
the interactive interpreter to register it as a global when it sets
up, as it only makes sense in the context of the interactive
interpreter?

Not that it's a big deal :)
From oliver.walczak at momatec.de  Thu Aug 26 14:30:50 2004
From: oliver.walczak at momatec.de (Oliver Walczak)
Date: Thu Aug 26 14:30:52 2004
Subject: AW: [Python-Dev] exit in __builtins__
In-Reply-To: <FB26DAD89E43D411B25F0050DA517F3831816A@MOMATEC01>
Message-ID: <FB26DAD89E43D411B25F0050DA517F382E45DC@MOMATEC01>

Yes, it's "only" in site.py:

__builtin__.quit = __builtin__.exit = exit

Allright, I think it makes sense like it is :-)

Thanx
Oliver

-----Urspr?ngliche Nachricht-----
Von: Rodrigo B. de Oliveira [mailto:rodrigobamboo@gmail.com] 
Gesendet: Donnerstag, 26. August 2004 14:07
An: Oliver Walczak
Cc: python-dev@python.org
Betreff: Re: [Python-Dev] exit in __builtins__


Hi!

Try to type 'exit' or 'quit' in the interactive shell and you see its
purpose. Pretty clever.

Cheers,
Rodrigo

On Thu, 26 Aug 2004 14:00:40 +0200, Oliver Walczak
<oliver.walczak@momatec.de> wrote:
> Dear list,
> ...
> 
> 
> Is it a bug or a feature? exit as builtin is not documented anywhere and I
> cannot find out where it gets included or attached to the str object in
the
> sources. Any idea?
> 
> Best regards
> Oliver

From mcherm at mcherm.com  Thu Aug 26 14:38:35 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Aug 26 14:34:48 2004
Subject: [Python-Dev] exit in __builtins__
Message-ID: <1093523915.412dd9cb5a57f@mcherm.com>

Andrew Durdin writes:
> But does this really belong in __builtins__? Wouldn't it be better for
> the interactive interpreter to register it as a global when it sets
> up, as it only makes sense in the context of the interactive
> interpreter?

It's a basic principle that the behavior of the interactive interpreter
and the behavior of Python run non-interactively should be as close
to the same as is possible. The requirement for blank lines after an
indented suite is unavoidable because you can't "look ahead" in
the interpreter. There are also the prompts. While I'm sure there are a
few others, we try to keep the differences minimal.

-- Michael Chermside

From walter at livinglogic.de  Thu Aug 26 14:47:36 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 26 14:47:42 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Lib/test	test_string.py, 1.25, 1.26
In-Reply-To: <412C38A2.9000909@iinet.net.au>
References: <E1BzOAO-0005Hx-Qc@sc8-pr-cvs1.sourceforge.net>	<412B0C84.30204@livinglogic.de>
	<412C38A2.9000909@iinet.net.au>
Message-ID: <412DDBE8.2070206@livinglogic.de>

Nick Coghlan wrote:

> Walter D?rwald wrote:
> 
>> test_string.py is supposed to test the string module. This new test
>> should be moved to test_str.py (or even better to string_test.py,
>> so that it can be reused for doing the same test with unicode objects).
> 
> 
> Ah, I think I see where the tests should have gone 
> (MixinStrUnicodeUserStringTest.test_join in string_test.py).
> 
> I'll work up a patch to bring those tests in sync with the rest of the 
> string tests (the structure of which I find harder to follow than the 
> string code itself. . .). However, I probably won't get to it for a week 
> or so.

I'm working on it, however I discovered that unicode.join()
doesn't optimize this special case:

s = "foo"
assert "".join([s]) is s

u = u"foo"
assert u"".join([s]) is s

The second assertion fails.

I'd say that this test (joining a one item sequence returns
the item itself) should be removed because it tests an
implementation detail.

I'm not sure, whether the optimization should be added to
unicode.find().

Bye,
    Walter D?rwald



From pinard at iro.umontreal.ca  Thu Aug 26 15:54:01 2004
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Thu Aug 26 15:54:36 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <1093367647.18858.62.camel@geddy.wooz.org>
References: <1093232832.28573.16.camel@geddy.wooz.org>
	<20040823132043.GA4501@titan.progiciels-bpi.ca>
	<1093367647.18858.62.camel@geddy.wooz.org>
Message-ID: <20040826135401.GA31295@alcyon.progiciels-bpi.ca>

[Barry Warsaw]

> > If my feeling is right, then the PEP should clearly explicit this
> > goal [of pushing `$' forward], it will make the PEP stronger.

> I will neither confirm nor deny whether the PSU is bankrolling the PEP
> 292 initiative, nor the actual existence of any 527 organization
> claiming to be called the "PSU", nor whether if they did exist, they
> were or weren't acting in coordination with the campaign organizations
> of any 2004 US presidential nominee.

Nice joking.  Still, yet, and nevertheless, I think making the PEP
closer to the truth might make it stronger.  It might also repair a bit
the feeling that the PEP process is sometimes mildly lacking (the @-PEP
having been ratehr publicised as an example of this, recently).

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard
From dave at boost-consulting.com  Thu Aug 26 16:18:39 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Aug 26 16:18:57 2004
Subject: [Python-Dev] list += string??
Message-ID: <ur7pu2da8.fsf@boost-consulting.com>


I just discovered the following behavior:

  C:\Documents and Settings\dave>python
  Python 2.3 (#46, Aug 25 2003, 18:37:29) [MSC v.1200 32 bit (Intel)] on win32
  Type "help", "copyright", "credits" or "license" for more information.
  >>> prefix = ['foo', 'bar', 'baz']
  >>> prefix += 'bomb'  # meant to write prefix += [ 'bomb' ]
  >>> prefix
  ['foo', 'bar', 'baz', 'b', 'o', 'm', 'b']
  >>>

Is it new and/or intentional?  I would have expected an error from the
+=.  I was very surprised when my error "passed silently".

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From aahz at pythoncraft.com  Thu Aug 26 16:23:25 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Aug 26 16:23:43 2004
Subject: [Python-Dev] list += string??
In-Reply-To: <ur7pu2da8.fsf@boost-consulting.com>
References: <ur7pu2da8.fsf@boost-consulting.com>
Message-ID: <20040826142325.GA2855@panix.com>

On Thu, Aug 26, 2004, David Abrahams wrote:
> 
> I just discovered the following behavior:
> 
>   C:\Documents and Settings\dave>python
>   Python 2.3 (#46, Aug 25 2003, 18:37:29) [MSC v.1200 32 bit (Intel)] on win32
>   Type "help", "copyright", "credits" or "license" for more information.
>   >>> prefix = ['foo', 'bar', 'baz']
>   >>> prefix += 'bomb'  # meant to write prefix += [ 'bomb' ]
>   >>> prefix
>   ['foo', 'bar', 'baz', 'b', 'o', 'm', 'b']
>   >>>
> 
> Is it new and/or intentional?  I would have expected an error from the
> +=.  I was very surprised when my error "passed silently".

>>> l = ['foo']
>>> l += ('bar', 'baz')
>>> l
['foo', 'bar', 'baz']

Augmented assignment for lists works with any sequence type (and
probably iterators, too, but I didn't check).  Strings are a sequence
type, which occasionally has unintended consequences.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From tim.peters at gmail.com  Thu Aug 26 16:30:53 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 26 16:30:56 2004
Subject: [Python-Dev] list += string??
In-Reply-To: <ur7pu2da8.fsf@boost-consulting.com>
References: <ur7pu2da8.fsf@boost-consulting.com>
Message-ID: <1f7befae04082607303b7bc26e@mail.gmail.com>

[David Abrahams]
> I just discovered the following behavior:
>
>  C:\Documents and Settings\dave>python
>  Python 2.3 (#46, Aug 25 2003, 18:37:29) [MSC v.1200 32 bit (Intel)] on win32
>  Type "help", "copyright", "credits" or "license" for more information.
>  >>> prefix = ['foo', 'bar', 'baz']
>  >>> prefix += 'bomb'  # meant to write prefix += [ 'bomb' ]
>  >>> prefix
>  ['foo', 'bar', 'baz', 'b', 'o', 'm', 'b']
>  >>>
>
> Is it new and/or intentional?  I would have expected an error from the
> +=.  I was very surprised when my error "passed silently".

It was new when the iterator protocol was introduced.  It wasn't
explicitly intended that you get surprises for strings specifically,
but it was explicitly intended that

    list += whatever

work like

    list.extend(whatever)

and that whatever can be any iterable object.

>>> prefix = ['foo', 'bar', 'baz']
>>> def g():
...     for ch in "bomb":
...         yield ch
...
>>> prefix += g()
>>> prefix
['foo', 'bar', 'baz', 'b', 'o', 'm', 'b']
>>>

Since strings are iterable objects, it's no more or less surprising
than that 'for ch in "bomb"' works too <wink>.
From dave at boost-consulting.com  Thu Aug 26 16:58:26 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Aug 26 16:58:42 2004
Subject: [Python-Dev] list += string??
In-Reply-To: <1f7befae04082607303b7bc26e@mail.gmail.com> (Tim Peters's message
	of "Thu, 26 Aug 2004 10:30:53 -0400")
References: <ur7pu2da8.fsf@boost-consulting.com>
	<1f7befae04082607303b7bc26e@mail.gmail.com>
Message-ID: <u8yc22bfx.fsf@boost-consulting.com>

Tim Peters <tim.peters@gmail.com> writes:

> [David Abrahams]
>> I just discovered the following behavior:
>>
>>  C:\Documents and Settings\dave>python
>>  Python 2.3 (#46, Aug 25 2003, 18:37:29) [MSC v.1200 32 bit (Intel)] on win32
>>  Type "help", "copyright", "credits" or "license" for more information.
>>  >>> prefix = ['foo', 'bar', 'baz']
>>  >>> prefix += 'bomb'  # meant to write prefix += [ 'bomb' ]
>>  >>> prefix
>>  ['foo', 'bar', 'baz', 'b', 'o', 'm', 'b']
>>  >>>
>>
>> Is it new and/or intentional?  I would have expected an error from the
>> +=.  I was very surprised when my error "passed silently".
>
> It was new when the iterator protocol was introduced.  It wasn't
> explicitly intended that you get surprises for strings specifically,
> but it was explicitly intended that
>
>     list += whatever
>
> work like
>
>     list.extend(whatever)
>
> and that whatever can be any iterable object.


I figured as much.

> Since strings are iterable objects, it's no more or less surprising
> than that 'for ch in "bomb"' works too <wink>.

I want my ,= operator!

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From tismer at stackless.com  Thu Aug 26 17:12:46 2004
From: tismer at stackless.com (Christian Tismer)
Date: Thu Aug 26 17:12:58 2004
Subject: [Python-Dev] Minimal 'stackless' PEP using generators?
In-Reply-To: <20040823153923.GA50408@prometheusresearch.com>
References: <20040823153923.GA50408@prometheusresearch.com>
Message-ID: <412DFDEE.5000108@stackless.com>

Clark C. Evans wrote:

...

> With these two changes, the "lower" function could be an async reactor
> like those found in Twisted, or async core.  While the result isn't true
> coutines, it would be a huge improvement for those who would like to do
> async coding.   I've done something similar with Twisted called Flow [1]
> and it works well, with the exception of being a painful syntax hack and
> being quite slow.   If this was moved into Python's core, we'd get most
> of the advantages of coroutines without the full cost.
> 
> Thoughts?

Well, I just think "no".
Generators, as limited as they are in Python, make
some sense.
Coroutines for me have the advantage to make a context
switch. While generators are very often called in
a context where they even could be inlined, coroutines
should be really independent. But they are not independent
if you just cannot switch, because one of them just happens
to call a different function.
A typical use of coroutines is the situation where it
cannot be deduced who is caller or callee.

What I mean, those situations which can be solved with
a stack are the trivial cases, and that is exactly
*not* what Stackless is about.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Carmerstr. 2                 :    *Starship* http://starship.python.net/
10623 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 31 86 04 18  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/
From mal at egenix.com  Thu Aug 26 17:28:43 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 26 17:28:59 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412CEEE9.60207@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>	<412A5693.4050008@livinglogic.de>	<412ACE79.4060605@v.loewis.de>	<412B0C2D.7030507@egenix.com>	<412BA1C5.5050808@livinglogic.de>	<412C297B.2040208@v.loewis.de>
	<412C4EA9.8010003@egenix.com> <412CEEE9.60207@v.loewis.de>
Message-ID: <412E01AB.8030808@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
> 
>> Martin, there are two reasons for hiding away these details:
>>
>> 1. we need to be able to change the codec state without
>>    breaking the APIs
> 
> 
> That will be possible with the currently-proposed patch.
> The _codecs methods are not public API, so changing them
> would not be an API change.

Uhm, I wasn't talking about the builtin codecs only (of course,
we can change those to our liking). I'm after a generic
interface for stateful codecs.

>> 2. we don't want the state to be altered by the user
> 
> 
> We are all consenting adults, and we can't *really*
> prevent it, anyway. For example, the user may pass an
> old state, or a state originating from a different codec
> (instance). We need to support this gracefully (i.e. with
> a proper Python exception).

True, but the codec writer should be in control of the
state object, its format and what the user can or cannot
change.

>> A single object serves this best and does not create
>> a whole plethora of new APIs in the _codecs module.
>> This is not over-design, but serves a reason.
> 
> It does put a burden on codec developers, which need
> to match the "official" state representation policy.
> Of course, if they are allowed to return a tuple
> representing their state, that would be fine with
> me.

They can use any object they like to keep the state
in whatever format they choose. I think this makes it
easier on the codec writer, rather than harder.

Furthermore, they can change the way they store state
e.g. to accomodate for new features they may want to
add to the codec, without breaking the interface.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 26 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From tim.peters at gmail.com  Thu Aug 26 20:21:25 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 26 20:21:32 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test
	test_string.py, 1.25, 1.26
In-Reply-To: <412DDBE8.2070206@livinglogic.de>
References: <E1BzOAO-0005Hx-Qc@sc8-pr-cvs1.sourceforge.net>	<412B0C84.30204@livinglogic.de>
	<412C38A2.9000909@iinet.net.au> <412DDBE8.2070206@livinglogic.de>
Message-ID: <1f7befae040826112168b99b08@mail.gmail.com>

[Walter D?rwald]
> I'm working on it, however I discovered that unicode.join()
> doesn't optimize this special case:
>
> s = "foo"
> assert "".join([s]) is s
>
> u = u"foo"
> assert u"".join([s]) is s
> 
> The second assertion fails.

Well, in that example it *has* to fail, because the input (s) wasn't a
unicode string to begin with, but u"".join() must return a unicode
string.  Maybe you intended to say that

    assert u"".join([u]) is u

fails (which is also true today, but doesn't need to be true tomorrow).

> I'd say that this test (joining a one item sequence returns
> the item itself) should be removed because it tests an
> implementation detail.

Neverthess, it's an important pragmatic detail.  We should never throw
away a test just because rearrangement makes a test less convenient.

> I'm not sure, whether the optimization should be added to
> unicode.find().

Believing you mean join(), yes.  Doing common endcases efficiently in
C code is an important quality-of-implementation concern, lest people
need to add reams of optimization test-&-branch guesses in their own
Python code.  For example, the SpamBayes tokenizer has many passes
that split input strings on magical separators of one kind or another,
pasting the remaining pieces together again via string.join().  It's
explicitly noted in the code that special-casing the snot out of
"separator wasn't found" in Python is a lot slower than letting
string.join(single_element_list) just return the list element, so that
simple, uniform Python code works well in all cases.  It's expected
that *most* of these SB passes won't find the separator they're
looking for, and it's important not to make endless copies of
unboundedly large strings in the expected case.  The more heavily used
unicode strings become, the more important that they treat users
kindly in such cases too.
From mal at egenix.com  Thu Aug 26 20:29:13 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu Aug 26 20:29:16 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Lib/test	test_string.py, 1.25, 1.26
In-Reply-To: <1f7befae040826112168b99b08@mail.gmail.com>
References: <E1BzOAO-0005Hx-Qc@sc8-pr-cvs1.sourceforge.net>	<412B0C84.30204@livinglogic.de>	<412C38A2.9000909@iinet.net.au>
	<412DDBE8.2070206@livinglogic.de>
	<1f7befae040826112168b99b08@mail.gmail.com>
Message-ID: <412E2BF9.7030601@egenix.com>

Tim Peters wrote:
> [Walter D?rwald]
>>I'm not sure, whether the optimization should be added to
>>unicode.find().
> 
> Believing you mean join(), yes. 

+1

Unicode objects should behave as much as string objects as
possible.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 26 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From bac at OCF.Berkeley.EDU  Thu Aug 26 21:02:14 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Aug 26 21:02:22 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
Message-ID: <412E33B6.8080701@ocf.berkeley.edu>

test__locale has been failing on OS X for quite a while now (I bet ever 
since we started compiling against the CoreFoundation framework since 
that is where the problem exists).  I would  like to deal with it before 
a3 goes out in case we need to change something semantically.

Bug http://www.python.org./sf/1005113 has most of the details. 
Basically it looks like Apple locks down the locale so that it can't be 
change through locale.h .  This means that all attempts to change the 
locale just fails silently.  Obviously this sucks.  I do have a contact 
at Apple who said it looks like it the function that caused this issue 
has been removed from the tree they are working with for Tiger, but that 
can obviously change and they could also have just moved the 
functionality to another function.

There are a few options on how to deal with this.  One is to just turn 
off test__locale for OS X.  We do this for test_locale already so 
skipping locale tests on OS X already has a precedent.

We can raise locale.Error on OS X for all attempts to set the locale. 
Problem I have with this is I don't know if this problem exists in other 
versions of OS X beyond 10.3 (test__locale passing for anyone on other 
OS X versions?).

Last option is to try to fix it so that we set the locale through 
CoreFoundation.  I have no clue how doable this is (quick searching 
turned up 
http://developer.apple.com/documentation/CoreFoundation/Conceptual/CFLocales/index.html 
so it might be possible).  Problem is I know I can't get it in before 
a3.  Possibly before b1, but no guarantee.  And of course this is 
assuming this works the way we need it to which it might not.

Personally I would like to turn off the test on OS X for now and then 
try to see if there is a proper way to solve this.

-Brett
From walter at livinglogic.de  Thu Aug 26 21:54:34 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 26 21:54:52 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test
	test_string.py, 1.25, 1.26
In-Reply-To: <1f7befae040826112168b99b08@mail.gmail.com>
References: <E1BzOAO-0005Hx-Qc@sc8-pr-cvs1.sourceforge.net>	<412B0C84.30204@livinglogic.de>	<412C38A2.9000909@iinet.net.au>
	<412DDBE8.2070206@livinglogic.de>
	<1f7befae040826112168b99b08@mail.gmail.com>
Message-ID: <412E3FFA.5030002@livinglogic.de>

Tim Peters wrote:

> [Walter D?rwald]
> 
>>I'm working on it, however I discovered that unicode.join()
>>doesn't optimize this special case:
>>
>>s = "foo"
>>assert "".join([s]) is s
>>
>>u = u"foo"
>>assert u"".join([s]) is s
>>
>>The second assertion fails.
> 
> Well, in that example it *has* to fail, because the input (s) wasn't a
> unicode string to begin with, but u"".join() must return a unicode
> string.  Maybe you intended to say that
> 
>     assert u"".join([u]) is u
> 
> fails

Argl, you're right.

> (which is also true today, but doesn't need to be true tomorrow).

I've removed the test today, so it won't fail tomorrow. ;)

>>I'd say that this test (joining a one item sequence returns
>>the item itself) should be removed because it tests an
>>implementation detail.
> 
> Neverthess, it's an important pragmatic detail.  We should never throw
> away a test just because rearrangement makes a test less convenient.

So, should I put the test back in (in test_str.py)?

>>I'm not sure, whether the optimization should be added to
>>unicode.find().
> 
> Believing you mean join(), yes.

Unfortunately the implementations of str.join and unicode.join
look completely different. str.join does a PySequence_Fast() and
then tests whether the sequence length is 0 or 1, unicode.join
iterates through the argument via PyObject_GetIter/PyIter_Next.

Adding the optimization might result in a complete rewrite of
PyUnicode_Join().

> Doing common endcases efficiently in
> C code is an important quality-of-implementation concern, lest people
> need to add reams of optimization test-&-branch guesses in their own
> Python code.  For example, the SpamBayes tokenizer has many passes
> that split input strings on magical separators of one kind or another,
> pasting the remaining pieces together again via string.join().  It's
> explicitly noted in the code that special-casing the snot out of
> "separator wasn't found" in Python is a lot slower than letting
> string.join(single_element_list) just return the list element, so that
> simple, uniform Python code works well in all cases.  It's expected
> that *most* of these SB passes won't find the separator they're
> looking for, and it's important not to make endless copies of
> unboundedly large strings in the expected case.  The more heavily used
> unicode strings become, the more important that they treat users
> kindly in such cases too.

Seems like we have to rewrite PyUnicode_Join().

Bye,
    Walter D?rwald


From walter at livinglogic.de  Thu Aug 26 22:06:18 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 26 22:06:28 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412C297B.2040208@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>
	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>
	<412A5693.4050008@livinglogic.de>
	<412ACE79.4060605@v.loewis.de> <412B0C2D.7030507@egenix.com>
	<412BA1C5.5050808@livinglogic.de> <412C297B.2040208@v.loewis.de>
Message-ID: <412E42B9.4090508@livinglogic.de>

Martin v. L?wis wrote:

> Walter D?rwald wrote:
> 
>> If a tuple is passed and returned this makes it possible
>> from Python code to mangle the state. IMHO this should be
>> avoided if possible.
> 
> Not necessarily. We are all consenting adults. So if the
> code checks whether the state is sane, from a typing point
> of view, i.e. if you can't crash the interpreter, then there
> is no need to hide the state from the user.

OK, that's true. Unfortunately I'm not really that familiar
with the bit twiddling in the UTF7 decoder to know it this could
break anything or not.

Bye,
    Walter D?rwald


From walter at livinglogic.de  Thu Aug 26 22:10:40 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 26 22:10:52 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412C50D6.6010801@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>
	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>
	<412A5693.4050008@livinglogic.de>	<412ACE79.4060605@v.loewis.de>
	<412B0C2D.7030507@egenix.com> <412BA1C5.5050808@livinglogic.de>
	<412C50D6.6010801@egenix.com>
Message-ID: <412E43C0.5010006@livinglogic.de>

M.-A. Lemburg wrote:

>>> [...]
>>> def decode_stateful(data, state=None):
>>>     ... decode and modify state ...
>>>     return (decoded_data, length_consumed, state)
>>
>> Another option might be that the decode function changes
>> the state object in place.
> 
> Good idea.

But that's totally up to the implementor.

>>> where the object type and contents of the state variable
>>> is defined per codec (e.g. could be a tuple, just a single
>>> integer or some other special object).
>>
>> If a tuple is passed and returned this makes it possible
>> from Python code to mangle the state. IMHO this should be
>> avoided if possible.
> 
>>> Otherwise we'll end up having different interface
>>> signatures for all codecs and extending them to accomodate
>>> for future enhancement will become unfeasable without
>>> introducing yet another set of APIs.
>>
>> We already have slightly different decoding functions:
>> utf_16_ex_decode() takes additional arguments.
> 
> Right - it was a step in the wrong direction. Let's not
> use a different path for the future.

utf_16_ex_decode() serves a purpose: it help implement
the UTF16 decoder, which has to switch to UTF-16-BE or
UTF-16-LE according to the BOM, so utf_16_ex_decode()
needs a way to comunicate that back to the caller.

>>> Let's discuss this some more and implement it for Python 2.5.
>>> For Python 2.4, I think we can get away with what we already
>>> have:
>>
>>  > [...]
>>
>> OK, I've updated the patch.
>>
>>> [...]
>>> The buffer logic should only be used for streams
>>> that do not support the interface to push back already
>>> read bytes (e.g. .unread()).
>>>
>>>  From a design perspective, keeping read data inside the
>>> codec is the wrong thing to do, simply because it leaves
>>> the input stream in an undefined state in case of an error
>>> and there's no way to correlate the stream's read position
>>> to the location of the error.
>>>
>>> With a pushback method on the stream, all the stream
>>> data will be stored on the stream, not the codec, so
>>> the above would no longer be a problem.
>>
>> On the other hand this requires special stream. Data
>> already read is part of the codec state, so why not
>> put it into the codec?
> 
> Ideally, the codec should not store data,

I consider the remaining undecoded bytes to be part of
the codec state once the have been read from the stream.

> but only
> reference it. It's better to keep things well
> separated which is why I think we need the .unread()
> interface and eventually a queue interface to support
> the feeding operation.
> 
>>> However, we can always add the .unread() support to the
>>> stream codecs at a later stage, so it's probably ok
>>> to default to the buffer logic for Python 2.4.
>>
>> OK.
>>
>>>> That still leaves the issue
>>>> of the last read operation, which I'm tempted to leave unresolved
>>>> for Python 2.4. No matter what the solution is, it would likely
>>>> require changes to all codecs, which is not good.
>>>
>>> We could have a method on the codec which checks whether
>>> the codec buffer or the stream still has pending data
>>> left. Using this method is an application scope consideration,
>>> not a codec issue.
>>
>> But this mean that the normal error handling can't be used
>> for those trailing bytes.
> 
> Right, but then: missing data (which usually causes the trailing
> bytes) is really something for the application to deal with,
> e.g. by requesting more data from the user, another application
> or trying to work around the problem in some way. I don't think
> that the codec error handler can practically cover these
> possibilities.

But in many cases the user might want to use "ignore" or "replace"
error handling.

Bye,
    Walter D?rwald


From walter at livinglogic.de  Thu Aug 26 22:13:17 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 26 22:13:21 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412CEEE9.60207@v.loewis.de>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>
	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>
	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>
	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>
	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>
	<412A5693.4050008@livinglogic.de>	<412ACE79.4060605@v.loewis.de>
	<412B0C2D.7030507@egenix.com>	<412BA1C5.5050808@livinglogic.de>
	<412C297B.2040208@v.loewis.de> <412C4EA9.8010003@egenix.com>
	<412CEEE9.60207@v.loewis.de>
Message-ID: <412E445D.2050806@livinglogic.de>

Martin v. L?wis wrote:

> M.-A. Lemburg wrote:
> 
>> Martin, there are two reasons for hiding away these details:
>>
>> 1. we need to be able to change the codec state without
>>    breaking the APIs
 >
> That will be possible with the currently-proposed patch.
> The _codecs methods are not public API, so changing them
> would not be an API change.

Exactly.

>> 2. we don't want the state to be altered by the user
> 
> We are all consenting adults, and we can't *really*
> prevent it, anyway. For example, the user may pass an
> old state, or a state originating from a different codec
> (instance). We need to support this gracefully (i.e. with
> a proper Python exception).

The state communicated in the UTF-7 decoder is just a bunch
of values. Checking the type is done via PyArg_ParseTuple().

>> A single object serves this best and does not create
>> a whole plethora of new APIs in the _codecs module.
>> This is not over-design, but serves a reason.
> 
> It does put a burden on codec developers, which need
> to match the "official" state representation policy.
> Of course, if they are allowed to return a tuple
> representing their state, that would be fine with
> me.

Looking at the UTF-7 decoder this seems to be the
simplest option.

Bye,
    Walter D?rwald


From walter at livinglogic.de  Thu Aug 26 22:19:48 2004
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu Aug 26 22:19:57 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412E01AB.8030808@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>	<412A5693.4050008@livinglogic.de>	<412ACE79.4060605@v.loewis.de>	<412B0C2D.7030507@egenix.com>	<412BA1C5.5050808@livinglogic.de>	<412C297B.2040208@v.loewis.de>
	<412C4EA9.8010003@egenix.com> <412CEEE9.60207@v.loewis.de>
	<412E01AB.8030808@egenix.com>
Message-ID: <412E45E4.2050504@livinglogic.de>

M.-A. Lemburg wrote:

> Martin v. L?wis wrote:
> 
>> M.-A. Lemburg wrote:
>>
>>> Martin, there are two reasons for hiding away these details:
>>>
>>> 1. we need to be able to change the codec state without
>>>    breaking the APIs
>>
>> That will be possible with the currently-proposed patch.
>> The _codecs methods are not public API, so changing them
>> would not be an API change.
> 
> Uhm, I wasn't talking about the builtin codecs only (of course,
> we can change those to our liking). I'm after a generic
> interface for stateful codecs.

But that interface is only between the StreamReader
and any helper function that the codec implementer
might want to use. If there ise no helper function
there is no interface.

>>> 2. we don't want the state to be altered by the user
>>
>> We are all consenting adults, and we can't *really*
>> prevent it, anyway. For example, the user may pass an
>> old state, or a state originating from a different codec
>> (instance). We need to support this gracefully (i.e. with
>> a proper Python exception).
> 
> True, but the codec writer should be in control of the
> state object, its format and what the user can or cannot
> change.

Yes, we should not dictate, how, why or if the codec
implementer has to pass around any state. The only thing
we have to dictate is that StreamReaders have to keep their
state between calls to read().

>>> A single object serves this best and does not create
>>> a whole plethora of new APIs in the _codecs module.
>>> This is not over-design, but serves a reason.
>>
>> It does put a burden on codec developers, which need
>> to match the "official" state representation policy.
>> Of course, if they are allowed to return a tuple
>> representing their state, that would be fine with
>> me.
> 
> They can use any object they like to keep the state
> in whatever format they choose. I think this makes it
> easier on the codec writer, rather than harder.
> 
> Furthermore, they can change the way they store state
> e.g. to accomodate for new features they may want to
> add to the codec, without breaking the interface.

That's basically the current state of the codec machinery,
so we don't have to change anything in the specification.

BTW, I hope that I can add updated documentation to the
patch tomorrow (for PyUnicode_DecodeUTF8Stateful() and
friends and for the additional arguments to read()),
because I'll be on vacation the next week.

Bye,
    Walter D?rwald


From dave at boost-consulting.com  Thu Aug 26 22:34:35 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Aug 26 22:34:51 2004
Subject: [Python-Dev] urllib.urlretrieve
Message-ID: <u7jrlzlic.fsf@boost-consulting.com>


Is there a known reliability problem with urllib.urlretrieve?  I'm
getting silent truncation of downloaded files.

http://sourceforge.net/tracker/?func=detail&aid=1016880&group_id=5470&atid=105470

has the details.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From martin at v.loewis.de  Thu Aug 26 23:00:44 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 26 23:01:06 2004
Subject: [Python-Dev] Decoding incomplete unicode
In-Reply-To: <412E01AB.8030808@egenix.com>
References: <4106BD91.4060009@livinglogic.de>	<4106C24F.3070203@egenix.com>	<4106C5D4.1020200@v.loewis.de>	<4106CC80.4070401@egenix.com>	<41076F8D.3030507@livinglogic.de>	<4107751C.2020207@egenix.com>	<4107DA6E.30500@livinglogic.de>	<4107F0E8.3010005@egenix.com>	<411920E4.9020404@livinglogic.de>	<411B6D79.2040503@egenix.com>	<411D1EB4.10103@livinglogic.de>	<412285D1.7010907@egenix.com>	<4122E19E.9050906@v.loewis.de>	<412314F6.5060502@egenix.com>	<4123BC56.7030607@v.loewis.de>	<4123C76B.3030100@livinglogic.de>	<4123D0C2.4080505@v.loewis.de>	<4124DA28.50106@livinglogic.de>	<4124FA07.1070701@v.loewis.de>	<412A5693.4050008@livinglogic.de>	<412ACE79.4060605@v.loewis.de>	<412B0C2D.7030507@egenix.com>	<412BA1C5.5050808@livinglogic.de>	<412C297B.2040208@v.loewis.de>
	<412C4EA9.8010003@egenix.com> <412CEEE9.60207@v.loewis.de>
	<412E01AB.8030808@egenix.com>
Message-ID: <412E4F7C.80201@v.loewis.de>

M.-A. Lemburg wrote:
> Uhm, I wasn't talking about the builtin codecs only (of course,
> we can change those to our liking). I'm after a generic
> interface for stateful codecs.

But we already have that! The StreamReader/StreamWriter interface
is perfectly suited for stateful codecs, and we have been supporting
it for several years now. The only problem is that a few builtin
codecs failed to implement that correctly, and Walter's patch fixes
that bug. There is no API change necessary whatsoever.

> True, but the codec writer should be in control of the
> state object, its format and what the user can or cannot
> change.

And indeed, with the current API, he is.

Regards,
Martin
From martin at v.loewis.de  Thu Aug 26 23:10:21 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Aug 26 23:10:54 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
In-Reply-To: <412E33B6.8080701@ocf.berkeley.edu>
References: <412E33B6.8080701@ocf.berkeley.edu>
Message-ID: <412E51BD.6090401@v.loewis.de>

Brett C. wrote:
> Personally I would like to turn off the test on OS X for now and then 
> try to see if there is a proper way to solve this.

In what way does that help? The test shows that the locale module is
broken on OS X. So the only sensible options are to remove the module
(or atleast the relevant functions), or to fix them. I fail to see how
skipping the test does any good. The test fails, and life goes on.

Regards,
Martin
From tim.peters at gmail.com  Thu Aug 26 23:20:28 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Aug 26 23:20:36 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test
	test_string.py, 1.25, 1.26
In-Reply-To: <412E3FFA.5030002@livinglogic.de>
References: <E1BzOAO-0005Hx-Qc@sc8-pr-cvs1.sourceforge.net>	<412B0C84.30204@livinglogic.de>	<412C38A2.9000909@iinet.net.au>
	<412DDBE8.2070206@livinglogic.de>
	<1f7befae040826112168b99b08@mail.gmail.com>
	<412E3FFA.5030002@livinglogic.de>
Message-ID: <1f7befae04082614206fc7722@mail.gmail.com>

[Walter D?rwald]
...
> Seems like we have to rewrite PyUnicode_Join().

We probably should anyway.  By eyeball just now, I noticed it slings
expressions like "reslen + itemlen + seplen" and "sz*2" without
worrying about integer overflow.  string_join() is careful about such
stuff; sooner or later, shortcuts of this nature turn into bug reports
From fredrik at pythonware.com  Thu Aug 26 23:28:28 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu Aug 26 23:26:45 2004
Subject: [Python-Dev] Re: urllib.urlretrieve
References: <u7jrlzlic.fsf@boost-consulting.com>
Message-ID: <cglkih$9fk$1@sea.gmane.org>

David Abrahams wrote:

> Is there a known reliability problem with urllib.urlretrieve?  I'm
> getting silent truncation of downloaded files.
>
> http://sourceforge.net/tracker/?func=detail&aid=1016880&group_id=5470&atid=105470
>
> has the details.

sourceforge is known to have reliability problems when downloading
large files.  here's a wget session on my machine (okay, it's not a silent
truncation, but SF is clearly not well):

18450K -> .......... .......... .......... .......... .......... [ 54%]
18500K -> .....                                                  [ 54%]

Closing fd 1944
23:11:52 (62.94 KB/s) - Connection closed at byte 18949128. Retrying.

--23:11:52--  http://cvs.sourceforge.net:80/cvstarballs/boost-cvsroot.tar.bz2
  (try: 2) => `boost-cvsroot.tar.bz2.3'
Connecting to cvs.sourceforge.net:80... Created fd 1944.
connected!
---request begin---
GET /cvstarballs/boost-cvsroot.tar.bz2 HTTP/1.0
User-Agent: Wget/1.5.3.1
Host: cvs.sourceforge.net:80
Accept: */*
Range: bytes=18949128-

---request end---
HTTP request sent, awaiting response... HTTP/1.1 502 Proxy Error
Date: Thu, 26 Aug 2004 21:10:04 GMT
Server: Apache/2.0.40 (Red Hat Linux)
Content-Length: 441
Connection: close
Content-Type: text/html; charset=iso-8859-1


Closing fd 1944
23:11:57 ERROR 502: Proxy Error. 



From raymond.hettinger at verizon.net  Thu Aug 26 23:38:56 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu Aug 26 23:39:29 2004
Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple String
	Substitutions
Message-ID: <002301c48bb5$17bfbe40$e841fea9@oemcomputer>

Before it's too late and the API gets frozen, I would like to propose an
alternate implementation for PEP292 that exposes two functions instead
of two classes.

Current way:    print Template('Turn $direction') %
dict(direction='right')

Proposed:       print dollarsub('Turn $direction',
dict(direction='right'))
 or:            print dollarsub('Turn $direction', direction='right')


My main issue with the current implementation is that we get no leverage
from using a class instead of a function.  Though the API is fairly
simple either way, it is easier to learn and document functions than
classes.  We gain nothing from instantiation -- the underlying data
remains immutable and no additional state is attached.  The only new
behavior is the ability to apply the mod operator.  Why not do it in one
step.

I had thought a possible advantage of classes was that they could be
usefully subclassed.  However, a little dickering around showed that to
do anything remotely interesting (beyond changing the pattern alphabet)
you have to rewrite everything by overriding both the method and the
pattern.  Subclassing gained you nothing, but added a little bit of
complexity. A couple of simple exercises show this clearly: write a
subclass using a different escape character or one using dotted
identifiers for attribute lookup in the local namespace -- either way
subclasses provide no help and only get in the way.

One negative effect of the class implementation is that it inherits from
unicode and always returns a unicode result even if all of the inputs
(mapping values and template) are regular strings.  With a function
implementation, that can be avoided (returning unicode only if one of
the inputs is unicode).

The function approach also makes it possible to have keyword arguments
(see the example above) as well as a mapping.  This isn't a big win, but
it is nice to have and reads well in code that is looping over multiple
substitutions (mailmerge style):
  
   for girl in littleblackbook:
      print dollarsub(loveletter, name=girl[0].title(),
favoritesong=girl[3])

Another minor advantage for a function is that it is easier to lookup in
the reference.  If a reader sees the % operator being applied and looks
it up in the reference, it is going to steer them in the wrong
direction.  This is doubly true if the Template instantiation is remote
from the operator application.


Summary for functions:
* is more appropriate when there is no state
* no unnecessary instantiation
* can be applied in a single step
* a little easier to learn/use/document
* doesn't force result to unicode
* allows keyword arguments
* easy to find in the docs


Raymond




----------- Sample Implementation -------------

def dollarsub(template, mapping=None, **kwds):
    """A function for supporting $-substitutions."""
    typ = type(template)
    if mapping is None:
        mapping = kwds
    def convert(mo):
        escaped, named, braced, bogus = mo.groups()  
        if escaped is not None:
            return '$'
        if bogus is not None:
            raise ValueError('Invalid placeholder at index %d' %
                             mo.start('bogus'))
        val = mapping[named or braced]
        return typ(val)
    return _pattern.sub(convert, template)

def safedollarsub(template, mapping=None, **kwds):
    """A function for $-substitutions.

    This function is 'safe' in the sense that you will never get
KeyErrors if
    there are placeholders missing from the interpolation dictionary.
In that
    case, you will get the original placeholder in the value string.
    """
    typ = type(template)
    if mapping is None:
        mapping = kwds    
    def convert(mo):
        escaped, named, braced, bogus = mo.groups()
        if escaped is not None:
            return '$'
        if bogus is not None:
            raise ValueError('Invalid placeholder at index %d' %
                             mo.start('bogus'))
        if named is not None:
            try:
                return typ(mapping[named])
            except KeyError:
                return '$' + named
        try:
            return typ(mapping[braced])
        except KeyError:
            return '${' + braced + '}'
    return _pattern.sub(convert, template)









From python at rcn.com  Thu Aug 26 23:41:33 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Aug 26 23:42:09 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Lib/testtest_string.py, 1.25, 1.26
In-Reply-To: <1f7befae04082614206fc7722@mail.gmail.com>
Message-ID: <002401c48bb5$765463c0$e841fea9@oemcomputer>

> [Walter D?rwald]
> ...
> > Seems like we have to rewrite PyUnicode_Join().
>

[Uncle Tim] 
> We probably should anyway.  By eyeball just now, I noticed it slings
> expressions like "reslen + itemlen + seplen" and "sz*2" without
> worrying about integer overflow.  string_join() is careful about such
> stuff; sooner or later, shortcuts of this nature turn into bug reports

Sounds like an excellent project for Nick.
Too bad I had just convinced him it was a can of worms ;-)


Raymond

From tim.peters at gmail.com  Fri Aug 27 00:20:50 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug 27 00:20:59 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Lib/testtest_string.py, 1.25, 1.26
In-Reply-To: <002401c48bb5$765463c0$e841fea9@oemcomputer>
References: <002401c48bb5$765463c0$e841fea9@oemcomputer>
Message-ID: <1f7befae04082615206760c806@mail.gmail.com>

[Walter D?rwald]
>>> ...
>>> Seems like we have to rewrite PyUnicode_Join().

[Uncle Timmy]
>> We probably should anyway.  By eyeball just now, I noticed it slings
>> expressions like "reslen + itemlen + seplen" and "sz*2" without
>> worrying about integer overflow.  string_join() is careful about such
>> stuff; sooner or later, shortcuts of this nature turn into bug reports

[Nephew Raymond]
> Sounds like an excellent project for Nick.
> Too bad I had just convinced him it was a can of worms ;-)

I needed a break from intractable database problems, and am almost
done with PyUnicode_Join().  I'm not doing auto-unicode(), though, so
there will still be plenty of fun left for Nick!
From dave at boost-consulting.com  Fri Aug 27 00:45:47 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Aug 27 00:46:02 2004
Subject: [Python-Dev] Re: urllib.urlretrieve
References: <u7jrlzlic.fsf@boost-consulting.com> <cglkih$9fk$1@sea.gmane.org>
Message-ID: <ueklty0v8.fsf@boost-consulting.com>

"Fredrik Lundh" <fredrik@pythonware.com> writes:

>> Is there a known reliability problem with urllib.urlretrieve?  I'm
>> getting silent truncation of downloaded files.
>>
>> http://sourceforge.net/tracker/?func=detail&aid=1016880&group_id=5470&atid=105470
>>
>> has the details.
>
> sourceforge is known to have reliability problems when downloading
> large files.  here's a wget session on my machine (okay, it's not a silent
> truncation, but SF is clearly not well):

Okay, that tells me something... but shouldn't urllib throw an
exception in case of a problem... or shouldn't it do something to
retry?

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From bac at OCF.Berkeley.EDU  Fri Aug 27 00:56:23 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug 27 00:56:34 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
In-Reply-To: <412E51BD.6090401@v.loewis.de>
References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de>
Message-ID: <412E6A97.5030509@ocf.berkeley.edu>

Martin v. L?wis wrote:

> Brett C. wrote:
> 
>> Personally I would like to turn off the test on OS X for now and then 
>> try to see if there is a proper way to solve this.
> 
> 
> In what way does that help? The test shows that the locale module is
> broken on OS X. So the only sensible options are to remove the module
> (or atleast the relevant functions), or to fix them. I fail to see how
> skipping the test does any good. The test fails, and life goes on.
> 

It prevents repetitive bug reports for an issue we already know about. 
And as I said in the email, test_locale is skipped for OS X so I thought 
same reasoning would apply here.

-Brett
From dave at boost-consulting.com  Fri Aug 27 01:12:30 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Aug 27 01:12:47 2004
Subject: [Python-Dev] Re: list += string??
References: <ur7pu2da8.fsf@boost-consulting.com>
	<1f7befae04082607303b7bc26e@mail.gmail.com>
	<u8yc22bfx.fsf@boost-consulting.com>
Message-ID: <u8yc1xzmp.fsf@boost-consulting.com>

David Abrahams <dave@boost-consulting.com> writes:

>> It was new when the iterator protocol was introduced.  It wasn't
>> explicitly intended that you get surprises for strings
>> specifically, but it was explicitly intended that
>>
>>     list += whatever
>>
>> work like
>>
>>     list.extend(whatever)
>>
>> and that whatever can be any iterable object.
>
>
> I figured as much.

Report from the field: I just discovered a bug that's been hiding in
my code for a few months.  The code was being exercised, but the
result was the same as if the code had been correct until I threw
some new examples at it.  The problem was:

   self.body[e[0]:e[1]] = s

self.body is a list of strings, and s is a string.  Later
''.join(self.body) gets called, so for many cases the bug was hidden.
No offense intended (okay, maybe a tiny little offense), but the
experience here is a bit Perl-like.

>> Since strings are iterable objects, it's no more or less surprising
>> than that 'for ch in "bomb"' works too <wink>.
>
> I want my ,= operator!

I still do, but it wouldn't have solved _this_ problem.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

From ncoghlan at iinet.net.au  Fri Aug 27 01:20:15 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Fri Aug 27 01:21:58 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Lib/test	test_string.py, 1.25, 1.26
In-Reply-To: <412DDBE8.2070206@livinglogic.de>
References: <E1BzOAO-0005Hx-Qc@sc8-pr-cvs1.sourceforge.net>	<412B0C84.30204@livinglogic.de>
	<412C38A2.9000909@iinet.net.au> <412DDBE8.2070206@livinglogic.de>
Message-ID: <412E702F.6030808@iinet.net.au>

Walter D?rwald wrote:
> I'm working on it, however I discovered that unicode.join()
> doesn't optimize this special case:

I noticed that during the implementation of the patch. I think it's 
because unicode join doesn't use a multi-pass approach, so there's no 
opportunity to notice that the optimisation is possible.

> s = "foo"
> assert "".join([s]) is s
> 
> u = u"foo"
> assert u"".join([s]) is s
> 
> The second assertion fails.
> 
> I'd say that this test (joining a one item sequence returns
> the item itself) should be removed because it tests an
> implementation detail.

Fine by me - I was uncertain about that when I posted the patch to SF 
(there's a question about it in the patch tracker item). The main reason 
I had that test there was to make sure that the second version of the 
patch didn't break this optimisation the way the first version did.

Regards,
Nick.
From bac at OCF.Berkeley.EDU  Fri Aug 27 01:41:10 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug 27 01:41:23 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
In-Reply-To: <412E51BD.6090401@v.loewis.de>
References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de>
Message-ID: <412E7516.5080805@ocf.berkeley.edu>

Martin v. L?wis wrote:

> Brett C. wrote:
> 
>> Personally I would like to turn off the test on OS X for now and then 
>> try to see if there is a proper way to solve this.
> 
> 
> In what way does that help? The test shows that the locale module is
> broken on OS X. So the only sensible options are to remove the module
> (or atleast the relevant functions), or to fix them.

I have an idea for detecting a broken setlocale.  When configure.in does 
its compiling does it compile with the same flags that the Makefile 
uses for compiling?  If it does we can test for something we know will 
be different (fr_FR has a different decimal point, for instance) and 
compare them and then have a value in pyconfig.h to signal that a busted 
setlocale exists.  setlocale can then raise locale.Error automatically 
for platforms that have that macro defined.  But the test has to be 
compiled with the ``-framework CoreFoundation`` option or else this 
won't work.

The reason I don't want to do a blanket ``#ifdef __APPLE__`` solution is 
because I don't know if this was an issue in OS X 10.2 or earlier 
(http://www.hmug.org/man/3/setlocale.html seems to suggest setlocale was 
deprecated which means it may have still worked).

-Brett
From ncoghlan at iinet.net.au  Fri Aug 27 01:53:19 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Fri Aug 27 01:55:01 2004
Subject: [Python-Dev] Auto-str and auto-unicode in join
In-Reply-To: <1f7befae04082615206760c806@mail.gmail.com>
References: <002401c48bb5$765463c0$e841fea9@oemcomputer>
	<1f7befae04082615206760c806@mail.gmail.com>
Message-ID: <412E77EF.80007@iinet.net.au>

Tim Peters wrote:
> I needed a break from intractable database problems, and am almost
> done with PyUnicode_Join().  I'm not doing auto-unicode(), though, so
> there will still be plenty of fun left for Nick!

I actually got that mostly working (off slightly out-of-date CVS though).

Joining a sequence of 10 integers with auto-str seems to take about 60% 
of the time of a str(x) list comprehension on that same sequence (and 
the PySequence_Fast call means that a generator is slightly slower than 
a list comp!). For a sequence which mixed strings and non-strings, the 
gains could only increase.

However, there is one somewhat curly problem I'm not sure what to do about.

To avoid slowing down the common case of string join (a list of only 
strings) it is necessary to do the promotion to string in the type-check 
& size-calculation pass.

That's fine in the case of a list that consists of only strings and 
non-basestrings, or the case of a unicode separator - every 
non-basestring is converted using either PyObject_Str or PyObject_Unicode.

Where it gets weird is something like this:
     ''.join([an_int, a_unicode_str])
     u''.join([an_int, a_unicode_str])

In the first case, the int will first be converted to a string via 
PyObject_Str, and then that string representation is what will get 
converted to Unicode after the detection of the unicode string causes 
the join to be handed over to Unicode join.

In the latter case, the int is converted directly to Unicode.

So my question would be, is it reasonable to expect that 
PyObject_Unicode(PyObject_Str(some_object)) give the same answer as 
PyObject_Unicode(some_object)?

If not, then the string join would have to do something whereby it kept 
a 'pristine' version of the sequence around to hand over to the Unicode 
join.

My first attempt at implementing this feature had that property, but 
also had the effect of introducing about a 1% slowdown of the standard 
sequence-of-strings case (it introduced an extra if statement to see if 
a 'stringisation' pass was required after the initial type checking and 
sizing pass). For longer sequences than 10 strings, I imagine the 
relative slowdown would be much less.

Hmm. . . I think I see a way to implement this, while still avoiding 
adding any code to the standard path through the function. It'd be 
slower for the case where an iterator is passed in, and we automatically 
invoke PyObject_Str but don't end up delegating to Unicode join, though, 
as it involves making a copy of the sequence that only gets used if the 
Unicode join is invoked. (If the original object is a real sequence, 
rather than an iterator, there is no extra overhead - we have to make 
the copy anyway, to avoid mutating the user's sequence).

If people are definitely interested in this feature, I could probably 
put a patch together next week.

Regards,
Nick.

From nbastin at opnet.com  Fri Aug 27 02:49:53 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Fri Aug 27 02:50:15 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
In-Reply-To: <412E7516.5080805@ocf.berkeley.edu>
References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de>
	<412E7516.5080805@ocf.berkeley.edu>
Message-ID: <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com>


On Aug 26, 2004, at 7:41 PM, Brett C. wrote:

> Martin v. L?wis wrote:
>
>> Brett C. wrote:
>>> Personally I would like to turn off the test on OS X for now and 
>>> then try to see if there is a proper way to solve this.
>> In what way does that help? The test shows that the locale module is
>> broken on OS X. So the only sensible options are to remove the module
>> (or atleast the relevant functions), or to fix them.
>
> I have an idea for detecting a broken setlocale.  When configure.in 
> does its compiling does it compile with the same flags that the 
> Makefile uses for compiling?  If it does we can test for something we 
> know will be different (fr_FR has a different decimal point, for 
> instance) and compare them and then have a value in pyconfig.h to 
> signal that a busted setlocale exists.  setlocale can then raise 
> locale.Error automatically for platforms that have that macro defined. 
>  But the test has to be compiled with the ``-framework 
> CoreFoundation`` option or else this won't work.

I think we should actually just fix this, rather than removing locale 
support for MacOS X builds.  The trick is that we need to call 
__setonlyClocaleconv() again, to remove the protection against calling 
setlocale() in framework builds (why they did this in the first place 
is beyond me).  Seems like it should be an easy gate against darwin, 
and then call that on startup.

--
Nick

From greg at cosc.canterbury.ac.nz  Fri Aug 27 03:32:44 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri Aug 27 03:32:50 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test
	test_string.py, 1.25, 1.26
In-Reply-To: <1f7befae040826112168b99b08@mail.gmail.com>
Message-ID: <200408270132.i7R1Wilx007968@cosc353.cosc.canterbury.ac.nz>

Tim Peters <tim.peters@gmail.com>:

> > u = u"foo"
> > assert u"".join([s]) is s
> 
> Well, in that example it *has* to fail, because the input (s) wasn't a
> unicode string to begin with, but u"".join() must return a unicode
> string.

It could be argued that joining a single-element list doesn't
involve using the separator at all, so its type shouldn't matter.

It could also be argued that joining with an empty string, of
whatever flavour, is equivalent to just concatenating the list
elements with no separator, so again the type of the separator
shouldn't matter.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From tim.peters at gmail.com  Fri Aug 27 03:46:24 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug 27 03:46:26 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Lib/test test_string.py, 1.25, 1.26
In-Reply-To: <200408270132.i7R1Wilx007968@cosc353.cosc.canterbury.ac.nz>
References: <200408270132.i7R1Wilx007968@cosc353.cosc.canterbury.ac.nz>
Message-ID: <1f7befae04082618465600f3aa@mail.gmail.com>

[Greg Ewing]
> It could be argued that joining a single-element list doesn't
> involve using the separator at all, so its type shouldn't matter.
>
> It could also be argued that joining with an empty string, of
> whatever flavour, is equivalent to just concatenating the list
> elements with no separator, so again the type of the separator
> shouldn't matter.

Since such arguments wouldn't prevail, I'd rather not bother.
From bac at OCF.Berkeley.EDU  Fri Aug 27 03:54:06 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug 27 03:54:17 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
In-Reply-To: <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com>
References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de>
	<412E7516.5080805@ocf.berkeley.edu>
	<01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com>
Message-ID: <412E943E.4060908@ocf.berkeley.edu>

Nick Bastin wrote:

> 
> I think we should actually just fix this, rather than removing locale 
> support for MacOS X builds.  The trick is that we need to call 
> __setonlyClocaleconv() again, to remove the protection against calling 
> setlocale() in framework builds (why they did this in the first place is 
> beyond me).  Seems like it should be an easy gate against darwin, and 
> then call that on startup.
> 

I don't like that solution since that function is obviously not part of 
any public API.

-Brett
From bob at redivi.com  Fri Aug 27 04:32:12 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Aug 27 04:32:48 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
In-Reply-To: <412E943E.4060908@ocf.berkeley.edu>
References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de>
	<412E7516.5080805@ocf.berkeley.edu>
	<01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com>
	<412E943E.4060908@ocf.berkeley.edu>
Message-ID: <4D5B054E-F7D1-11D8-B43E-000A95686CD8@redivi.com>

On Aug 26, 2004, at 9:54 PM, Brett C. wrote:

> Nick Bastin wrote:
>
>> I think we should actually just fix this, rather than removing locale 
>> support for MacOS X builds.  The trick is that we need to call 
>> __setonlyClocaleconv() again, to remove the protection against 
>> calling setlocale() in framework builds (why they did this in the 
>> first place is beyond me).  Seems like it should be an easy gate 
>> against darwin, and then call that on startup.
>
> I don't like that solution since that function is obviously not part 
> of any public API.

Well there are two ways to fix it:

1) Call __setonlyClocaleconv() if it's there via weak linking or 
whatnot around any setlocale or the like.  This will at least affect OS 
X 10.3, I'm not sure about 10.2, and the rumor says it's fixed "now" in 
10.4.

2) Write a whole new module that uses Apple API for localization.

Obviously 2 is the "best" solution, but requires the most time.  1 is 
easy-ish and will work reliably on all the machines that need it 
(assuming the rumor is correct) unless Apple does something totally 
strange and changes the behavior of a previous-release OS for reasons 
other than security flaws :)

-bob
From bac at OCF.Berkeley.EDU  Fri Aug 27 05:33:54 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Aug 27 05:33:59 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
In-Reply-To: <4D5B054E-F7D1-11D8-B43E-000A95686CD8@redivi.com>
References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de>
	<412E7516.5080805@ocf.berkeley.edu>
	<01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com>
	<412E943E.4060908@ocf.berkeley.edu>
	<4D5B054E-F7D1-11D8-B43E-000A95686CD8@redivi.com>
Message-ID: <412EABA2.2000502@ocf.berkeley.edu>

Bob Ippolito wrote:

> 
> Well there are two ways to fix it:
> 
> 1) Call __setonlyClocaleconv() if it's there via weak linking or whatnot 
> around any setlocale or the like.  This will at least affect OS X 10.3, 
> I'm not sure about 10.2, and the rumor says it's fixed "now" in 10.4.
> 
> 2) Write a whole new module that uses Apple API for localization.
> 
> Obviously 2 is the "best" solution, but requires the most time.  1 is 
> easy-ish and will work reliably on all the machines that need it 
> (assuming the rumor is correct) unless Apple does something totally 
> strange and changes the behavior of a previous-release OS for reasons 
> other than security flaws :)
> 

OK, starting to sound like detecting __setonlyClocaleconv() in 
configure.in and using that info to deal with it is winning with other 
people, at least as an initial solution.  Everyone else agree with this?

I just checked and it looks like calling the function with a non-zero 
argument will force the locale back to "C" even if you just set it to a 
specific locale; so the function seems to force the locale to "C" and 
lock it down.  So it will most likely need to be called right before the 
first setlocale call made by Python (I think it is in PyInitialize() ) 
and then not call it again.

-Brett
From bob at redivi.com  Fri Aug 27 06:59:06 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Aug 27 06:59:44 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
In-Reply-To: <412EABA2.2000502@ocf.berkeley.edu>
References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de>
	<412E7516.5080805@ocf.berkeley.edu>
	<01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com>
	<412E943E.4060908@ocf.berkeley.edu>
	<4D5B054E-F7D1-11D8-B43E-000A95686CD8@redivi.com>
	<412EABA2.2000502@ocf.berkeley.edu>
Message-ID: <D295D24D-F7E5-11D8-B43E-000A95686CD8@redivi.com>


On Aug 26, 2004, at 11:33 PM, Brett C. wrote:

> Bob Ippolito wrote:
>
>> Well there are two ways to fix it:
>> 1) Call __setonlyClocaleconv() if it's there via weak linking or 
>> whatnot around any setlocale or the like.  This will at least affect 
>> OS X 10.3, I'm not sure about 10.2, and the rumor says it's fixed 
>> "now" in 10.4.
>> 2) Write a whole new module that uses Apple API for localization.
>> Obviously 2 is the "best" solution, but requires the most time.  1 is 
>> easy-ish and will work reliably on all the machines that need it 
>> (assuming the rumor is correct) unless Apple does something totally 
>> strange and changes the behavior of a previous-release OS for reasons 
>> other than security flaws :)
>
> OK, starting to sound like detecting __setonlyClocaleconv() in 
> configure.in and using that info to deal with it is winning with other 
> people, at least as an initial solution.  Everyone else agree with 
> this?

No.  Don't use configure.  The machine you compile Python with is not 
necessarily the machine you will use Python with (for example, 
compiling a 10.2 compatible Python and running on 10.4).  Use dyld or 
CFBundle API to look up the function dynamically, or use weak linking.

> I just checked and it looks like calling the function with a non-zero 
> argument will force the locale back to "C" even if you just set it to 
> a specific locale; so the function seems to force the locale to "C" 
> and lock it down.  So it will most likely need to be called right 
> before the first setlocale call made by Python (I think it is in 
> PyInitialize() ) and then not call it again.

Calling the function with a non-zero argument forces the locale to stay 
at "C", and it returns the last value that was passed to it.  I would 
say that any Python function that depends on locale should do:

x = __setonlyClocaleconv(0)
# do python stuff carefully
__setonlyClocaleconv(x)

Otherwise you will probably break CoreFoundation (which could break 
just about anything, since nearly all Apple APIs depend on it).

Maybe this is just about as much work as writing a CoreFoundation based 
locale module?  I'm not sure which parts of Python are affected by 
setlocale().

-bob
From nbastin at opnet.com  Fri Aug 27 07:10:57 2004
From: nbastin at opnet.com (Nick Bastin)
Date: Fri Aug 27 07:11:17 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
In-Reply-To: <D295D24D-F7E5-11D8-B43E-000A95686CD8@redivi.com>
References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de>
	<412E7516.5080805@ocf.berkeley.edu>
	<01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com>
	<412E943E.4060908@ocf.berkeley.edu>
	<4D5B054E-F7D1-11D8-B43E-000A95686CD8@redivi.com>
	<412EABA2.2000502@ocf.berkeley.edu>
	<D295D24D-F7E5-11D8-B43E-000A95686CD8@redivi.com>
Message-ID: <7A882E04-F7E7-11D8-8D32-000D932927FE@opnet.com>


On Aug 27, 2004, at 12:59 AM, Bob Ippolito wrote:

> Maybe this is just about as much work as writing a CoreFoundation 
> based locale module?  I'm not sure which parts of Python are affected 
> by setlocale().

As a side note, I'm currently working on wrapping ICU as a Python 
module, which would give you reliable cross-platform locale support, 
independent of the system locale.  Incidentally, ICU is what Apple uses 
in MacOS X anyhow...

This module will not be ready before 2.4... :-(

--
Nick

From ronaldoussoren at mac.com  Fri Aug 27 08:28:58 2004
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri Aug 27 08:29:16 2004
Subject: [Python-Dev] Dealing with test__locale failure on OS X before a3
In-Reply-To: <01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com>
References: <412E33B6.8080701@ocf.berkeley.edu> <412E51BD.6090401@v.loewis.de>
	<412E7516.5080805@ocf.berkeley.edu>
	<01FEECB2-F7C3-11D8-8D32-000D932927FE@opnet.com>
Message-ID: <60F976B0-F7F2-11D8-8627-000D93AD379E@mac.com>


On 27-aug-04, at 2:49, Nick Bastin wrote:

>
> On Aug 26, 2004, at 7:41 PM, Brett C. wrote:
>
>> Martin v. L?wis wrote:
>>
>>> Brett C. wrote:
>>>> Personally I would like to turn off the test on OS X for now and 
>>>> then try to see if there is a proper way to solve this.
>>> In what way does that help? The test shows that the locale module is
>>> broken on OS X. So the only sensible options are to remove the module
>>> (or atleast the relevant functions), or to fix them.
>>
>> I have an idea for detecting a broken setlocale.  When configure.in 
>> does its compiling does it compile with the same flags that the 
>> Makefile uses for compiling?  If it does we can test for something we 
>> know will be different (fr_FR has a different decimal point, for 
>> instance) and compare them and then have a value in pyconfig.h to 
>> signal that a busted setlocale exists.  setlocale can then raise 
>> locale.Error automatically for platforms that have that macro 
>> defined.  But the test has to be compiled with the ``-framework 
>> CoreFoundation`` option or else this won't work.
>
> I think we should actually just fix this, rather than removing locale 
> support for MacOS X builds.  The trick is that we need to call 
> __setonlyClocaleconv() again, to remove the protection against calling 
> setlocale() in framework builds (why they did this in the first place 
> is beyond me).  Seems like it should be an easy gate against darwin, 
> and then call that on startup.
Why do you think it is save to re-enable setlocale? There must be some 
reason for disabling setlocale. If you re-enable setlocale some random 
CoreFoundation function might suddenly fail.

Ronald

From fredrik at pythonware.com  Fri Aug 27 09:45:02 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri Aug 27 09:43:24 2004
Subject: [Python-Dev] Re: urllib.urlretrieve
References: <u7jrlzlic.fsf@boost-consulting.com> <cglkih$9fk$1@sea.gmane.org>
	<ueklty0v8.fsf@boost-consulting.com>
Message-ID: <cgmomi$c3q$1@sea.gmane.org>

David Abrahams wrote:

>> sourceforge is known to have reliability problems when downloading
>> large files.  here's a wget session on my machine (okay, it's not a silent
>> truncation, but SF is clearly not well):
>
> Okay, that tells me something... but shouldn't urllib throw an
> exception in case of a problem... or shouldn't it do something to
> retry?

from the HTTP speification:

    HTTP/1.1 user agents MUST notify the user when an invalid
    length is received and detected.

is urllib a user agent?   or is that better left to your application?

    file, headers = urllib.urlretrieve(...)
    if os.path.getsize(file) != int(headers.get("content-length", 0)):
        print "oops!"

</F> 



From mal at egenix.com  Fri Aug 27 11:02:05 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Aug 27 11:02:08 2004
Subject: [Python-Dev] Auto-str and auto-unicode in join
In-Reply-To: <412E77EF.80007@iinet.net.au>
References: <002401c48bb5$765463c0$e841fea9@oemcomputer>	<1f7befae04082615206760c806@mail.gmail.com>
	<412E77EF.80007@iinet.net.au>
Message-ID: <412EF88D.2050908@egenix.com>

Nick Coghlan wrote:
> Tim Peters wrote:
> 
>> I needed a break from intractable database problems, and am almost
>> done with PyUnicode_Join().  I'm not doing auto-unicode(), though, so
>> there will still be plenty of fun left for Nick!
> 
> 
> I actually got that mostly working (off slightly out-of-date CVS though).
> 
> Joining a sequence of 10 integers with auto-str seems to take about 60% 
> of the time of a str(x) list comprehension on that same sequence (and 
> the PySequence_Fast call means that a generator is slightly slower than 
> a list comp!). For a sequence which mixed strings and non-strings, the 
> gains could only increase.
> 
> However, there is one somewhat curly problem I'm not sure what to do about.
> 
> To avoid slowing down the common case of string join (a list of only 
> strings) it is necessary to do the promotion to string in the type-check 
> & size-calculation pass.
> 
> That's fine in the case of a list that consists of only strings and 
> non-basestrings, or the case of a unicode separator - every 
> non-basestring is converted using either PyObject_Str or PyObject_Unicode.
> 
> Where it gets weird is something like this:
>     ''.join([an_int, a_unicode_str])
>     u''.join([an_int, a_unicode_str])

This gives you a TypeError, so it's a non-issue (.join() does
not do an implicit call to str(obj) on the list elements).

The real issue is the case where you have [a_str, a_unicode_obj]
and for that the current implementation already does the right
thing, namely to look for Unicode objects in the length checking pass.

> In the first case, the int will first be converted to a string via 
> PyObject_Str, and then that string representation is what will get 
> converted to Unicode after the detection of the unicode string causes 
> the join to be handed over to Unicode join.
> 
> In the latter case, the int is converted directly to Unicode.
> 
> So my question would be, is it reasonable to expect that 
> PyObject_Unicode(PyObject_Str(some_object)) give the same answer as 
> PyObject_Unicode(some_object)?
 >
> If not, then the string join would have to do something whereby it kept 
> a 'pristine' version of the sequence around to hand over to the Unicode 
> join.
> 
> My first attempt at implementing this feature had that property, but 
> also had the effect of introducing about a 1% slowdown of the standard 
> sequence-of-strings case (it introduced an extra if statement to see if 
> a 'stringisation' pass was required after the initial type checking and 
> sizing pass). For longer sequences than 10 strings, I imagine the 
> relative slowdown would be much less.
> 
> Hmm. . . I think I see a way to implement this, while still avoiding 
> adding any code to the standard path through the function. It'd be 
> slower for the case where an iterator is passed in, and we automatically 
> invoke PyObject_Str but don't end up delegating to Unicode join, though, 
> as it involves making a copy of the sequence that only gets used if the 
> Unicode join is invoked. (If the original object is a real sequence, 
> rather than an iterator, there is no extra overhead - we have to make 
> the copy anyway, to avoid mutating the user's sequence).
> 
> If people are definitely interested in this feature, I could probably 
> put a patch together next week.
> 
> Regards,
> Nick.
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/mal%40egenix.com

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 27 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Fri Aug 27 11:20:50 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Aug 27 11:20:52 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
 unicodeobject.c, 2.219, 2.220
In-Reply-To: <E1C0Vrz-00040g-H0@sc8-pr-cvs1.sourceforge.net>
References: <E1C0Vrz-00040g-H0@sc8-pr-cvs1.sourceforge.net>
Message-ID: <412EFCF2.4030608@egenix.com>

tim_one@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/dist/src/Objects
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15103/Objects
> 
> Modified Files:
> 	unicodeobject.c 
> Log Message:
> PyUnicode_Join():  Two primary aims:
> 
> 1. u1.join([u2]) is u2
> 2. Be more careful about C-level int overflow.
> 
> Since PySequence_Fast() isn't needed to achieve #1, it's not used -- but
> the code could sure be simpler if it were.

Hmm, you've now made PyUnicode_Join() to work with iterators
whereas PyString_Join() only works for sequences.

What are the performance implications of this for PyUnicode_Join() ?

Since the string and Unicode implementations have to be in sync,
we'd also need to convert PyString_Join() to work on iterators.

Which brings up the second question:
What are the performance implications of this for PyString_Join() ?

The join operation is a widely used method, so both implementations
need to be as fast as possible. It may be worthwhile making the
PySequence_Fast() approach a special case in both routines and
using the iterator approach as fallback if no sequence is found.

Note that PyString_Join() with iterator support will also
have to be careful about not trying to iterate twice, so
it will have to use a similiar logic to the one applied
in PyString_Format() where the work already done up to the
point where it finds a Unicode string is reused when calling
PyUnicode_Format().

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 27 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From aahz at pythoncraft.com  Fri Aug 27 15:56:50 2004
From: aahz at pythoncraft.com (Aahz)
Date: Fri Aug 27 15:56:53 2004
Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple
	String Substitutions
In-Reply-To: <002301c48bb5$17bfbe40$e841fea9@oemcomputer>
References: <002301c48bb5$17bfbe40$e841fea9@oemcomputer>
Message-ID: <20040827135650.GB25839@panix.com>

On Thu, Aug 26, 2004, Raymond Hettinger wrote:
>
> * doesn't force result to unicode

This is the main reason I'm +0, pending further arguments.  OTOH, I also
like using %, so you'd have to come up with more points to move me
beyond +0.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From fumanchu at amor.org  Fri Aug 27 16:14:23 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Fri Aug 27 16:19:55 2004
Subject: [Python-Dev] J2 proposal final
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3022E91@exchange.hqamor.amorhq.net>

Hello, Guido,

The J2 patch and proposal are finished
(http://www.aminus.org/rbre/python/pydec.html) and ready for your
consideration. In the interest of time, I'm submitting the arguments to
you now. The "call for signatories" is still open, but I think you can
get a feel for the level of support based upon the current endorsements,
and modify that as more names come in.

I'll be in Ensenada (at a friend's bachelor party, if you must know :)
until Sunday evening, at which time I will add in any new signatories.
If you have any questions until that time, I'm sure Michael Sparks
(among others) would be happy to answer or delegate them. Your decision
schedule is of course your own.

Thanks for your consideration and support throughout this process! For
the Python community,


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org


P.S. For the record, I had selected and am now submitting the J2 syntax
based on a community vote on c.l.p., wherein each voter cast 3 votes,
for a total of 70 voters with 209 votes. The top contenders as of today
(a good time to officially close that vote):

	1st	2nd	3rd	Total	%
A1	4	2	3	9	
C1	9	9	12	30	0.14
C2	7	7	2	16	
J2	33	34	30	97	0.46

The total number of votes for J2 beat the total number of votes for C1
(the next contender) by over three to one. It was unclear to many in
that poll whether A1 was a valid vote; it's not recommended to draw any
conclusions regarding the preference of voters for A1 from that
vote--that may be more correctly drawn from the list of signatories of
the proposal.
From tim.peters at gmail.com  Fri Aug 27 16:30:27 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Fri Aug 27 16:30:30 2004
Subject: [Python-Dev] Re: Re: [Python-checkins] python/dist/src/Objects
	unicodeobject.c, 2.219, 2.220
In-Reply-To: <412EFCF2.4030608@egenix.com>
References: <E1C0Vrz-00040g-H0@sc8-pr-cvs1.sourceforge.net>
	<412EFCF2.4030608@egenix.com>
Message-ID: <1f7befae040827073056affd4d@mail.gmail.com>

[M.-A. Lemburg]
> Hmm, you've now made PyUnicode_Join() to work with iterators
> whereas PyString_Join() only works for sequences.

They have both worked with iterators since the release in which
iterators were introduced.  Nothing changed now in this respect.

> What are the performance implications of this for PyUnicode_Join() ?

None.

> Since the string and Unicode implementations have to be in sync,
> we'd also need to convert PyString_Join() to work on iterators.

It already does.  I replied earlier this week on the same topic --
maybe you didn't see that, or maybe you misunderstand what
PySequence_Fast does.

> Which brings up the second question:
> What are the performance implications of this for PyString_Join() ?

None.

> The join operation is a widely used method, so both implementations
> need to be as fast as possible. It may be worthwhile making the
> PySequence_Fast() approach a special case in both routines and
> using the iterator approach as fallback if no sequence is found.

string_join uses PySequence_Fast already; the Unicode join didn't, and
still doesn't.  In the cases of exact list or tuple arguments,
PySequence_Fast would be quicker in Unicode join.  But in any cases
other than those,  PySequence_Fast materializes a concrete tuple
containing the full materialized iteration, so could be more
memory-consuming.  That's probably a good tradeoff, though.

> Note that PyString_Join() with iterator support will also
> have to be careful about not trying to iterate twice,

It already is.  Indeed, the primary reason it uses PySequence_Fast is
to guarantee that it never iterates over an iterator argument more
than once.  The Unicode join doesn't have that potential problem.

> so it will have to use a similiar logic to the one applied
> in PyString_Format() where the work already done up to the
> point where it finds a Unicode string is reused when calling
> PyUnicode_Format().

>>> def g():
...     for piece in 'a', 'b', u'c', 'd': # force Unicode promotion on 3rd yield
...         yield piece
...
>>> ' '.join(g())
u'a b c d'
>>>
From mal at egenix.com  Fri Aug 27 16:47:49 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Aug 27 16:47:54 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
 unicodeobject.c, 2.219, 2.220
In-Reply-To: <1f7befae040827073056affd4d@mail.gmail.com>
References: <E1C0Vrz-00040g-H0@sc8-pr-cvs1.sourceforge.net>	<412EFCF2.4030608@egenix.com>
	<1f7befae040827073056affd4d@mail.gmail.com>
Message-ID: <412F4995.6080606@egenix.com>

Tim Peters wrote:
> [M.-A. Lemburg]
> 
>>Hmm, you've now made PyUnicode_Join() to work with iterators
>>whereas PyString_Join() only works for sequences.
> 
> 
> They have both worked with iterators since the release in which
> iterators were introduced.  Nothing changed now in this respect.
> 
 >
>>What are the performance implications of this for PyUnicode_Join() ?
> 
> 
> None.
> 
> 
>>Since the string and Unicode implementations have to be in sync,
>>we'd also need to convert PyString_Join() to work on iterators.
> 
> 
> It already does.  I replied earlier this week on the same topic --
> maybe you didn't see that, or maybe you misunderstand what
> PySequence_Fast does.

Indeed. At the time Fredrik added this API, it was optimized
for lists and tuples and had a fallback mechanism for arbitrary
sequences. Didn't know that it now also works for iterators. Nice !

>>Which brings up the second question:
>>What are the performance implications of this for PyString_Join() ?
> 
> 
> None.
> 
> 
>>The join operation is a widely used method, so both implementations
>>need to be as fast as possible. It may be worthwhile making the
>>PySequence_Fast() approach a special case in both routines and
>>using the iterator approach as fallback if no sequence is found.
> 
> 
> string_join uses PySequence_Fast already; the Unicode join didn't, and
> still doesn't.  In the cases of exact list or tuple arguments,
> PySequence_Fast would be quicker in Unicode join.  But in any cases
> other than those,  PySequence_Fast materializes a concrete tuple
> containing the full materialized iteration, so could be more
> memory-consuming.  That's probably a good tradeoff, though.

Indeed. I'd opt for going the PySequence_Fast() way
for Unicode as well.

>>Note that PyString_Join() with iterator support will also
>>have to be careful about not trying to iterate twice,
> 
> 
> It already is.  Indeed, the primary reason it uses PySequence_Fast is
> to guarantee that it never iterates over an iterator argument more
> than once.  The Unicode join doesn't have that potential problem.
> 
> 
>>so it will have to use a similiar logic to the one applied
>>in PyString_Format() where the work already done up to the
>>point where it finds a Unicode string is reused when calling
>>PyUnicode_Format().
> 
> 
>>>>def g():
> 
> ...     for piece in 'a', 'b', u'c', 'd': # force Unicode promotion on 3rd yield
> ...         yield piece
> ...
> 
>>>>' '.join(g())
> 
> u'a b c d'

Nice :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 27 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From anthony at interlink.com.au  Fri Aug 27 17:28:04 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug 27 17:28:24 2004
Subject: [Python-Dev] J2 proposal final
In-Reply-To: <3A81C87DC164034AA4E2DDFE11D258E3022E91@exchange.hqamor.amorhq.net>
References: <3A81C87DC164034AA4E2DDFE11D258E3022E91@exchange.hqamor.amorhq.net>
Message-ID: <412F5304.1040105@interlink.com.au>

Robert Brewer wrote:
> Hello, Guido,
> 
> The J2 patch and proposal are finished
> (http://www.aminus.org/rbre/python/pydec.html) and ready for your
> consideration. In the interest of time, I'm submitting the arguments to
> you now. The "call for signatories" is still open, but I think you can
> get a feel for the level of support based upon the current endorsements,
> and modify that as more names come in.


The patch from Michael Sparks is at http://www.python.org/sf/1013835


Anthony
From fredrik at pythonware.com  Fri Aug 27 17:30:31 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri Aug 27 17:28:49 2004
Subject: [Python-Dev] Re: Re: def fn (args) [dec,dec]:
References: <41136453.1040908@interlink.com.au><200408061445.i76EjqT07026@guido.python.org>
	<e8bf7a5304080607524990942d@mail.gmail.com>
Message-ID: <cgnjvd$l17$1@sea.gmane.org>

Jeremy Hylton wrote:

>> (If I could design a language from scratch, I might want to move
>> docstrings to before the 'def' as well; I've come to appreciate this
>> in Java code.  Fredrik Lundh's PythonDoc also uses comments prefixing
>> the method definition; he also uses @ so he should be happy with this
>> syntax, as he seemed to be when I mentioned it to him at EuroPython.)
>
> It's not at all clear that Fredrik likes it, but he doesn't participate
> in Python development any more so we may never know :-(.

you can ask.  I do read mail, eventually. ;-)

</F> 



From jhylton at gmail.com  Fri Aug 27 18:30:17 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Fri Aug 27 18:30:55 2004
Subject: [Python-Dev] Re: list += string??
In-Reply-To: <u8yc1xzmp.fsf@boost-consulting.com>
References: <ur7pu2da8.fsf@boost-consulting.com>
	<1f7befae04082607303b7bc26e@mail.gmail.com>
	<u8yc22bfx.fsf@boost-consulting.com>
	<u8yc1xzmp.fsf@boost-consulting.com>
Message-ID: <e8bf7a53040827093017faaec8@mail.gmail.com>

On Thu, 26 Aug 2004 19:12:30 -0400, David Abrahams
<dave@boost-consulting.com> wrote:
> Report from the field: I just discovered a bug that's been hiding in
> my code for a few months.  The code was being exercised, but the
> result was the same as if the code had been correct until I threw
> some new examples at it.  The problem was:
> 
>    self.body[e[0]:e[1]] = s
> 
> self.body is a list of strings, and s is a string.  Later
> ''.join(self.body) gets called, so for many cases the bug was hidden.
> No offense intended (okay, maybe a tiny little offense), but the
> experience here is a bit Perl-like.

It's a fair complaint.  I've made the mistake myself, and it usually
takes a few head scratching and investigation before I realize that
I'm passing a string instead of some other sequence.  On the other
hand, I like slicing lists and occasionally I iterate over their
characters.  What's the alternative?  A typecheck seems like it would
be handy, but I'm not sure what type I would want "all sequences
except strings"?

Jeremy
From python at rcn.com  Fri Aug 27 19:37:11 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Aug 27 19:37:48 2004
Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple String
	Substitutions
Message-ID: <000601c48c5c$7e88f040$e841fea9@oemcomputer>

> > * doesn't force result to unicode

[Aahz]
> This is the main reason I'm +0, pending further arguments.

Employing class logic for a function problem is my main reason.  Any
function can be wrapped in class logic and use an overloaded operator
(string.template for example) -- it's invariably the wrong thing to do
for reasons of complexity, separation of instantiation and application,
and the issues that always arise when an operator is overloaded (a good
technique that should only be applied sparingly).



[Aahz]
>  OTOH, I also
> like using %, so you'd have to come up with more points to move me
> beyond +0.

The reasons center around the remoteness of instantiation from
application, the differences from current uses of %, and an issue with
the % operator itself.

When application is remote from instantiation (for instance, when a
template argument is supplied to a function), we get several problems.
Given a line like "r = t % m", it is hard to verify that the code is
correct.  Should there be KeyError trapping?  Can m be a tuple? Is t a
%template, a safetemplate, or dollartemplate?  Is the result a str or
Unicode object?  Should the decision of safe vs regular substitution be
made at the point of instantiation or application?

The temptation will be to reuse existing code that uses the % operator
which unfortunately is not the same (especially with respect to applying
tuples, return types, and auto-stringizing).  The % operator is hard to
search for in the docs and has precedence issues arising from its
primary use for modulo arithemetic.  Also, using a function makes it
possible to use both % formatting and $ formatting (I do have use a case
for this).  Further, the % operator mnemonically only makes sense with %
identifier tags -- it makes less sense with $ tags.

Whatever answer is chosen, it should be forward looking and consider
that others will want to add functionality (local namespace lookups for
example) and that some poor bastards are going to waste time figuring
out how to subclass the current implementation.  In time, there will
likely be more than two of these -- do you want more classes or more
functions?



Raymond

From anthony at interlink.com.au  Fri Aug 27 19:49:06 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Aug 27 19:49:29 2004
Subject: [Python-Dev] PEP-0273 (zip imports) needs to be finished off
Message-ID: <412F7412.4050001@interlink.com.au>

I just noticed that PEP-0273 is not yet up to date. As the
code's been in there since 2.3, it'd be a shame if we had to
rip it out of 2.4a3 for lack of a current PEP <wink>

And yes, I've not had time to finish PEP-0318. This _will_ be
done before 2.4a3. Hopefully _this_ weekend I'll get time to
finish it...

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From dan.gass at gmail.com  Fri Aug 27 20:06:16 2004
From: dan.gass at gmail.com (Dan Gass)
Date: Fri Aug 27 20:06:20 2004
Subject: [Python-Dev] HTML Side By Side Differencing -- Alpha3
Message-ID: <bc15ba280408271106721cef48@mail.gmail.com>

I'm looking for a developer to apply this patch for inclusion in
Alpha3.  The patch provides methods to generate side by side
differences in HTML format (highlights and hyperlinks changed areas). 
The patch adds functionality to the difflib.py module without changing
any of the existing code (low risk of breaking anything).  There has
been a good deal of participation in improving and reviewing the
patch.

What I need is someone who has check in privileges and feels
comfortable implementing the patch.  I've had good support (see patch
comments) from a number of people with the exception of some
hesitation from Martin v. L?wis which was countered by arguments from
Tim Peters and Jim Jewett.

In addition to having a wider audience to solicit help from, this is
being posted on Python-Dev to provide an opportunity for others to
comment on the patch as well since it is being considered for
inclusion in Alpha3.

For that person who steps forward I can promise that I will provide
any needed support in a timely manner.

Thanks,
Dan Gass
From dan.gass at gmail.com  Fri Aug 27 20:18:55 2004
From: dan.gass at gmail.com (Dan Gass)
Date: Fri Aug 27 20:19:01 2004
Subject: [Python-Dev] HTML Side By Side Differencing -- Alpha3
In-Reply-To: <bc15ba280408271106721cef48@mail.gmail.com>
References: <bc15ba280408271106721cef48@mail.gmail.com>
Message-ID: <bc15ba2804082711182b065866@mail.gmail.com>

Sorry,

I forgot to include the link:

Python Patch #914575
(https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914575&group_id=5470)
From pythondev at bitfurnace.com  Fri Aug 27 20:24:34 2004
From: pythondev at bitfurnace.com (damien morton)
Date: Fri Aug 27 20:29:08 2004
Subject: [Python-Dev] Re: list += string??
In-Reply-To: <e8bf7a53040827093017faaec8@mail.gmail.com>
References: <ur7pu2da8.fsf@boost-consulting.com>	<1f7befae04082607303b7bc26e@mail.gmail.com>	<u8yc22bfx.fsf@boost-consulting.com>	<u8yc1xzmp.fsf@boost-consulting.com>
	<e8bf7a53040827093017faaec8@mail.gmail.com>
Message-ID: <412F7C62.8090508@bitfurnace.com>

An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040827/7e493a75/attachment.html
From martin at v.loewis.de  Fri Aug 27 21:14:50 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Aug 27 21:14:52 2004
Subject: [Python-Dev] Re: list += string??
In-Reply-To: <412F7C62.8090508@bitfurnace.com>
References: <ur7pu2da8.fsf@boost-consulting.com>	<1f7befae04082607303b7bc26e@mail.gmail.com>	<u8yc22bfx.fsf@boost-consulting.com>	<u8yc1xzmp.fsf@boost-consulting.com>	<e8bf7a53040827093017faaec8@mail.gmail.com>
	<412F7C62.8090508@bitfurnace.com>
Message-ID: <412F882A.4010604@v.loewis.de>

damien morton wrote:
> Maybe this is the time to thow out the idea that strings arent 
> sequences. Not backwards comaptable, but strings might grow a 
> string.chars property that _is_ a sequence, but strings themselves arent 
> sequences.

I very much doubt users will accept such a change - especially if the
rationale is that it is made to prevent a certain class of errors; the
change itself is likely to *introduce* new errors, both for changing
the existing code, and for the feature itself. We would need to come
up with a type for string.chars, and people would pass *that* around,
recreating the original problem.

I would hope that pychecker could diagnose this kind of problem in many
cases.

Regards,
Martin
From pf_moore at yahoo.co.uk  Fri Aug 27 22:12:04 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Fri Aug 27 22:11:53 2004
Subject: [Python-Dev] Re: PEP-0273 (zip imports) needs to be finished off
References: <412F7412.4050001@interlink.com.au>
Message-ID: <ud61c49yj.fsf@yahoo.co.uk>

Anthony Baxter <anthony@interlink.com.au> writes:

> I just noticed that PEP-0273 is not yet up to date. As the
> code's been in there since 2.3, it'd be a shame if we had to
> rip it out of 2.4a3 for lack of a current PEP <wink>

I'm not sure how (directly) relevant PEP 273 is any more. It
documented Jim Ahlstrom's approach, which was superseded by PEP 302's
import hooks, and the zipimport module that uses them.

I believe that PEP 302 is more or less up to date (there is
discussion of a "Phase 2" rationalisation in the PEP, which hasn't
been implemented, but IIRC that is mainly an internal detail and
doesn't affect the user-visible interface). I'll do a quick check of
the PEP against reality, but if there are mismatches, I can't offer
to fix them (Just was the coding wizard, I just wrote some of the
blurb).

There needs to be some documentation of the zipimport module,
certainly. I'm not sure if that should be in PEP 273 (zipimport is
acknowledged in that PEP, but as a competitor rather than an
implementation), in a separate PEP, or as a part of PEP 203 (with PEP
273 marked as rejected in the latter 2 cases). Personally, I wouldn't
feel comfortable modifying PEP 273 (it would be more or less a
complete rewrite) without Jim's agreement.

In practice, my preference would be to write a section for the
documentation covering all of this, and just put a prominent pointer
at the top of PEP 273 - something like "A variation on this proposal
was implemented in Python 2.3. For details see section XXX of the
documentation".

But where should this go? Even packages aren't too well documented
(see the "XXX Can't be bothered to spell this out" comment at the end
of section 6.12 of the language reference). Would a new section
6.12.2 "Import Semantics" be appropriate? Hmm, that would be my
preference. I'll see if I can work up a documentation patch...

Always-promising-more-than-I-can-deliver-ly y'rs
Paul.
-- 
The brain is a wonderful organ. It starts working the moment you get
up in the morning and does not stop until you get into the office. --
Robert Frost

From zathras at thwackety.com  Fri Aug 27 23:53:38 2004
From: zathras at thwackety.com (Michael Sparks)
Date: Fri Aug 27 23:31:34 2004
Subject: [Python-Dev] J2 proposal final
In-Reply-To: <412F5304.1040105@interlink.com.au>
Message-ID: <Pine.LNX.4.44.0408272242040.24914-100000@pingu.thwackety.com>

On Sat, 28 Aug 2004, Anthony Baxter wrote:
> Robert Brewer wrote:
> > Hello, Guido,
> >
> > The J2 patch and proposal are finished
> > (http://www.aminus.org/rbre/python/pydec.html) and ready for your
> > consideration. In the interest of time, I'm submitting the arguments to
> > you now. The "call for signatories" is still open, but I think you can
> > get a feel for the level of support based upon the current endorsements,
> > and modify that as more names come in.
>
>
> The patch from Michael Sparks is at http://www.python.org/sf/1013835

Hopefully it goes without saying, but obviously I'll be willing to make
any necessary changes if/as required. The implementation is deliberately
simplistic - hopefully it covers all the areas that would be expected.

Best Regards,


Michael.

From pje at telecommunity.com  Fri Aug 27 23:37:56 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Aug 27 23:37:36 2004
Subject: [Python-Dev] PEP 333: Python Web Server Gateway Interface v1.0
Message-ID: <5.1.1.6.0.20040827173709.025f5c50@mail.telecommunity.com>

(Separately posted to python-list to avoid cross-traffic.)

PEP: 333
Title: Python Web Server Gateway Interface v1.0
Version: $Revision: 1.1 $
Last-Modified: $Date: 2004/08/27 17:30:09 $
Author: Phillip J. Eby <pje at telecommunity.com>
Discussions-To: Python Web-SIG <web-sig at python.org>
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 07-Dec-2003
Post-History: 07-Dec-2003, 08-Aug-2004, 20-Aug-2004, 27-Aug-2004


Abstract
========

This document specifies a proposed standard interface between web
servers and Python web applications or frameworks, to promote web
application portability across a variety of web servers.


Rationale and Goals
===================

Python currently boasts a wide variety of web application frameworks,
such as Zope, Quixote, Webware, SkunkWeb, PSO, and Twisted Web -- to
name just a few [1]_.  This wide variety of choices can be a problem
for new Python users, because generally speaking, their choice of web
framework will limit their choice of usable web servers, and vice
versa.

By contrast, although Java has just as many web application frameworks
available, Java's "servlet" API makes it possible for applications
written with any Java web application framework to run in any web
server that supports the servlet API.

The availability and widespread use of such an API in web servers for
Python -- whether those servers are written in Python (e.g. Medusa),
embed Python (e.g. mod_python), or invoke Python via a gateway
protocol (e.g. CGI, FastCGI, etc.) -- would separate choice of
framework from choice of web server, freeing users to choose a pairing
that suits them, while freeing framework and server developers to
focus on their preferred area of specialization.

This PEP, therefore, proposes a simple and universal interface between
web servers and web applications or frameworks: the Python Web Server
Gateway Interface (WSGI).

But the mere existence of a WSGI spec does nothing to address the
existing state of servers and frameworks for Python web applications.
Server and framework authors and maintainers must actually implement
WSGI for there to be any effect.

However, since no existing servers or frameworks support WSGI, there
is little immediate reward for an author who implements WSGI support.
Thus, WSGI *must* be easy to implement, so that an author's initial
investment in the interface can be reasonably low.

Thus, simplicity of implementation on *both* the server and framework
sides of the interface is absolutely critical to the utility of the
WSGI interface, and is therefore the principal criterion for any
design decisions.

Note, however, that simplicity of implementation for a framework
author is not the same thing as ease of use for a web application
author.  WSGI presents an absolutely "no frills" interface to the
framework author, because bells and whistles like response objects and
cookie handling would just get in the way of existing frameworks'
handling of these issues.  Again, the goal of WSGI is to facilitate
easy interconnection of existing servers and applications or
frameworks, not to create a new web framework.

Note also that this goal precludes WSGI from requiring anything that
is not already available in deployed versions of Python.  Therefore,
new standard library modules are not proposed or required by this
specification, and nothing in WSGI requires a Python version greater
than 2.2.2.  (It would be a good idea, however, for future versions
of Python to include support for this interface in web servers
provided by the standard library.)

In addition to ease of implementation for existing and future
frameworks and servers, it should also be easy to create request
preprocessors, response postprocessors, and other WSGI-based
"middleware" components that look like an application to their
containing server, while acting as a server for their contained
applications.

If middleware can be both simple and robust, and WSGI is widely
available in servers and frameworks, it allows for the possibility
of an entirely new kind of Python web application framework: one
consisting of loosely-coupled WSGI middleware components.  Indeed,
existing framework authors may even choose to refactor their
frameworks' existing services to be provided in this way, becoming
more like libraries used with WSGI, and less like monolithic
frameworks.  This would then allow application developers to choose
"best-of-breed" components for specific functionality, rather than
having to commit to all the pros and cons of a single framework.

Of course, as of this writing, that day is doubtless quite far off.
In the meantime, it is a sufficient short-term goal for WSGI to
enable the use of any framework with any server.

Finally, it should be mentioned that the current version of WSGI
does not prescribe any particular mechanism for "deploying" an
application for use with a web server or server gateway.  At the
present time, this is necessarily implementation-defined by the
server or gateway.  After a sufficient number of servers and
frameworks have implemented WSGI to provide field experience with
varying deployment requirements, it may make sense to create
another PEP, describing a deployment standard for WSGI servers and
application frameworks.


[Remainder truncated due to python-dev posting size limit; please see:

http://python.org/peps/pep-0333.html

for remaining text.]

From aahz at pythoncraft.com  Sat Aug 28 01:39:58 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sat Aug 28 01:40:01 2004
Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple
	String Substitutions
In-Reply-To: <000601c48c5c$7e88f040$e841fea9@oemcomputer>
References: <000601c48c5c$7e88f040$e841fea9@oemcomputer>
Message-ID: <20040827233958.GA5560@panix.com>

On Fri, Aug 27, 2004, Raymond Hettinger wrote:
>
> When application is remote from instantiation (for instance, when a
> template argument is supplied to a function), we get several problems.
> Given a line like "r = t % m", it is hard to verify that the code is
> correct.  Should there be KeyError trapping?  Can m be a tuple? Is t a
> %template, a safetemplate, or dollartemplate?  Is the result a str or
> Unicode object?  Should the decision of safe vs regular substitution be
> made at the point of instantiation or application?

All right, this moves me to +1, in the absence of any good arguments in
favor of keeping the class-based system.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From anthony at interlink.com.au  Sat Aug 28 09:08:55 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Aug 28 09:09:10 2004
Subject: [Python-Dev] __mangled in stdlib considered poor form
Message-ID: <41302F87.1000805@interlink.com.au>

I'd like to see all occurrances of __mangledName in the stdlib
removed and replaced with just _mangledName.

Rationale:

   The double-under "private" names are a hack. They don't really
protect variables, except against accidental stomping by subclasses.
Looking at the uses in the std lib, they're not being used for that -
they're being used for internal details. A single underscore is
sufficient for that.

   The mangled names cause problems when someone _needs_ to override
the implementation details in a subclass - I can recall a case with
ConfigParser in SpamBayes, where the end result was the subclass with
code like __ConfigParser_foo (a subsequent release of Python fixed
that instance of double-under madness). Yes, you shouldn't be fiddling
with internals of a class in a base class, and any time you do this,
you run the risk of being refactored into oblivion, but sometimes it's
necessary.

   Double-under mangled names are the worst of both worlds - they don't
actually protect the variables, but they force someone who needs to
poke into the class to use a mangled name.

   Given that part of the purpose of the stdlib is to act as a sort of
"best practice" example of Python code, I'd like to see all instances
of it in the std library removed.

   I'm not sure if it's feasible to attack this before 2.4b1, or if it
should wait until 2.5, but I _would_ like to see it happen.

   What sayeth the rest of python-dev?

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From mwh at python.net  Sat Aug 28 13:36:44 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Aug 28 13:36:48 2004
Subject: [Python-Dev] __mangled in stdlib considered poor form
In-Reply-To: <41302F87.1000805@interlink.com.au> (Anthony Baxter's message
	of "Sat, 28 Aug 2004 17:08:55 +1000")
References: <41302F87.1000805@interlink.com.au>
Message-ID: <2md61b5wab.fsf@starship.python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

> I'd like to see all occurrances of __mangledName in the stdlib
> removed and replaced with just _mangledName.

[...]

>    What sayeth the rest of python-dev?

+1.

Make sure you get unittest.py too.

Cheers,
mwh

-- 
  <MFen> want to write my requirements for me?
  <radix> Sure!
  <radix> "show a dancing monkey in the about box"
                                                -- from Twisted.Quotes
From fredrik at pythonware.com  Sat Aug 28 11:19:36 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat Aug 28 19:58:21 2004
Subject: [Python-Dev] Re: python/dist/src/Lib/lib-tk tkFont.py,1.6,1.7
Message-ID: <200408280917.i7S9Hqk10867@pythonware.com>

loewis@users.sourceforge.net wrote:

> Expand tabs.
> 
> Index: tkFont.py
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Lib/lib-tk/tkFont.py,v
> retrieving revision 1.6
> retrieving revision 1.7
> diff -C2 -d -r1.6 -r1.7
> *** tkFont.py	18 Aug 2004 11:06:44 -0000	1.6
> --- tkFont.py	18 Aug 2004 17:47:40 -0000	1.7
> ***************
> *** 92,96 ****
>               # if font config info supplied, apply it
>               if font:
> !             	print "font=%r" % font
>                   root.tk.call("font", "configure", self.name, *font)
>           else:
> --- 92,96 ----
>               # if font config info supplied, apply it
>               if font:
> !                 print "font=%r" % font
>                   root.tk.call("font", "configure", self.name, *font)
>           else:

ahem.  tab or not, that sure looks like a debugging statement, right?

</F>

From gvanrossum at gmail.com  Sat Aug 28 20:50:45 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sat Aug 28 20:50:54 2004
Subject: [Python-Dev] __mangled in stdlib considered poor form
In-Reply-To: <2md61b5wab.fsf@starship.python.net>
References: <41302F87.1000805@interlink.com.au>
	<2md61b5wab.fsf@starship.python.net>
Message-ID: <ca471dc204082811505b76e6f5@mail.gmail.com>

> > I'd like to see all occurrances of __mangledName in the stdlib
> > removed and replaced with just _mangledName.
> 
> [...]
> 
> >    What sayeth the rest of python-dev?
> 
> +1.

+0.5 here. Let's first get the terminology straight: __foo is a
private name, _Class__foo is the mangled form of that. Whenever the
stdlib uses a mangled name, clearly what ought to be done is change
the private name to a single underscore. (I don't want to call that
protected, because that has a different meaning in Java etc.)

There's one potential use case where I'd like to keep the private name
even if there's a mangled use of it nearby: when the private name
really is intended to be private, and the mangled use is a "friend" in
the C++ sense. Sometimes (though rarely) the stdlib has to indulge in
practices that aren't good example code, because it's the stdlib, and
it should implement functionality in the most bulletproof way
possible. Using private names for internal details is the right thing
sometimes.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From python at rcn.com  Sat Aug 28 21:20:37 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sat Aug 28 21:21:26 2004
Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple
	String Substitutions
In-Reply-To: <20040827233958.GA5560@panix.com>
Message-ID: <000501c48d34$19ce9000$e841fea9@oemcomputer>

> > When application is remote from instantiation (for instance, when a
> > template argument is supplied to a function), we get several
problems.
> > Given a line like "r = t % m", it is hard to verify that the code is
> > correct.  Should there be KeyError trapping?  Can m be a tuple? Is t
a
> > %template, a safetemplate, or dollartemplate?  Is the result a str
or
> > Unicode object?  Should the decision of safe vs regular substitution
be
> > made at the point of instantiation or application?

[Aahz]
> All right, this moves me to +1, in the absence of any good arguments
in
> favor of keeping the class-based system.

FWIW, I've loaded the proposed improvements to the sandbox:

   \nondist\sandbox\string\alt292.py

It includes a doctest section that illustrates all of the improvements
in action (appropriate return type, error handling, commenting, code
simplicity, keyword arguments, function API, etc).

I'm not sure what to do next.  Aside from Aahz, no one on python-dev
seems to be interested and Barry appears to have been off-line for a few
days.  Ideally, I would like to get this in a few days ahead of the
alpha release.


Raymond

From skip at pobox.com  Sat Aug 28 22:58:49 2004
From: skip at pobox.com (Skip Montanaro)
Date: Sat Aug 28 22:59:12 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>
References: <Your message of "Tue,
	17 Aug 2004 13:49:21 BST." <2mbrhaaq0e.fsf@starship.python.net>
	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>
Message-ID: <16688.61961.182410.162379@montanaro.dyndns.org>


    >> > Is there any consensus forming on whether bytes() instances are
    >> > mutable or not?
    >> 
    >> Mutable!

    Phillip> So, how will it be different from:

    Phillip>      from array import array

    Phillip>      def bytes(*initializer):
    Phillip>          return array('B',*initializer)

That might well be a decent trial implementation, though it seems that if we
allow strings as initializers we should also allow strings in assignment:

    >>> b = bytes("abc\xff")
    >>> b
    array('B', [97, 98, 99, 255])
    >>> print buffer(b)
    abc?
    >>> b[3] = '\xfe'
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    TypeError: an integer is required
    >>> b += "\xfe"
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    TypeError: can only extend array with array (not "str")

I must admit I'm having a bit of trouble getting past this point with a more
traditional subclass (can array objects not be subclassed?) in part I think
because I don't understand new-style classes very well.  In particular, I
couldn't find a description of __new__, and once I fumbled past that, I
didn't seem to be able to override append() or extend().

Skip
From gvanrossum at gmail.com  Sat Aug 28 23:17:01 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sat Aug 28 23:17:08 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <16688.61961.182410.162379@montanaro.dyndns.org>
References: <Your message of "Tue,
	17 Aug 2004 13:49:21 BST." <2mbrhaaq0e.fsf@starship.python.net>
	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>
	<16688.61961.182410.162379@montanaro.dyndns.org>
Message-ID: <ca471dc204082814176a5ebb42@mail.gmail.com>

> I must admit I'm having a bit of trouble getting past this point with a more
> traditional subclass (can array objects not be subclassed?) in part I think
> because I don't understand new-style classes very well.  In particular, I
> couldn't find a description of __new__, and once I fumbled past that, I
> didn't seem to be able to override append() or extend().

Alas, the array type doesn't seem completely fit for subclassing; I
tried similar things and couldn't gt it to work! Even more mysterious
is that the array implementation appears to support the buffer API and
yet it can't be used as an argument to write(). What's going on?


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From python at rcn.com  Sun Aug 29 00:10:14 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sun Aug 29 00:11:05 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <ca471dc204082814176a5ebb42@mail.gmail.com>
Message-ID: <001901c48d4b$cb86f6e0$e841fea9@oemcomputer>

[GvR]
> Alas, the array type doesn't seem completely fit for subclassing;

Famous last words:  It doesn't look like it would be hard to make
array's subclassable.  I'll work on that this weekend.

Reminder:  there is an outstanding array feature request awaiting your
adjudication:  
    www.python.org/sf/992967



Raymond

From martin at v.loewis.de  Sun Aug 29 01:57:50 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug 29 01:57:51 2004
Subject: [Python-Dev] Re: python/dist/src/Lib/lib-tk tkFont.py,1.6,1.7
In-Reply-To: <200408280917.i7S9Hqk10867@pythonware.com>
References: <200408280917.i7S9Hqk10867@pythonware.com>
Message-ID: <41311BFE.4050405@v.loewis.de>

> ahem.  tab or not, that sure looks like a debugging statement, right?

Right, and it is also gone a while ago.

Martin
From martin at v.loewis.de  Sun Aug 29 02:06:54 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug 29 02:06:56 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <16688.61961.182410.162379@montanaro.dyndns.org>
References: <Your message of "Tue, 17 Aug 2004 13:49:21 BST."
	<2mbrhaaq0e.fsf@starship.python.net>	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>	<200408170134.i7H1YDR14560@guido.python.org>	<4121ABAB.9060605@interlink.com.au>	<200408171045.i7HAjQZ15411@guido.python.org>	<4121E859.8000908@interlink.com.au>	<2mbrhaaq0e.fsf@starship.python.net>	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>
	<16688.61961.182410.162379@montanaro.dyndns.org>
Message-ID: <41311E1E.2040801@v.loewis.de>

Skip Montanaro wrote:
>     >>> b[3] = '\xfe'

I think you meant to write

    b[3] = 0xfe # or byte(0xfe)

here. Assigning to an index always takes the element type in
Python. It's only strings where reading an index returns the
container type.

>     >>> b += "\xfe"

And here, you probably meant to write

    b += bytes("\xfe")

Regards,
Martin
From dima at trit.org  Sun Aug 29 02:42:22 2004
From: dima at trit.org (Dima Dorfman)
Date: Sun Aug 29 02:44:20 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <ca471dc204082814176a5ebb42@mail.gmail.com>
References: <2mbrhaaq0e.fsf@starship.python.net>
	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>
	<16688.61961.182410.162379@montanaro.dyndns.org>
	<ca471dc204082814176a5ebb42@mail.gmail.com>
Message-ID: <20040829004221.GB1593@trit.org>

Guido van Rossum <gvanrossum@gmail.com> wrote:
> Even more mysterious
> is that the array implementation appears to support the buffer API and
> yet it can't be used as an argument to write(). What's going on?

It supports the "read" buffer API but not the "character" buffer API,
so the file has to be opened in binary mode for it to work:

  >>> a = array.array('c', 'fish')
  >>> open('/dev/null', 'w').write(a)
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  TypeError: argument 1 must be string or read-only character buffer, not array.array
  >>> open('/dev/null', 'wb').write(a)
  >>>

That restriction(?) comes from this in file_write:

  PyArg_ParseTuple(args, f->f_binary ? "s#" : "t#", ...

where s# requires a read buffer and t# requires a character buffer.

array.array is the only type in the core that's a read buffer but not
a character buffer, and I can't find any semantic differences between
read and character buffers. If someone can explain the differences or
confirm that there aren't any, I'll make this work. The easiest thing
to do would be to make array support the character buffer API (but
maybe only for [cbBu] types?).

Dima.
From tim.peters at gmail.com  Sun Aug 29 03:51:48 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Aug 29 03:51:51 2004
Subject: [Python-Dev] Auto-str and auto-unicode in join
In-Reply-To: <412E77EF.80007@iinet.net.au>
References: <002401c48bb5$765463c0$e841fea9@oemcomputer>
	<1f7befae04082615206760c806@mail.gmail.com>
	<412E77EF.80007@iinet.net.au>
Message-ID: <1f7befae040828185146444b6d@mail.gmail.com>

If we were to do auto-str, it would be better to rewrite str.join() as
a 1-pass algorithm, using the kind of "double allocated space as
needed" gimmick unicode.join uses.  It would be less efficient if
auto-promotion to Unicode turns out to be required, but it's hard to
measure how little I care about that; it might be faster if auto-str
and Unicode promotion aren't needed (as only 1 pass would be needed).

auto-str couldn't really *mean* string.join(map(str, seq)) either. 
The problem with the latter is that if a seq element x is a unicode
instance, str(x) will convert it into an encoded (8-bit) str, which
would not be backward compatible.  So the logic would be more (in
outline):

class string:
    def join(self, seq):
        seq = PySequence_Fast(seq)
        if seq is NULL:
            return NULL

        if len(seq) == 0:
            return ""
        elif len(seq) == 1 and type(seq[0]) is str:
            return seq[0]

        allocate a string object with (say) 100 bytes of space
        let p point to the first free byte

        for x in seq:
            if type(x) is str:
                copy x's guts into p, getting more space if needed
            elif isinstance(x, unicode):
                return unicode,join(self, seq)
            else:
                x = PyObject_Str(x)
                if x is NULL:
                    return NULL
                copy x's guts into p, etc

            if not the last element:
                copy the separator's guts into p, etc

        cut p back to the space actually used
        return p's string object

Note a peculiarity:  if x is neither str nor unicode, but has a
__str__ or __repr__ method that returns a unicode object,
PyObject_Str() will convert that into an 8-bit str.  That may be
surprising.  It would be ugly to duplicate most of the logic from
PyObject_Unicode() to try to guess whether there's "a natural" Unicode
spelling of x.  I think I'd rather say "tough luck -- use unicode.join
if that's what you want".
From gvanrossum at gmail.com  Sun Aug 29 03:53:47 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sun Aug 29 03:53:50 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <20040829004221.GB1593@trit.org>
References: <2mbrhaaq0e.fsf@starship.python.net>
	<200408170113.i7H1DWSo021301@cosc353.cosc.canterbury.ac.nz>
	<200408170134.i7H1YDR14560@guido.python.org>
	<4121ABAB.9060605@interlink.com.au>
	<200408171045.i7HAjQZ15411@guido.python.org>
	<4121E859.8000908@interlink.com.au>
	<2mbrhaaq0e.fsf@starship.python.net>
	<5.1.1.6.0.20040817114242.02c9dec0@mail.telecommunity.com>
	<16688.61961.182410.162379@montanaro.dyndns.org>
	<ca471dc204082814176a5ebb42@mail.gmail.com>
	<20040829004221.GB1593@trit.org>
Message-ID: <ca471dc2040828185310ab434f@mail.gmail.com>

> array.array is the only type in the core that's a read buffer but not
> a character buffer, and I can't find any semantic differences between
> read and character buffers. If someone can explain the differences or
> confirm that there aren't any, I'll make this work. The easiest thing
> to do would be to make array support the character buffer API (but
> maybe only for [cbBu] types?).

Ah, that makes sense. And no please, keep it that way. It's compatible
with the idea that bytes arrays should be read from / written to
binary files.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From mal at egenix.com  Sun Aug 29 13:21:14 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Sun Aug 29 13:21:22 2004
Subject: [Python-Dev] Auto-str and auto-unicode in join
In-Reply-To: <1f7befae040828185146444b6d@mail.gmail.com>
References: <002401c48bb5$765463c0$e841fea9@oemcomputer>	<1f7befae04082615206760c806@mail.gmail.com>	<412E77EF.80007@iinet.net.au>
	<1f7befae040828185146444b6d@mail.gmail.com>
Message-ID: <4131BC2A.9060206@egenix.com>

Tim Peters wrote:
> If we were to do auto-str, it would be better to rewrite str.join() as
> a 1-pass algorithm, using the kind of "double allocated space as
> needed" gimmick unicode.join uses.  It would be less efficient if
> auto-promotion to Unicode turns out to be required, but it's hard to
> measure how little I care about that; it might be faster if auto-str
> and Unicode promotion aren't needed (as only 1 pass would be needed).

FWIW, I'm -1 on doing auto-conversion using str()/unicode() in
.join().

It's easy enough to make the stringification explicit in the
application. Making this automatic (with all the problems
that go with it), would only hide programming errors related
to mismatching types and create a whole new set of weird error
situations.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 29 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From pf_moore at yahoo.co.uk  Sun Aug 29 15:48:19 2004
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sun Aug 29 15:48:22 2004
Subject: [Python-Dev] Re: Alternative Implementation for PEP 292: Simple
	String Substitutions
References: <20040827233958.GA5560@panix.com>
	<000501c48d34$19ce9000$e841fea9@oemcomputer>
Message-ID: <uk6vicaxo.fsf@yahoo.co.uk>

"Raymond Hettinger" <python@rcn.com> writes:

> I'm not sure what to do next.  Aside from Aahz, no one on python-dev
> seems to be interested and Barry appears to have been off-line for a few
> days.  Ideally, I would like to get this in a few days ahead of the
> alpha release.

I'm not too worried either way, as I don't think that this will be
something I use a lot (I may be wrong, though..)

Having said that, I do feel that your points should be addressed. I
haven't looked at either implementation, but I'd assume that the key
benefit of a class-based implementation would be the ability to
subclass to modify behaviour. If that isn't happening in practice,
then I agree that a function is probably a better option.

As a test case, how would I implement the case that has come up here
recently, of supporting $obj.attr substitutions, in each of the two
implementations? It would be interesting to see the relative
simplicity. I'm assuming that the function-based case would require
effectively a cut & paste reimplementation.

[FX: Dives into source code...]
The Template class is pretty trivial, but it doesn't seem to be
designed for extension. Thus, the above test case would *still* need
pretty much a rewrite. However, if the Template class was rewritten
with overriding in mind, it probably could make things easier:

1. Split the match pattern into pieces, such that adding new syntax
   like $obj.attr is just a case of adding a new pattern to a list.
2. Make the __mod__ code more case-like, having separate actions
   based on group name. Then overriding code could just add actions
   for the groups it added, and call the base class for the rest.

This isn't fully thought through, and may well not be worth it in
practice, but it gives the sort of benefits I'd expect from a
class-based approach.

Paul.
-- 
Home computers are being called upon to perform many new functions,
including the consumption of homework formerly eaten by the dog --
Doug Larson

From gvanrossum at gmail.com  Sun Aug 29 19:37:30 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sun Aug 29 19:37:46 2004
Subject: [Python-Dev] Re: Alternative Implementation for PEP 292: Simple
	String Substitutions
In-Reply-To: <uk6vicaxo.fsf@yahoo.co.uk>
References: <20040827233958.GA5560@panix.com>
	<000501c48d34$19ce9000$e841fea9@oemcomputer>
	<uk6vicaxo.fsf@yahoo.co.uk>
Message-ID: <ca471dc2040829103762804113@mail.gmail.com>

> As a test case, how would I implement the case that has come up here
> recently, of supporting $obj.attr substitutions, in each of the two
> implementations? It would be interesting to see the relative
> simplicity. I'm assuming that the function-based case would require
> effectively a cut & paste reimplementation.
> 
> [FX: Dives into source code...]
> The Template class is pretty trivial, but it doesn't seem to be
> designed for extension. Thus, the above test case would *still* need
> pretty much a rewrite. However, if the Template class was rewritten
> with overriding in mind, it probably could make things easier:

Subclassing is only one possibility. With the templating version, you
could also write another class that implements the same interface --
very Pythonic. If Raymond's function approach were chosen, you're
stuck with what that function can and can't do.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From raymond.hettinger at verizon.net  Sun Aug 29 22:37:17 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun Aug 29 22:37:57 2004
Subject: [Python-Dev] os.urandom API
Message-ID: <000701c48e07$f9cea040$e841fea9@oemcomputer>

I would like to change the API for the new os.urandom(n) function to
return a long integer instead of a string.  The former better serves
more use cases and fits better with existing modules.

In favor of a long integer:

1) The call random.seed(os.random(100)) is a likely use case.  If the
intermediate value is a string, then random.seed() will hash it and only
use 32 bits.  If the intermediate value is a long integer, all bits are
used.  In the given example, the latter is clearly what the user expects
(otherwise, they would only request 4 bytes).

2) Another likely use case is accessing all the tools in the random
module with a subclass that overrides random() and getrandbits().  Both
can be done easier and faster if os.random() returns long integers.  If
the starting point is a string, the code gets ugly and slow.

3) Most use cases for random values involve numeric manipulation.
Simple tasks like finding a random integer in the range [0,100000)
become unnecessarily more complicated when starting from a string.

4) The decimal module supports instantiation directly from long integers
but not from binary strings.


In favor of a string of bytes:

1) This form is handy for cyptoweenies to xor with other byte strings
(perhaps for a one-time pad).



Raymond


From martin at v.loewis.de  Sun Aug 29 23:26:20 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Aug 29 23:26:20 2004
Subject: [Python-Dev] os.urandom API
In-Reply-To: <000701c48e07$f9cea040$e841fea9@oemcomputer>
References: <000701c48e07$f9cea040$e841fea9@oemcomputer>
Message-ID: <413249FC.10206@v.loewis.de>

Raymond Hettinger wrote:
> I would like to change the API for the new os.urandom(n) function to
> return a long integer instead of a string.  The former better serves
> more use cases and fits better with existing modules.

-1. Bytes is what the underlying system returns, and it is also
conceptually the right thing. We are really talking about a stream
of random bytes here (where u signals unlimitedness).

> 1) The call random.seed(os.random(100)) is a likely use case.  If the
> intermediate value is a string, then random.seed() will hash it and only
> use 32 bits.  If the intermediate value is a long integer, all bits are
> used.  In the given example, the latter is clearly what the user expects
> (otherwise, they would only request 4 bytes).

Then add an os.randint if you think this is important. Given the
easiness of using the struct module, I don't think it is important to
provide this out of the box.

> 2) Another likely use case is accessing all the tools in the random
> module with a subclass that overrides random() and getrandbits().  Both
> can be done easier and faster if os.random() returns long integers.  If
> the starting point is a string, the code gets ugly and slow.

Don't try guessing use cases too much. I don't think either the original
submitter, nor the original reviewer, had sequences of pseudo-random
numbers as their use case. Instead, the typical application will be a
one-time token for some crypto algorithm, in which case sequences of
pseudo-randomness are evil. What kind of data structure these things
will need is hard to guess, but "sequence of bytes" is a good bet.

> 3) Most use cases for random values involve numeric manipulation.
> Simple tasks like finding a random integer in the range [0,100000)
> become unnecessarily more complicated when starting from a string.

That is not true. Most use cases of random numbers involve bit
manipulation.

> 1) This form is handy for cyptoweenies to xor with other byte strings
> (perhaps for a one-time pad).

And indeed, cryptoweenies have contributed that code. He who writes
the code choses the interface.

Regards,
Martin
From tim.peters at gmail.com  Sun Aug 29 23:55:57 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sun Aug 29 23:56:02 2004
Subject: [Python-Dev] os.urandom API
In-Reply-To: <413249FC.10206@v.loewis.de>
References: <000701c48e07$f9cea040$e841fea9@oemcomputer>
	<413249FC.10206@v.loewis.de>
Message-ID: <1f7befae040829145555b219cb@mail.gmail.com>

I agree with Martin that the patch submitter and reviewers really want
strings to come back -- there are many use cases for that in
cryptoweenie land, as simple as generating a random session key.

Long ints might be cool too, but if so that deserves a distinct API. 
Something like "urandom(n) -- return a uniform random int from 0
through 256**n-1" would be a pretty bizarre spec.

Note that it's easy (abeit obscure) to generate a long from a string s:

    long(binascii.hexlify(s), 16)

It's harder to go in the other direction.  First you do hex(n). Then
you chop off the leading "0x".  Then there may or may not be a
trailing "L", and iff there is you have to lose that too.  Then
binascii.unhexlify() gets the string.  Then you may have to go on to
pad with one or more leading NUL bytes.

Regardless of whether there's another API added, it might be good to
change random.seed() under the covers to use (say) 4 urandom bytes for
default initialization (when os.urandom is available).  Initializing
based on time.time() is weak, and especially on Windows (where there
are typically only 18.2 distinct time.time() values per second).
From barry at python.org  Mon Aug 30 01:37:40 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 30 01:37:47 2004
Subject: [Python-Dev] __mangled in stdlib considered poor form
In-Reply-To: <ca471dc204082811505b76e6f5@mail.gmail.com>
References: <41302F87.1000805@interlink.com.au>
	<2md61b5wab.fsf@starship.python.net>
	<ca471dc204082811505b76e6f5@mail.gmail.com>
Message-ID: <1093822659.18459.1.camel@geddy.wooz.org>

On Sat, 2004-08-28 at 14:50, Guido van Rossum wrote:

> There's one potential use case where I'd like to keep the private name
> even if there's a mangled use of it nearby: when the private name
> really is intended to be private, and the mangled use is a "friend" in
> the C++ sense. Sometimes (though rarely) the stdlib has to indulge in
> practices that aren't good example code, because it's the stdlib, and
> it should implement functionality in the most bulletproof way
> possible. Using private names for internal details is the right thing
> sometimes.

Please don't just mindlessly change all leading double underscores to
singles.  +1, when following Guido's guidelines above though.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040829/59eee9a3/attachment.pgp
From greg at cosc.canterbury.ac.nz  Mon Aug 30 03:05:39 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Aug 30 03:05:47 2004
Subject: [Python-Dev] Re: adding a bytes sequence type to Python
In-Reply-To: <20040829004221.GB1593@trit.org>
Message-ID: <200408300105.i7U15dJb013777@cosc353.cosc.canterbury.ac.nz>

Dima Dorfman <dima@trit.org>:

> It supports the "read" buffer API but not the "character" buffer API,
> so the file has to be opened in binary mode for it to work:

If we're serious about bytes not being characters, isn't
this the *right* behaviour for a byte array?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+
From barry at python.org  Mon Aug 30 05:29:09 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 30 05:29:11 2004
Subject: [Python-Dev] Re: PEP 292 - Simpler String Substitutions
In-Reply-To: <20040826135401.GA31295@alcyon.progiciels-bpi.ca>
References: <1093232832.28573.16.camel@geddy.wooz.org>
	<20040823132043.GA4501@titan.progiciels-bpi.ca>
	<1093367647.18858.62.camel@geddy.wooz.org>
	<20040826135401.GA31295@alcyon.progiciels-bpi.ca>
Message-ID: <1093836549.18459.85.camel@geddy.wooz.org>

On Thu, 2004-08-26 at 09:54, Fran?ois Pinard wrote:
> [Barry Warsaw]
> 
> > > If my feeling is right, then the PEP should clearly explicit this
> > > goal [of pushing `$' forward], it will make the PEP stronger.
> 
> > I will neither confirm nor deny whether the PSU is bankrolling the PEP
> > 292 initiative, nor the actual existence of any 527 organization
> > claiming to be called the "PSU", nor whether if they did exist, they
> > were or weren't acting in coordination with the campaign organizations
> > of any 2004 US presidential nominee.
> 
> Nice joking.  Still, yet, and nevertheless, I think making the PEP
> closer to the truth might make it stronger.  It might also repair a bit
> the feeling that the PEP process is sometimes mildly lacking (the @-PEP
> having been ratehr publicised as an example of this, recently).

Um, PEP 318 has nothing to do with this, and I think the PEP 292 process
has been fairly accurate, disagreements about interface
notwithstanding.  Maybe $'s make sense for Python 3000, but that was
never my goal, explicit or subversive, for PEP 292.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040829/984c2d9d/attachment-0001.pgp
From barry at python.org  Mon Aug 30 05:37:25 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Aug 30 05:37:29 2004
Subject: [Python-Dev] Alternative Implementation for PEP 292: Simple
	String Substitutions
In-Reply-To: <002301c48bb5$17bfbe40$e841fea9@oemcomputer>
References: <002301c48bb5$17bfbe40$e841fea9@oemcomputer>
Message-ID: <1093837045.18457.94.camel@geddy.wooz.org>

On Thu, 2004-08-26 at 17:38, Raymond Hettinger wrote:

> My main issue with the current implementation is that we get no leverage
> from using a class instead of a function.  Though the API is fairly
> simple either way, it is easier to learn and document functions than
> classes.  We gain nothing from instantiation -- the underlying data
> remains immutable and no additional state is attached.  The only new
> behavior is the ability to apply the mod operator.  Why not do it in one
> step.

Weren't you the one who gave the Cheetah example?  What was interesting
about that was that the instance's attributes formed the substitution
namespace.  That's a use case I instantly liked.  So there you have
state attached to an instance.  Another case for that would be in i18n
applications where you might want to attach information such as the
gettext domain to the instance.  You might also want to build up the
namespace in several locations, and delay performing the substitution
until the last possible moment.  In all those cases you have state
attached to an instance (and would immediately invent such an instance
for those use cases if you didn't have one).

> One negative effect of the class implementation is that it inherits from
> unicode and always returns a unicode result even if all of the inputs
> (mapping values and template) are regular strings.  With a function
> implementation, that can be avoided (returning unicode only if one of
> the inputs is unicode).

To me that's not a disadvantage.  For i18n applications, unicode is the
only reasonable thing for human readable text.  str's are only useful
for byte arrays <wink>.  It's not a disadvantage for Jython or
IronPython either. :)

> Another minor advantage for a function is that it is easier to lookup in
> the reference.  If a reader sees the % operator being applied and looks
> it up in the reference, it is going to steer them in the wrong
> direction.  

The mod operator was chosen because that's what people are familiar
with, but it would probably be okay to pick a different method name.  I
think Guido's suggested using __call__() -- which I want to think more
about.

> This is doubly true if the Template instantiation is remote
> from the operator application.

Which, in some use cases, it most definitely will be.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040829/6f22ce3d/attachment.pgp
From jcarlson at uci.edu  Mon Aug 30 05:48:49 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon Aug 30 05:49:14 2004
Subject: [Python-Dev] os.urandom API
In-Reply-To: <1f7befae040829145555b219cb@mail.gmail.com>
References: <000701c48e07$f9cea040$e841fea9@oemcomputer>	<413249FC.10206@v.loewis.de>
	<1f7befae040829145555b219cb@mail.gmail.com>
Message-ID: <4132A3A1.5040305@uci.edu>


(Quick apology to Tim for accidentally sending this to him only, I 
recently switched email clients and am still getting used to it)

> Note that it's easy (abeit obscure) to generate a long from a string s:
> 
>     long(binascii.hexlify(s), 16)

or even:
     long(s.encode('hex'), 16)


> It's harder to go in the other direction.  First you do hex(n). Then

Perhaps what is needed is a method to easily convert between large 
integers and strings.  It seems as though a new struct conversion code 
is in order, something that works similarly to the way the 's' code works:

#pack the integer bigint as a signed big-endian packed binary string,
#null-filling as necessary, for 64 bytes of precision
a = struct.pack('>64g', bigint)

#unpack an unsigned little-endian packed binary string of 24 bytes to a 
#python long
b = struct.unpack('<24G', a)


With such a change, I think many of the string/integer translation 
complaints would disappear.

  - Josiah

From python at rcn.com  Mon Aug 30 07:48:46 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Aug 30 07:49:22 2004
Subject: [Python-Dev] Alternative Implementation for PEP 292: SimpleString
	Substitutions
In-Reply-To: <1093837045.18457.94.camel@geddy.wooz.org>
Message-ID: <003301c48e55$04942ac0$e841fea9@oemcomputer>

[Raymond]
> > One negative effect of the class implementation is that it inherits
from
> > unicode and always returns a unicode result even if all of the
inputs
> > (mapping values and template) are regular strings

[Barry]
> To me that's not a disadvantage.

By not inheriting from unicode, the bug can be fixed while retaining a
class implementation (see sandbox\curry292.py for an example).

But, be clear, it *is* a bug.

If all the inputs are strings, Unicode should not magically appear.  See
all the other string methods as an example.  Someday, all will be
Unicode, until then, some apps choose to remain Unicode free.  Also,
there is a build option to not even compile Unicode support -- it would
be a bummer to have the $ templates fail as a result.


Raymond 


P.S.  Here's the doctest from the sandbox code.  What is at issue is the
result of the first test:

>>> Template('the $xxx and')(xxx='10')
'the 10 and'
>>> Template(u'the $xxx and')(xxx='10')
u'the 10 and'
>>> Template('the $xxx and')(xxx=u'10')
u'the 10 and'
>>> Template(u'the $xxx and')(xxx=u'10')
u'the 10 and'




From adurdin at gmail.com  Mon Aug 30 08:11:34 2004
From: adurdin at gmail.com (Andrew Durdin)
Date: Mon Aug 30 08:11:42 2004
Subject: [Python-Dev] Alternative placeholder delimiters for PEP 292
Message-ID: <59e9fd3a040829231141cd3fe4@mail.gmail.com>

A Yet Simpler Proposal, modifying that of PEP 292

    I propose that the Template module not use $ to set off
    placeholders; instead, placeholders are delimited by braces {}.
    The following rules for {}-placeholders apply:

    1. {{ and }} are escapes; they are replaced with a single { or }
       respectively.

    2. {identifier} names a substitution placeholder matching a
       mapping key of "identifier".  By default, "identifier" must
       spell a Python identifier as defined in Identifiers and
       Keywords[1].

    No other characters have special meaning.

    If the left-brace { is unmatched, appears at the end of the
    string, or is followed by any non-identifier character, a
    ValueError will be raised at interpolation time[2].

    If a single, unmatched right-brace } occurs in the string, a
    ValueError will be raised at interpolation time. This avoids
    ambiguity: did the user want a single right-brace, or did they
    inadvertently omit the left-brace? This will also cause a
    probably erroneous "{foo}}" or "{{foo}" to raise a ValueError.

Rationale

    There are several reasons for preferring the paired delimiters {}
    to a single prefixed $:

    1. The placeholder name stands out more clearly from its
       surroundings, due to the presence of a closing delimiter, and
       also to the fact that the braces bear less resemblance to any
       alphabetic characters than the dollar sign:

       "Hello, {name}, how are you?" vs "Hello, $name, how are you?"

    2. Only two characters have special meanings in the string, as
       opposed to three. Additionally, dollar signs are expected to
       be more often used in templated strings (e.g. for currency
       values) than braces:

       "The {item} costs ${price}." vs "The $item costs $$$price."

    3. The placeholder syntax is consistent, and does not change even
       when valid identifier characters follow the placeholder but
       are not part of the placeholder:

       "Look at the {plural}s!" vs "Look at the ${plural}s!"

    4. The template substitution could be changed in future to
       support dotted names without breaking existing code. The
       example below would break if the $-template were changed to
       allow dotted names:

       "Evaluate {obj}.__doc__" vs "Evaluate $obj.__doc__"


    There are two drawbacks to the proposal when compared with
    $-templates:

    1. An extra character must be typed for each placeholder in the
       common case:

       "{name}, welcome to {site}!" vs "$name, welcome to $site!"

    2. Templated strings containing braces become more complicated:

       "dict = {{'{key}': '{value}'}}" vs "dict = {'$key': '$value'}"

    The first is not a real issue; the extra closing braces needed
    for the placeholder when compared with the number of other
    characters in the templated string will usually be insignificant.
    Furthermore, the {}-placeholders require fewer characters to be
    typed in the less common case when valid identifier characters
    follow the placeholder but are not part of it.

    The need for braces in a templated string is not expected to
    occur frequently. Because of this, the second drawback is
    considered of minor importance.

Reference Implementation

    If the general nature of feedback on this proposal is positive,
    or expressive of interest in an implementation, then a reference
    implementation will be created forthwith.

References and Notes

    [1] Identifiers and Keywords
    http://www.python.org/doc/current/ref/identifiers.html

    [2] The requirement for interpolation-time error raising is the
        same as in PEP 292. Although not a part of this proposal, I
        suggest that it would be better if the error occured when the
        Template instance is created.

        It is worth noting that the PEP 292 reference implementation
        (in python/nondist/sandbox/string/string/template.py) does
        not fully conform to the PEP as regards raising errors for
        invalid template strings:

         >>> t = Template("This should cause an error: $*")
         >>> t % {}
         u'This should cause an error: $*'
         >>> t = Template("This also should cause an error: $")
         >>> t % {}
         u'This also should cause an error: $'
From jlgijsbers at planet.nl  Mon Aug 30 12:23:53 2004
From: jlgijsbers at planet.nl (Johannes Gijsbers)
Date: Mon Aug 30 12:22:23 2004
Subject: [Python-Dev] Adding 'lexists()' to os.path
In-Reply-To: <200408221858.i7MIwV502491@guido.python.org>
References: <20040822110835.GA7452@mail.planet.nl>
	<200408221858.i7MIwV502491@guido.python.org>
Message-ID: <20040830102353.GA14784@mail.planet.nl>

> Looks like you already got the go-ahead.  Just make sure with any API
> changes/additions to os.path, that there are many individual *path
> modules that need to be updated -- if a platform doesn't support
> symlinks, lexists() should probably just be an alias for exists().
> 
> posixpath.py, ntpath.py, macpath.py, os2emxpath.py.

Yes, cben's patch takes cares of that. I've checked it in just now.

Johannes
From anthony at interlink.com.au  Mon Aug 30 15:22:10 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Aug 30 15:40:47 2004
Subject: [Python-Dev] (my) revisions to PEP318 finally done.
Message-ID: <41332A02.1040902@interlink.com.au>


Here's the version of PEP 318 I just checked in. If you have any
additions or changes, I prefer patches mailed to me in unified
diff format. Comments like "You should cover point X, Y and Z",
without the accompanying words, I won't be acting on (others with
checkin privs are more than welcome to do so). Sorry about that,
but I _cannot_ spend any more time on this document now - this
whole decorators thing has consumed far more of my time than I
can afford right now.

-------------- next part --------------
PEP: 318
Title: Decorators for Functions and Methods
Version: $Revision: 1.28 $
Last-Modified: $Date: 2004/08/30 13:16:56 $
Author: Kevin D. Smith, Jim Jewett, Skip Montanaro, Anthony Baxter
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 05-Jun-2003
Python-Version: 2.4
Post-History: 09-Jun-2003, 10-Jun-2003, 27-Feb-2004, 23-Mar-2004, 30-Aug-2004


WarningWarningWarning
=====================

The final decision on the syntax for 2.4a3 is not yet made. This 
will be done before 2.4a3, and this document will be updated to 
match. Note also that this document does not attempt to cover the
huge number of potential alternative syntaxes, nor is it an attempt
to exhaustively list all the positives and negatives of each form.

Abstract
========

The current method for transforming functions and methods (for
instance, declaring them as a class or static method) is awkward and
can lead to code that is difficult to understand.  Ideally, these
transformations should be made at the same point in the code where the
declaration itself is made.  This PEP introduces new syntax for
transformations of a function or method declaration.


Motivation
==========

The current method of applying a transformation to a function or
method places the actual translation after the function body.  For
large functions this separates a key component of the function's
behavior from the definition of the rest of the function's external
interface.  For example::

    def foo(self):
        perform method operation
    foo = classmethod(foo)

This becomes less readable with longer methods.  It also seems less
than pythonic to name the function three times for what is
conceptually a single declaration.  A solution to this problem is to
move the transformation of the method closer to the method's own
declaration.  While the new syntax is not yet final, the intent is to
replace::

    def foo(cls):
        pass
    foo = synchronized(lock)(foo)
    foo = classmethod(foo)

with an alternative that places the decoration in the function's
declaration::

    @classmethod
    @synchronized(lock)
    def foo(cls):
        pass

Modifying classes in this fashion is also possible, though the
benefits are not as immediately apparent.  Almost certainly, anything
which could be done with class decorators could be done using
metaclasses, but using metaclasses is sufficiently obscure that there
is some attraction to having an easier way to make simple
modifications to classes.  For Python 2.4, only function/method
decorators are being added.


Why Is This So Hard?
--------------------

Two decorators (``classmethod()`` and ``staticmethod()``) have
been available in Python since version 2.2.  It's been assumed since
approximately that time that some syntactic support for them would
eventually be added to the language.  Given this assumption, one might
wonder why it's been so difficult to arrive at a consensus.
Discussions have raged off-and-on at times in both comp.lang.python
and the python-dev mailing list about how best to implement function
decorators.  There is no one clear reason why this should be so, but a
few problems seem to be most problematic.

* Disagreement about where the "declaration of intent" belongs.
  Almost everyone agrees that decorating/transforming a function at
  the end of its definition is suboptimal.  Beyond that there seems to
  be no clear consensus where to place this information.

* Syntactic constraints.  Python is a syntactically simple language
  with fairly strong constraints on what can and can't be done without
  "messing things up" (both visually and with regards to the language
  parser).  There's no obvious way to structure this information so
  that people new to the concept will think, "Oh yeah, I know what
  you're doing."  The best that seems possible is to keep new users
  from creating a wildly incorrect mental model of what the syntax
  means.

* Overall unfamiliarity with the concept.  For people who have a
  passing acquaintance with algebra (or even basic arithmetic) or have
  used at least one other programming language, much of Python is
  intuitive.  Very few people will have had any experience with the
  decorator concept before encountering it in Python.  There's just no
  strong preexisting meme that captures the concept.

* Syntax discussions in general appear to cause more contention than
  almost anything else. Readers are pointed to the ternary operator 
  discussions that were associated with PEP 308 for another example of
  this.

Background
==========

There is general agreement that syntactic support is desirable to the
current state of affairs.  Guido mentioned `syntactic support for
decorators`_ in his DevDay keynote presentation at the `10th Python
Conference`_, though `he later said`_ it was only one of several
extensions he proposed there "semi-jokingly".  `Michael Hudson raised
the topic`_ on ``python-dev`` shortly after the conference,
attributing the initial bracketed syntax to an earlier proposal on
``comp.lang.python`` by `Gareth McCaughan`_.

.. _syntactic support for decorators:
   http://www.python.org/doc/essays/ppt/python10/py10keynote.pdf
.. _10th python conference:
   http://www.python.org/workshops/2002-02/
.. _michael hudson raised the topic:
   http://mail.python.org/pipermail/python-dev/2002-February/020005.html
.. _he later said:
   http://mail.python.org/pipermail/python-dev/2002-February/020017.html
.. _gareth mccaughan:
   http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=slrna40k88.2h9o.Gareth.McCaughan%40g.local

Class decorations seem like an obvious next step because class
definition and function definition are syntactically similar.

The discussion continued on and off on python-dev from February 2002
through July 2004.  Hundreds and hundreds of posts were made, with
people proposing many possible syntax variations.  Guido took a list
of proposals to `EuroPython 2004`_, where a discussion took place.
Subsequent to this, he decided that for 2.4a2 we'd have the
`Java-style`_ @decorator syntax.  Barry Warsaw named this the
'pie-decorator' syntax, in honor of the Pie-thon Parrot shootout which
was announced about the same time as the decorator syntax, and because
the @ looks a little like a pie.  Guido `outlined his case`_ on
Python-dev, including `this piece`_ on the various rejected forms.

.. _EuroPython 2004:
    http://www.python.org/doc/essays/ppt/euro2004/euro2004.pdf
.. _outlined his case:
    http://mail.python.org/pipermail/python-dev/2004-August/author.html
.. _this piece:
    http://mail.python.org/pipermail/python-dev/2004-August/046672.html
..  _Java-style:
    http://java.sun.com/j2se/1.5.0/docs/guide/language/annotations.html


On the name 'Decorator'
=======================

There's been a number of complaints about the choice of the name
'decorator' for this feature.  The major one is that the name is not
consistent with its use in the `GoF book`_.  The name 'decorator'
probably owes more to its use in the compiler area -- a syntax tree is
walked and annotated.  It's quite possible that a better name may turn
up.

.. _GoF book:
    http://patterndigest.com/patterns/Decorator.html


Design Goals
============

The new syntax should

* work for arbitrary wrappers, including user-defined callables and
  the existing builtins ``classmethod()`` and ``staticmethod()``.  This
  requirement also means that a decorator syntax must support passing
  arguments to the wrapper constructor

* work with multiple wrappers per definition

* make it obvious what is happening; at the very least it should be
  obvious that new users can safely ignore it when writing their own
  code

* be a syntax "that ... [is] easy to remember once explained"

* not make future extensions more difficult

* be easy to type; programs that use it are expected to use it very
  frequently

* not make it more difficult to scan through code quickly.  It should
  still be easy to search for all definitions, a particular
  definition, or the arguments that a function accepts

* not needlessly complicate secondary support tools such as
  language-sensitive editors and other "`toy parser tools out
  there`_"

* allow future compilers to optimize for decorators.  With the hope of
  a JIT compiler for Python coming into existence at some point this
  tends to require the syntax for decorators to come before the
  function definition

* move from the end of the function, where it's currently hidden, to
  the front where it is more `in your face`_

Andrew Kuchling has links to a bunch of the discussions about
motivations and use cases `in his blog`_.  Particularly notable is `Jim
Huginin's list of use cases`_.

.. _toy parser tools out there:
   http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=mailman.1010809396.32158.python-list%40python.org
.. _in your face:
    http://mail.python.org/pipermail/python-dev/2004-August/047112.html
.. _in his blog:
    http://www.amk.ca/diary/archives/cat_python.html#003255
.. _Jim Huginin's list of use cases:
    http://mail.python.org/pipermail/python-dev/2004-April/044132.html


Current Syntax
==============

The current syntax for function decorators as implemented in Python
2.4a2 is::

    @dec2
    @dec1
    def func(arg1, arg2, ...):
        pass

This is equivalent to::

    def func(arg1, arg2, ...):
        pass
    func = dec2(dec1(func))

without the intermediate assignment to the variable ``func``.  The
decorators are near the function declaration.  The @ sign makes it
clear that something new is going on here.

The decorator statement is limited in what it can accept -- arbitrary
expressions will not work.  Guido preferred this because of a `gut
feeling`_.

.. _gut feeling:
    http://mail.python.org/pipermail/python-dev/2004-August/046711.html


Syntax Alternatives
===================

There have been `a large number`_ of different syntaxes proposed --
rather than attempting to work through these individual syntaxes, it's
worthwhile to break the syntax discussion down into a number of areas.
Attempting to discuss `each possible syntax`_ individually would be an
act of madness, and produce a completely unwieldy PEP.

.. _a large number:
    http://www.python.org/moin/PythonDecorators
.. _each possible syntax:
    http://ucsu.colorado.edu/~bethard/py/decorators-output.py


Decorator Location
------------------

The first syntax point is the location of the decorators.  For the
following examples, we use the @syntax used in 2.4a2.

Decorators before the def statement are the first alternative,
and the syntax used in 2.4a2::

    @classmethod
    def foo(arg1,arg2):
        pass

    @accepts(int,int)
    @returns(float)
    def bar(low,high):
        pass

There have been a number of objections raised to this location --
the primary one is that it's the first real Python case where a
line of code has a result on a following line.  The syntax that
will be in 2.4a3 will also require one decorator per line (in a2,
multiple decorators can be specified on the same line).

People also complained that the syntax got unworldly quickly when
multiple decorators were used.  The point was made, though, that the
chances of a large number of decorators being used on a single function
were small and thus this was not a large worry.

Some of the advantages of this form are that the decorators live
outside the method body -- they are obviously executed at the time
the function is defined.

Another advantage is that being prefix to the function definition fit the
idea of knowing about a change to the semantics of the code before the
code itself, thus knowing how to interpret the code's semantics
properly without having to go back and change your initial perceptions
if the syntax did not come before the function definition.

Guido decided `he preferred`_ having the decorators on the line before
the 'def', because it was felt that a long argument list would mean
that the decorators would be 'hidden'

.. _he preferred:
    http://mail.python.org/pipermail/python-dev/2004-March/043756.html

The second form is the decorators between the def and the function
name, or the function name and the argument list::

    def @classmethod foo(arg1,arg2):
        pass

    def @accepts(int,int),@returns(float) bar(low,high):
        pass

    def foo @classmethod (arg1,arg2):
        pass

    def bar @accepts(int,int),@returns(float) (low,high):
        pass

There are a couple of objections to this form.  The first is that it
breaks easily 'greppability' of the source -- you can no longer search
for 'def foo(' and find the definition of the function.  The second,
more serious, objection is that in the case of multiple decorators,
the syntax would be extremely unwieldy.

The next form, which has had a number of strong proponents, is to
have the decorators between the argument list and the trailing ``:``
in the 'def' line::

    def foo(arg1,arg2) @classmethod:
        pass

    def bar(low,high) @accepts(int,int),@returns(float):
        pass

Guido `summarized the arguments`_ against this form (many of which
also apply to the previous form) as:

- it hides crucial information (e.g. that it is a static method)
  after the signature, where it is easily missed

- it's easy to miss the transition between a long argument list and a
  long decorator list

- it's cumbersome to cut and paste a decorator list for reuse, because
  it starts and ends in the middle of a line

.. _summarized the arguments:
    http://mail.python.org/pipermail/python-dev/2004-August/047112.html

The next form is that the decorator syntax go inside the method
body at the start, in the same place that docstrings currently
live:

    def foo(arg1,arg2):
        @classmethod
        pass

    def bar(low,high):
        @accepts(int,int)
        @returns(float)
        pass

The primary objection to this form is that it requires "peeking
inside" the method body to determine the decorators.  In addition,
even though the code is inside the method body, it is not executed
when the method is run.  Guido felt that docstrings were not a good
counter-example, and that it was quite possible that a 'docstring'
decorator could help move the docstring to outside the function body.

The final form is a new block that encloses the method's code.  For
this example, we'll use a 'decorate' keyword, as it makes no sense
with the @syntax. ::

    decorate:
        classmethod
        def foo(arg1,arg2):
            pass

    decorate:
        accepts(int,int)
        returns(float)
        def bar(low,high):
            pass

This form would result in inconsistent indentation for decorated and
undecorated methods.  In addition, a decorated method's body would
start three indent levels in.


Syntax forms
------------

* ``@decorator``::

    @classmethod
    def foo(arg1,arg2):
        pass

    @accepts(int,int)
    @returns(float)
    def bar(low,high):
        pass

  The major objections against this syntax are that the @ symbol is
  not currently used in Python (and is used in both IPython and Leo),
  and that the @ symbol is not meaningful. Another objection is that 
  this "wastes" a currently unused character (from a limited set) on
  something that is not perceived as a major use.

* ``|decorator``::

    |classmethod
    def foo(arg1,arg2):
        pass

    |accepts(int,int)
    |returns(float)
    def bar(low,high):
        pass

  This is a variant on the @decorator syntax -- it has the advantage
  that it does not break IPython and Leo.  Its major disadvantage
  compared to the @syntax is that the | symbol looks like both a
  capital I and a lowercase l.

* list syntax::

    [classmethod]
    def foo(arg1,arg2):
        pass

    [accepts(int,int), returns(float)] 
    def bar(low,high): 
        pass 

  The major objection to the list syntax is that it's currently
  meaningful (when used in the form before the method).  It's also
  lacking any indication that the expression is a decorator.

* list syntax using other brackets (``<...>``, ``[[...]]``, ...)::

    <classmethod>
    def foo(arg1,arg2):
        pass

    <accepts(int,int), returns(float)>
    def bar(low,high): 
        pass 

  None of these alternatives gained much traction. The alternatives
  which involve square brackets only serve to make it obvious that the
  decorator construct is not a list. They do nothing to make parsing any
  easier. The '<...>' alternative presents parsing problems because '<'
  and '>' already parse as un-paired. They present a further parsing
  ambiguity because a right angle bracket might be a greater than symbol
  instead of a closer for the decorators.

* ``decorate()``

  The ``decorate()`` proposal was that no new syntax be implemented --
  instead a magic function that used introspection to manipulate the
  following function.  Both Jp Calderone and Philip Eby produced
  implementations of functions that did this.  Guido was pretty firmly
  against this -- with no new syntax, the magicness of a function like
  this is extremely high:

    Using functions with "action-at-a-distance" through
    sys.settraceback may be okay for an obscure feature that can't be
    had any other way yet doesn't merit changes to the language, but
    that's not the situation for decorators.  The widely held view
    here is that decorators need to be added as a syntactic feature to
    avoid the problems with the postfix notation used in 2.2 and 2.3.
    Decorators are slated to be an important new language feature and
    their design needs to be forward-looking, not constrained by what
    can be implemented in 2.3.

* new keyword (and block)

  This idea was the consensus alternate from comp.lang.python. Robert
  Brewer wrote up a detailed `J2 proposal`_ document outlining the arguments
  in favor of this. The issues with this form are that it requires a new
  keyword (and therefore a ``from __future__ import decorators`` statement),
  it seems like there is no obvious choice for the keyword (``using`` is 
  the choice of the proposal and implementation), and that the form 
  produces something that looks like a normal code block, but isn't. Attempts
  to use statements in this block will cause a syntax error. It's also 
  going to potentially confuse users who don't expect a block to contain
  bare expressions.


.. _J2 proposal:
    http://www.aminus.org/rbre/python/pydec.html


There are plenty of other variants and proposals on `the wiki page`_.

.. _the wiki page:
    http://www.python.org/moin/PythonDecorators

Why @?
------

There is some history in Java using @ initially as a marker in
`Javadoc comments`_ and later in Java 1.5 for `annotations`_, which
are similar to Python decorators.  The fact that @ was previously
unused as a token in Python also means it's clear there is no
possibility of such code being parsed by an earlier version of Python,
leading to possibly subtle semantic bugs.  It also means that ambiguity
of what is a decorator and what isn't is removed. of That said, @ is
still a fairly arbitrary choice.  Some have suggested using | instead.

For syntax options which use a list-like syntax (no matter where it
appears) to specify the decorators a few alternatives were proposed:
``[|...|]``, ``*[...]*``, and ``<...>``.  


.. _Javadoc comments:
    http://java.sun.com/j2se/javadoc/writingdoccomments/
.. _annotations:
    http://java.sun.com/j2se/1.5.0/docs/guide/language/annotations.html


Current Implementation, History
===============================

Guido asked for a volunteer to implement his preferred syntax, and Mark
Russell stepped up and posted a `patch`_ to SF.  The syntax accepted
for 2.4a2 is::

    @dec2
    @dec1
    def func(arg1, arg2, ...):
        pass

This is equivalent to::

    def func(arg1, arg2, ...):
        pass
    func = dec2(dec1(func))

though without the intermediate creation of a variable named ``func``.

A `previous patch`_ from Michael Hudson which implements the
list-after-def syntax is also still kicking around.

.. _patch: http://www.python.org/sf/979728
.. _previous patch: http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar-3.diff

After 2.4a2 was released, in response to community reaction, Guido 
stated that he'd re-examine a community proposal, if the community 
could come up with a community consensus, a decent proposal, and an
implementation.

After an amazing number of posts, collecting a vast number of alternatives
in the `Python wiki`_, the proposed J2 syntax (the new keyword ``using`` 
in a block before the def). Robert Brewer wrote a `detailed proposal`_ 
for this form, and Michael Sparks produced `a patch`_. As at time of 
writing, we're waiting for Guido's decision. 

.. _Python wiki:
    http://www.python.org/moin/PythonDecorators
.. _detailed proposal:
    http://www.aminus.org/rbre/python/pydec.html
.. _a patch: 
    http://www.python.org/sf/1013835


Examples
========

Much of the discussion on ``comp.lang.python`` and the ``python-dev``
mailing list focuses on the use of decorators as a cleaner way to use
the ``staticmethod()`` and ``classmethod()`` builtins.  This
capability is much more powerful than that.  This section presents
some examples of use.

1. Define a function to be executed at exit.  Note that the function
   isn't actually "wrapped" in the usual sense. ::

       def onexit(f):
           import atexit
           atexit.register(f)
           return f

       @onexit
       def func():
           ...

   Note that this example is probably not suitable for real usage, but
   is for example purposes only.

2. Define a class with a singleton instance.  Note that once the class
   disappears enterprising programmers would have to be more creative
   to create more instances.  (From Shane Hathaway on ``python-dev``.)
   ::

       def singleton(cls):
           instances = {}
           def getinstance():
               if cls not in instances:
                   instances[cls] = cls()
               return instances[cls]
           return getinstance

       @singleton
       class MyClass:
           ...

3. Add attributes to a function.  (Based on an example posted by
   Anders Munch on ``python-dev``.) ::

       def attrs(**kwds):
           def decorate(f):
               for k in kwds:
                   setattr(f, k, kwds[k])
               return f
           return decorate

       @attrs(versionadded="2.2",
              author="Guido van Rossum")
       def mymethod(f):
           ...

4. Enforce function argument and return types.  Note that this 
   copies the func_name attribute from the old to the new function.
   func_name was made writable in Python 2.4a3::

       def accepts(*types):
           def check_accepts(f):
               assert len(types) == f.func_code.co_argcount
               def new_f(*args, **kwds):
                   for (a, t) in zip(args, types):
                       assert isinstance(a, t), \
                              "arg %r does not match %s" % (a,t)
                   return f(*args, **kwds)
               new_f.func_name = f.func_name
               return new_f
           return check_accepts

       def returns(rtype):
           def check_returns(f):
               def new_f(*args, **kwds):
                   result = f(*args, **kwds)
                   assert isinstance(result, rtype), \
                          "return value %r does not match %s" % (result,rtype)
                   return result
               new_f.func_name = f.func_name
               return new_f
           return check_returns

       @accepts(int, (int,float))
       @returns((int,float))
       def func(arg1, arg2):
           return arg1 * arg2

5. Declare that a class implements a particular (set of) interface(s).
   This is from a posting by Bob Ippolito on ``python-dev`` based on
   experience with `PyProtocols`_. ::

       def provides(*interfaces):
            """
            An actual, working, implementation of provides for
            the current implementation of PyProtocols.  Not
            particularly important for the PEP text.
            """
            def provides(typ):
                declareImplementation(typ, instancesProvide=interfaces)
                return typ
            return provides

       class IBar(Interface):
            """Declare something about IBar here"""

       @provides(IBar)
       class Foo(object):
               """Implement something here..."""

   .. _PyProtocols: http://peak.telecommunity.com/PyProtocols.html

Of course, all these examples are possible today, though without
syntactic support.


Open Issues
===========

1. It's not yet certain that class decorators will be incorporated
   into the language at this point.  Guido expressed skepticism about
   the concept, but various people have made some `strong arguments`_
   (search for ``PEP 318 -- posting draft``) on their behalf in
   ``python-dev``.

   .. _strong arguments:
      http://mail.python.org/pipermail/python-dev/2004-March/thread.html


Copyright
=========

This document has been placed in the public domain.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:
From anthony at interlink.com.au  Mon Aug 30 16:22:35 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Aug 30 16:23:02 2004
Subject: [Python-Dev] (my) revisions to PEP318 finally done.
In-Reply-To: <41332A02.1040902@interlink.com.au>
References: <41332A02.1040902@interlink.com.au>
Message-ID: <4133382B.7070308@interlink.com.au>

Anthony Baxter wrote:
> 
> Here's the version of PEP 318 I just checked in. 

Forgot to note - as the block at the start says, there
will be changes made once Guido makes a decision on the
J2-vs-pie thing.

Anthony


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From goodger at python.org  Mon Aug 30 17:04:38 2004
From: goodger at python.org (David Goodger)
Date: Mon Aug 30 17:04:56 2004
Subject: [Python-Dev] PEP 318 & Community Consensus
In-Reply-To: <41332A02.1040902@interlink.com.au>
References: <41332A02.1040902@interlink.com.au>
Message-ID: <cgvfmc$tvf$1@sea.gmane.org>

I just made some modifications: Made J2 proposal more prominent, and 
removed some perceived bias.  Pushed to web: 
<http://www.python.org/peps/pep-0318.html>.  See the "Community 
Consensus" section.

-- 
David Goodger <http://python.net/~goodger>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040830/90345397/signature.pgp
From anthony at interlink.com.au  Mon Aug 30 17:13:21 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Aug 30 17:13:45 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.28, 1.29
In-Reply-To: <E1C1nei-0004Kr-CO@sc8-pr-cvs1.sourceforge.net>
References: <E1C1nei-0004Kr-CO@sc8-pr-cvs1.sourceforge.net>
Message-ID: <41334411.1040906@interlink.com.au>

goodger@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/nondist/peps
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16653
> 
> Modified Files:
> 	pep-0318.txt 
> Log Message:
> Made J2 proposal more prominent, and removed some perceived bias.  Minor cleanup.

It's quite possible there was some bias in there. I considered going
through it to make sure it was scrupulously even and balanced, but
I was a bit too burned out on the topic to be sure of it. Figured
that any grotesque slanders would get feedback <wink>

Thanks for the additions!

Anthony
From fredrik at pythonware.com  Mon Aug 30 17:35:55 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Aug 30 17:34:09 2004
Subject: [Python-Dev] Re: PEP 318 & Community Consensus
References: <41332A02.1040902@interlink.com.au> <cgvfmc$tvf$1@sea.gmane.org>
Message-ID: <cgvhdc$4io$1@sea.gmane.org>

David Goodger wrote:

> It may confuse users who don't expect a block to contain bare
> expressions.

blocks can contain bare expressions today, and such expressions are
evaluated, but the results are discarded.  plain function calls belong to
this category, for example.

> On the other hand, the ``@`` syntax is bound to cause at least as much
> confusion.

and that qualifies as an unbiased observation in exactly in what way?

</F> 



From goodger at python.org  Mon Aug 30 18:05:42 2004
From: goodger at python.org (David Goodger)
Date: Mon Aug 30 18:05:46 2004
Subject: [Python-Dev] Re: PEP 318 & Community Consensus
In-Reply-To: <cgvhdc$4io$1@sea.gmane.org>
References: <41332A02.1040902@interlink.com.au> <cgvfmc$tvf$1@sea.gmane.org>
	<cgvhdc$4io$1@sea.gmane.org>
Message-ID: <41335056.8090800@python.org>

[Fredrik Lundh]
 > David Goodger wrote:
 >> It may confuse users who don't expect a block to contain bare
 >> expressions.
 >
 > blocks can contain bare expressions today, and such
 > expressions are evaluated, but the results are discarded.
 > plain function calls belong to this category, for example.

Corrected in CVS & on the web.

 >> On the other hand, the ``@`` syntax is bound to cause at least as
 >> much confusion.
 >
 > and that qualifies as an unbiased observation in exactly in what
 > way?

Perhaps in trying to balance the text I overshot.  Removed.

-- 
David Goodger <http://python.net/~goodger>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20040830/97d3134a/signature.pgp
From ndbecker2 at verizon.net  Mon Aug 30 19:08:33 2004
From: ndbecker2 at verizon.net (Neal D. Becker)
Date: Mon Aug 30 19:08:38 2004
Subject: [Python-Dev] python vs. valgrind
Message-ID: <cgvmuh$l2n$1@sea.gmane.org>

I just tried running valgrind-2.1.2 on python-2.3.3.

[nbecker@rpppc1 ~]$ valgrind --tool=memcheck /usr/bin/python ./Hello.py
==12284== Memcheck, a memory error detector for x86-linux.
==12284== Copyright (C) 2002-2004, and GNU GPL'd, by Julian Seward et al.
==12284== Using valgrind-2.1.2, a program supervision framework for
x86-linux.
==12284== Copyright (C) 2000-2004, and GNU GPL'd, by Julian Seward et al.
==12284== For more details, rerun with: -v
==12284== 
==12284== Conditional jump or move depends on uninitialised value(s)
==12284==    at 0x807AFA4: PyObject_Free (in /usr/bin/python)
==12284==    by 0x8074A4F: (within /usr/bin/python)
==12284==    by 0x8082AF0: PyString_InternInPlace (in /usr/bin/python)
==12284==    by 0x8082BA2: PyString_InternFromString (in /usr/bin/python)
==12284== 
==12284== Use of uninitialised value of size 4
==12284==    at 0x807AFB2: PyObject_Free (in /usr/bin/python)
==12284==    by 0x8074A4F: (within /usr/bin/python)
==12284==    by 0x8082AF0: PyString_InternInPlace (in /usr/bin/python)
==12284==    by 0x8082BA2: PyString_InternFromString (in /usr/bin/python)
[many, many more complaints...]

Does this indicate a problem?

From fredrik at pythonware.com  Mon Aug 30 19:25:58 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Aug 30 19:24:12 2004
Subject: [Python-Dev] Re: python vs. valgrind
References: <cgvmuh$l2n$1@sea.gmane.org>
Message-ID: <cgvnro$o7i$1@sea.gmane.org>

Neal D. Becker wrote:

>I just tried running valgrind-2.1.2 on python-2.3.3.

> [many, many more complaints...]

compare and contrast:

$ valgrind python2.1
Python 2.1.3 ...
>>>
==12393== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 17 from 2)
==12393== malloc/free: in use at exit: 78911 bytes in 220 blocks.
==12393== malloc/free: 6367 allocs, 6147 frees, 764773 bytes allocated.

$ valgrind python2.3
Python 2.3.3
==12397== ERROR SUMMARY: 893 errors from 71 contexts (suppressed: 19 from 2)
==12397== malloc/free: in use at exit: 704466 bytes in 227 blocks.
==12397== malloc/free: 2033 allocs, 1806 frees, 1540716 bytes allocated.

(using a slightly older version of valgrind 2.x)

most (all?) errors occur in PyObject_Free and PyObject_Realloc, which indicates
that the problem is either in pymalloc or valgrind itself (and in my experience, while
vladimir and tim almost never mess up, valgrind is *never* wrong...)

    http://www.python.org/doc/2.3.3/whatsnew/section-pymalloc.html

</F> 



From tim.peters at gmail.com  Mon Aug 30 19:32:51 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug 30 19:32:54 2004
Subject: [Python-Dev] python vs. valgrind
In-Reply-To: <cgvmuh$l2n$1@sea.gmane.org>
References: <cgvmuh$l2n$1@sea.gmane.org>
Message-ID: <1f7befae0408301032773b7ba9@mail.gmail.com>

[Neal D. Becker]
> I just tried running valgrind-2.1.2 on python-2.3.3.

See Misc/README.valgrind in 2.4a2, or from current CVS.
From fredrik at pythonware.com  Mon Aug 30 19:36:27 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Aug 30 19:34:42 2004
Subject: [Python-Dev] Re: python vs. valgrind
References: <cgvmuh$l2n$1@sea.gmane.org>
Message-ID: <cgvofd$q6g$1@sea.gmane.org>

> most (all?) errors occur in PyObject_Free and PyObject_Realloc, which indicates
> that the problem is either in pymalloc or valgrind itself (and in my experience, while
> vladimir and tim almost never mess up, valgrind is *never* wrong...)

on the other hand:

http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Objects/obmalloc.c?r1=2.51&r2=2.52

(but I must be tired; can anyone explain how "making the function last"
prevents the code from looking at uninitialized data?)

</F> 



From tim.peters at gmail.com  Mon Aug 30 19:45:33 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug 30 19:45:36 2004
Subject: [Python-Dev] Re: python vs. valgrind
In-Reply-To: <cgvofd$q6g$1@sea.gmane.org>
References: <cgvmuh$l2n$1@sea.gmane.org> <cgvofd$q6g$1@sea.gmane.org>
Message-ID: <1f7befae04083010459fb6b7e@mail.gmail.com>

[Fredrik Lundh]
> (but I must be tired; can anyone explain how "making the function last"
> prevents the code from looking at uninitialized data?)

It doesn't.  obmalloc can read uninitialized memory, and by design
(see REAMDE.valgrind).  Putting the function last was, as Neal said in
his comment there. to prevent gcc from inlining it.  That in turn is
required so that the Valgrind suppression file can name the function.
From amk at amk.ca  Mon Aug 30 21:23:55 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Aug 30 21:24:27 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <20040809211517.GB11199@rogue.amk.ca>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>
	<20040809211517.GB11199@rogue.amk.ca>
Message-ID: <20040830192355.GA11836@rogue.amk.ca>

The following list of suggestions has been sitting around since the
beginning of August?  If not, I'll proceed with the actions described
below.

--amk

On Mon, Aug 09, 2004 at 05:15:17PM -0400, A.M. Kuchling wrote:
> Anyway, here's a revised action list:
> 
> * Remove TERMIOS, mpz, statcache, xreadlines, rotor.
>   (Unchanged from my earlier proposal.)
> 
>   These modules have raised DeprecationWarning for a while, but only
>   rotor is listed in PEP 4.  I'll add them to the PEP in any event; 
>   is not being listed sufficient grounds for delaying their removal to
>   2.5?
> 
>   (I would say we don't need to wait until 2.5; they've been raising
>   DeprecationWarnings for a long time, so users should be well aware 
>   the modules' days are numbered.)
> 
> * Leave rfc822, mimetools alone; the stdlib will have to avoid their use 
>   first, and I'm not going to do that right now.
> 
> * Make mimify, MimeWriter raise DeprecationWarnings; nothing in the
>   stdlib uses them.
From amk at amk.ca  Mon Aug 30 21:30:30 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Aug 30 21:31:01 2004
Subject: [Python-Dev] Volunteer for next bug day?
Message-ID: <20040830193030.GB11836@rogue.amk.ca>

Currently 2.4a3 is scheduled for Thursday Sept. 2nd.  It's been an
age-old tradition for three months now to have a bug day the weekend
after a release.  I won't be organizing this one because I can't
guarantee being around all day on Saturday.  Anyone want to volunteer
to run it?  (Tasks: update the Wiki page; send out announcements to
python-dev, c.l.py.a.; be on IRC during the bug day to answer
questions and help people.)

--amk
From tim.peters at gmail.com  Mon Aug 30 21:45:17 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug 30 21:45:55 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib random.py,
	1.62, 1.63
In-Reply-To: <E1C1fRX-0006zB-EJ@sc8-pr-cvs1.sourceforge.net>
References: <E1C1fRX-0006zB-EJ@sc8-pr-cvs1.sourceforge.net>
Message-ID: <1f7befae04083012456c67acc@mail.gmail.com>

[rhettinger@users.sourceforge.net]
> Modified Files:
>        random.py
> Log Message:
> Teach the random module about os.urandom().
> 
...
> * Provide an alternate generator based on it.
...
> +    _tofloat = 2.0 ** (-7*8)    # converts 7 byte integers to floats
...
> +class HardwareRandom(Random):
...
> +    def random(self):
...
> +        return long(_hexlify(_urandom(7)), 16) * _tofloat

Feeding in more bits than actually fit in a float leads to bias due to
rounding.  Here:

"""
import random
import math
import sys

def main(n, useHR):
    from math import ldexp
    if useHR:
        get = random.HardwareRandom().random
    else:
        get = random.random
    counts = [0, 0]
    for i in xrange(n):
        x = long(ldexp(get(), 53)) & 1
        counts[x] += 1
    print counts
    expected = n / 2.0
    chisq = (counts[0] - expected)**2 / expected + \
            (counts[1] - expected)**2 / expected
    print "chi square statistic, 1 df, =", chisq
            
n, useNR = map(int, sys.argv[1:])
main(n, useNR)
"""

Running with the Mersenne random gives comfortable chi-squared values
for the distribution of bit 2**-53:

C:\Code\python\PCbuild>python temp.py 100000 0
[50082, 49918]
chi square statistic, 1 df, = 0.26896

C:\Code\python\PCbuild>python temp.py 100000 0
[49913, 50087]
chi square statistic, 1 df, = 0.30276

C:\Code\python\PCbuild>python temp.py 100000 0
[50254, 49746]
chi square statistic, 1 df, = 2.58064

Running with HardwareRandom instead gives astronomically unlikely values:

C:\Code\python\PCbuild>python temp.py 100000 1
[52994, 47006]
chi square statistic, 1 df, = 358.56144

C:\Code\python\PCbuild>python temp.py 100000 1
[53097, 46903]
chi square statistic, 1 df, = 383.65636

C:\Code\python\PCbuild>python temp.py 100000 1
[53118, 46882]
chi square statistic, 1 df, = 388.87696

One way to repair that is to replace the computation with

        return _ldexp(long(_hexlify(_urandom(7)), 16) >> 3, -BPF)

where _ldexp is math.ldexp (and BPF is already a module constant).

Of course that would also be biased on a box where C double had fewer
than BPF (53) bits of precision (but the Twister implementation would
show the same bias then).
From gvanrossum at gmail.com  Mon Aug 30 21:57:37 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Mon Aug 30 21:57:48 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <20040830192355.GA11836@rogue.amk.ca>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>
	<20040809211517.GB11199@rogue.amk.ca>
	<20040830192355.GA11836@rogue.amk.ca>
Message-ID: <ca471dc2040830125729fa790d@mail.gmail.com>

> > * Remove TERMIOS, mpz, statcache, xreadlines, rotor.
> >   (Unchanged from my earlier proposal.)
> >
> >   These modules have raised DeprecationWarning for a while, but only
> >   rotor is listed in PEP 4.  I'll add them to the PEP in any event;
> >   is not being listed sufficient grounds for delaying their removal to
> >   2.5?
> >
> >   (I would say we don't need to wait until 2.5; they've been raising
> >   DeprecationWarnings for a long time, so users should be well aware
> >   the modules' days are numbered.)
> >
> > * Leave rfc822, mimetools alone; the stdlib will have to avoid their use
> >   first, and I'm not going to do that right now.
> >
> > * Make mimify, MimeWriter raise DeprecationWarnings; nothing in the
> >   stdlib uses them.

All +1 from me. (I still haven't started using the new email package,
so I'm reluctant to see my beloved rfc822.py disappearing in the
future; but I trust its fate is sealed, and I'm happy it gets an
extension. :-)


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
Ask me about gmail.
From nas at arctrix.com  Mon Aug 30 22:18:16 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon Aug 30 22:18:22 2004
Subject: [Python-Dev] unicode and __str__
Message-ID: <20040830201816.GA32565@mems-exchange.org>

With Python 2.4:

    >>> u = u'\N{WHITE SMILING FACE}'
    >>> class A:
    ...   def __str__(self):
    ...     return u
    ... 
    >>> class B:
    ...   def __unicode__(self):
    ...     return u
    ... 
    >>> u'%s' % A()
    u'\u263a'
    >>> u'%s' % B()
    u'\u263a'

With Python 2.3:

    >>> u'%s' % A()
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    UnicodeEncodeError: 'ascii' codec can't encode character u'\u263a' in
        position 0: ordinal not in range(128)
    >>> u'%s' % B()
    u'<__main__.B instance at 0x401f910c>'

The only thing I found in the NEWS file that seemed relevant is
this note:

  u'%s' % obj will now try obj.__unicode__() first and fallback to
  obj.__str__() if no __unicode__ method can be found.

I don't think that describes the behavior difference.  Allowing
__str__ return unicode strings seems like a pretty noteworthy
change (assuming that's what actually happened).

Also, I'm a little unclear on the purpose of the __unicode__ method.
If you can return unicode from __str__ then why would I want to
provide a __unicode__ method?  Perhaps it is meant for objects that
can either return a unicode or a string representation depending on
what the caller prefers.  I have a hard time imagining a use for
that.

  Neil
From tim.peters at gmail.com  Mon Aug 30 22:41:10 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Mon Aug 30 22:41:13 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <20040830201816.GA32565@mems-exchange.org>
References: <20040830201816.GA32565@mems-exchange.org>
Message-ID: <1f7befae040830134170e54697@mail.gmail.com>

[Neil Schemenauer]
> ...
> The only thing I found in the NEWS file that seemed relevant is
> this note:
>
>  u'%s' % obj will now try obj.__unicode__() first and fallback to
>  obj.__str__() if no __unicode__ method can be found.
>
> I don't think that describes the behavior difference.  Allowing
> __str__ return unicode strings seems like a pretty noteworthy
> change (assuming that's what actually happened).

It's confusing.  A __str__ method or tp_str type slot can return
unicode, but what happens after that depends on the caller. 
PyObject_Str() and PyObject_Repr() try to encode it as an 8-bit string
then.  But unicode.__mod__ says "oh, cool -- I'm done".

> Also, I'm a little unclear on the purpose of the __unicode__ method.
> If you can return unicode from __str__ then why would I want to
> provide a __unicode__ method? 

Is the purpose clearer if you purge your mind of the belief that str()
(as opposed to __str__!) can return unicode?  Here w/ current CVS:

>>> class A:
...     def __str__(self): return u'a'
>>> print A()
a
>>> type(str(A()))
<type 'str'>
>>>

>>> class A:
...     def __str__(self): return u'\u1234'
>>> print A()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character u'\u1234' in
position 0: ordinal not in range(128)
>>>

>>> '%s' % A()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character u'\u1234' in
position 0: ordinal not in range(128)

>>> u'%s' % A()
u'\u1234'
>>>

So unicode.__mod__ is what's special here,  But not sure that helps <wink>.
From martin at v.loewis.de  Mon Aug 30 22:57:36 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug 30 22:57:35 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <20040830201816.GA32565@mems-exchange.org>
References: <20040830201816.GA32565@mems-exchange.org>
Message-ID: <413394C0.3090005@v.loewis.de>

Neil Schemenauer wrote:
> Also, I'm a little unclear on the purpose of the __unicode__ method.
> If you can return unicode from __str__ then why would I want to
> provide a __unicode__ method?  Perhaps it is meant for objects that
> can either return a unicode or a string representation depending on
> what the caller prefers.  I have a hard time imagining a use for
> that.

It's what unicode() returns:

 >>> class A:
...   def __str__(self):return "1"
...   def __unicode__(self): return u"2"
...
 >>> a=A()
 >>> str(a)
'1'
 >>> unicode(a)
u'2'

str() is guaranteed to return a byte string; unicode() is guaranteed
to return a unicode string.

Regards,
Martin
From nas at arctrix.com  Mon Aug 30 23:23:18 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Mon Aug 30 23:23:20 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <413394C0.3090005@v.loewis.de>
References: <20040830201816.GA32565@mems-exchange.org>
	<413394C0.3090005@v.loewis.de>
Message-ID: <20040830212317.GA419@mems-exchange.org>

On Mon, Aug 30, 2004 at 10:57:36PM +0200, "Martin v. L?wis" wrote:
> It's what unicode() returns:

But unicode() will also return __str__, eg.

    >>> class A:
    ...   def __str__(self): 
    ...     return u'\u1234'
    ... 
    >>> unicode(A())
    u'\u1234'

Why would I want to use __unicode__?  Shouldn't we be heading to a
world where __str__ always returns unicode objects?

Now, say your class stores unicode character data.  You could have
__unicode__ return a unicode object and have __str__ encode it
somehow and return a str.  However, that seems like a horrible
design to me.  The class can't know what kind of encoding the caller
expects.

  Neil
From martin at v.loewis.de  Mon Aug 30 23:35:17 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Aug 30 23:35:41 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <20040830212317.GA419@mems-exchange.org>
References: <20040830201816.GA32565@mems-exchange.org>
	<413394C0.3090005@v.loewis.de>
	<20040830212317.GA419@mems-exchange.org>
Message-ID: <41339D95.7050507@v.loewis.de>

Neil Schemenauer wrote:
> But unicode() will also return __str__, eg.
> 
>     >>> class A:
>     ...   def __str__(self): 
>     ...     return u'\u1234'
>     ... 
>     >>> unicode(A())
>     u'\u1234'
> 
> Why would I want to use __unicode__?  

This class is incorrect: it does not support str().

> Shouldn't we be heading to a
> world where __str__ always returns unicode objects?

No. In some cases, str() needs to compromise, where unicode() doesn't.

> Now, say your class stores unicode character data.  You could have
> __unicode__ return a unicode object and have __str__ encode it
> somehow and return a str.  However, that seems like a horrible
> design to me.

Perhaps. What are you proposing to do about this? Ban, from the face
of the earth, what seems like a horrible design to you?

Regards,
Martin
From skip at pobox.com  Mon Aug 30 23:36:18 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon Aug 30 23:36:36 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <20040830212317.GA419@mems-exchange.org>
References: <20040830201816.GA32565@mems-exchange.org>
	<413394C0.3090005@v.loewis.de>
	<20040830212317.GA419@mems-exchange.org>
Message-ID: <16691.40402.872902.297882@montanaro.dyndns.org>


    Neil> Shouldn't we be heading to a world where __str__ always returns
    Neil> unicode objects?

Which reminds me...  Didn't someone (MAL?) add an experimental configure
flag or something similar to allow building Python with PyString* ==
PyUnicode* (or something like that)?  I think the idea was to see what needs
to be fixed before Python can become 100% Unicode.  Last I recall it was
lots.

Skip
From nas at arctrix.com  Tue Aug 31 00:38:52 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue Aug 31 00:38:55 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <41339D95.7050507@v.loewis.de>
References: <20040830201816.GA32565@mems-exchange.org>
	<413394C0.3090005@v.loewis.de>
	<20040830212317.GA419@mems-exchange.org>
	<41339D95.7050507@v.loewis.de>
Message-ID: <20040830223852.GA662@mems-exchange.org>

On Mon, Aug 30, 2004 at 11:35:17PM +0200, "Martin v. L?wis" wrote:
> Neil Schemenauer wrote:
> >But unicode() will also return __str__, eg.
> >
> >    >>> class A:
> >    ...   def __str__(self): 
> >    ...     return u'\u1234'
> >    ... 
> >    >>> unicode(A())
> >    u'\u1234'
> >
> >Why would I want to use __unicode__?  
> 
> This class is incorrect: it does not support str().

Forgive me if I'm being obtuse, but I'm trying to understand the
overall Python unicode design.  This works:

    >>> sys.getdefaultencoding()
    'utf-8'
    >>> str(A())
    '\xe1\x88\xb4'

Can you be more specific about what is incorrect with the above
class?

> >Shouldn't we be heading to a
> >world where __str__ always returns unicode objects?
> 
> No. In some cases, str() needs to compromise, where unicode()
> doesn't.

Sorry, I don't understand that statement.  Are you saying that we
will eventually get rid of __str__ and only have __unicode__?

[on having __str__ assume some character encoding]
> Perhaps. What are you proposing to do about this? Ban, from the face
> of the earth, what seems like a horrible design to you?

If only we could. :-)  Seriously though, I'm trying to understand
the point of __unicode__.  To me it seems to make the transition to
unicode string needlessly more complicated.

  Neil
From mwh at python.net  Tue Aug 31 00:43:20 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Aug 31 00:43:23 2004
Subject: [Python-Dev] Volunteer for next bug day?
In-Reply-To: <20040830193030.GB11836@rogue.amk.ca> (A. M. Kuchling's message
	of "Mon, 30 Aug 2004 15:30:30 -0400")
References: <20040830193030.GB11836@rogue.amk.ca>
Message-ID: <2mwtzg4587.fsf@starship.python.net>

"A.M. Kuchling" <amk@amk.ca> writes:

> Currently 2.4a3 is scheduled for Thursday Sept. 2nd.  It's been an
> age-old tradition for three months now to have a bug day the weekend
> after a release.  I won't be organizing this one because I can't
> guarantee being around all day on Saturday.  Anyone want to volunteer
> to run it?  (Tasks: update the Wiki page; send out announcements to
> python-dev, c.l.py.a.; be on IRC during the bug day to answer
> questions and help people.)

Um, er, maybe.  I need to think about my diary :-)

And no offense, but if I'm running it, I'll run it my own way...

Cheers,
mwh

-- 
 "Well, the old ones go Mmmmmbbbbzzzzttteeeeeep as they start up and
  the new ones go whupwhupwhupwhooopwhooooopwhooooooommmmmmmmmm."
                         -- Graham Reed explains subway engines on asr
From scav at blueyonder.co.uk  Tue Aug 31 00:59:39 2004
From: scav at blueyonder.co.uk (Peter Harris)
Date: Tue Aug 31 00:44:23 2004
Subject: [Python-Dev] PEP 309 updated slightly
In-Reply-To: <20040830192447.4AB841E400A@bag.python.org>
References: <20040830192447.4AB841E400A@bag.python.org>
Message-ID: <4133B15B.702@blueyonder.co.uk>

I have updated PEP309 a little, mostly to add the SF patch numbers for 
the implementation (still waiting for someone to check it in), and to 
note that the '@' character has been claimed for decorators, although 
nobody really wanted it for PEP309 anyway.

I hope there will still be time to squeeze it in before 2.4 beta 1, and 
if there is any reason why not (apart from there has to be a cut-off 
point somewhere, which I accept), please someone let me know what work 
is still needed to make it ready to go in.

Thanks,

Peter Harris
scav at blueyonder co (uk)



From martin at v.loewis.de  Tue Aug 31 06:56:26 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 31 06:56:35 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <16691.40402.872902.297882@montanaro.dyndns.org>
References: <20040830201816.GA32565@mems-exchange.org>
	<413394C0.3090005@v.loewis.de>
	<20040830212317.GA419@mems-exchange.org>
	<16691.40402.872902.297882@montanaro.dyndns.org>
Message-ID: <413404FA.2080102@v.loewis.de>

Skip Montanaro wrote:
>     Neil> Shouldn't we be heading to a world where __str__ always returns
>     Neil> unicode objects?
> 
> Which reminds me...  Didn't someone (MAL?) add an experimental configure
> flag or something similar to allow building Python with PyString* ==
> PyUnicode* (or something like that)?  I think the idea was to see what needs
> to be fixed before Python can become 100% Unicode.  Last I recall it was
> lots.

The option is -U, and it causes string literals to be treated as Unicode
objects. Currently, it gets so far as to start the interactive
interpreter.

Regards,
Martin
From martin at v.loewis.de  Tue Aug 31 07:09:40 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 31 07:09:40 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <20040830223852.GA662@mems-exchange.org>
References: <20040830201816.GA32565@mems-exchange.org>
	<413394C0.3090005@v.loewis.de>
	<20040830212317.GA419@mems-exchange.org>
	<41339D95.7050507@v.loewis.de>
	<20040830223852.GA662@mems-exchange.org>
Message-ID: <41340814.4010508@v.loewis.de>

Neil Schemenauer wrote:
> Forgive me if I'm being obtuse, but I'm trying to understand the
> overall Python unicode design.  This works:
> 
>     >>> sys.getdefaultencoding()
>     'utf-8'
>     >>> str(A())
>     '\xe1\x88\xb4'

Ah, ok, so you have changed sys.getdefaultencoding on your
installation. Doing so means that some programs will only
run on your installation, but not on others (e.g. mine).
One shouldn't change the default encoding away from ASCII
except to work around buggy applications which would fail
because of their unicode-unawareness.

> Can you be more specific about what is incorrect with the above
> class?

In the default installation, it gives a UnicodeEncodeError.

>>No. In some cases, str() needs to compromise, where unicode()
>>doesn't.
> 
> 
> Sorry, I don't understand that statement.  Are you saying that we
> will eventually get rid of __str__ and only have __unicode__?

No. Eventually, when strings are Unicode objects, the string
conversion function will return such a thing. Whether this will
be called __str__, __unicode__, or __string__, I don't know.
However, this won't happen until Python 3, and it is not clear
to me how it will look. We may also need a conversion routine
into byte strings.

> If only we could. :-)  Seriously though, I'm trying to understand
> the point of __unicode__.  To me it seems to make the transition to
> unicode string needlessly more complicated.

Why do you say that? You don't *have* to implement __unicode__
if you don't need it - just like as you don't have to implement
__len__ or __nonzero__: If your class is fine with the standard
"non-None is false", implement neither. If your conceptually
have a sequence type, implement __len__ for "empty is false".
If you have a more different class, implement __nonzero__ for
"I decide what false is".

Likewise, if you are happy with the standard '<Foo instance>',
implement neither __str__ nor __unicode__. If your class has
a canonical byte string representation, implement __str__. If
this byte string representation is not meaningful ASCII, and
if a more meaningful string representation using other Unicode
characters would be possible, also implement __unicode__. Never
rely on the default encoding being something other than ASCII,
though. Eventually, when strings are Unicode objects, you
won't be able to change it.

Regards,
Martin
From martin at v.loewis.de  Tue Aug 31 08:27:00 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Aug 31 08:26:58 2004
Subject: [Python-Dev] Rename tcl84 directory
Message-ID: <41341A34.5070606@v.loewis.de>

In order to reduce the number of places that depend on the Tcl
version, I have changed the INSTALLDIR directory of Tcl on the
Windows build to tcltk.
If you have a VC 7.1 build, please rename the tcl84 directory
to tcltk.

Unfortunately, I could not eliminate all version dependencies:
the library is still tcl84.lib.

Regards,
Martin
From mal at egenix.com  Tue Aug 31 10:23:33 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue Aug 31 10:23:44 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <20040830201816.GA32565@mems-exchange.org>
References: <20040830201816.GA32565@mems-exchange.org>
Message-ID: <41343585.6070808@egenix.com>

Neil Schemenauer wrote:
> With Python 2.4:
> 
>     >>> u = u'\N{WHITE SMILING FACE}'
>     >>> class A:
>     ...   def __str__(self):
>     ...     return u
>     ... 
>     >>> class B:
>     ...   def __unicode__(self):
>     ...     return u
>     ... 
>     >>> u'%s' % A()
>     u'\u263a'
>     >>> u'%s' % B()
>     u'\u263a'
> 
> With Python 2.3:
> 
>     >>> u'%s' % A()
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in ?
>     UnicodeEncodeError: 'ascii' codec can't encode character u'\u263a' in
>         position 0: ordinal not in range(128)
>     >>> u'%s' % B()
>     u'<__main__.B instance at 0x401f910c>'
> 
> The only thing I found in the NEWS file that seemed relevant is
> this note:
> 
>   u'%s' % obj will now try obj.__unicode__() first and fallback to
>   obj.__str__() if no __unicode__ method can be found.
> 
> I don't think that describes the behavior difference.  Allowing
> __str__ return unicode strings seems like a pretty noteworthy
> change (assuming that's what actually happened).

__str__ is indeed allowed to return Unicode objects
(and has been for quite a while).

The reason we added __unicode__ was to provide a hook for
PyObject_Unicode() to try before reverting to __str__. It is
needed because even though returning Unicode objects from
__str__ is allowed, in most cases PyObject_Str() gets to talk
to it and this API always converts Unicode to a string using
the default encoding which can easily fail.

> Also, I'm a little unclear on the purpose of the __unicode__ method.
> If you can return unicode from __str__ then why would I want to
> provide a __unicode__ method?  Perhaps it is meant for objects that
> can either return a unicode or a string representation depending on
> what the caller prefers.  I have a hard time imagining a use for
> that.

That's indeed the use case. An object might want to return
an approximate string representation in some form if ask for
a string, but a true content representation when asked for
Unicode. Because of the default encoding problems you might
run into with __str__, we need two slots to provide this kind of
functionality.

In Py3k we will probably see __str__ and __unicode__ reunite.

Now back to your original question: the change you see
in %-formatting was actually a bug fix. Python 2.3 should
have exposed the same behavior as 2.4 does now.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 31 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From amk at amk.ca  Tue Aug 31 15:16:16 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Tue Aug 31 15:16:50 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <ca471dc2040830125729fa790d@mail.gmail.com>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>
	<20040809211517.GB11199@rogue.amk.ca>
	<20040830192355.GA11836@rogue.amk.ca>
	<ca471dc2040830125729fa790d@mail.gmail.com>
Message-ID: <20040831131616.GA15320@rogue.amk.ca>

On Mon, Aug 30, 2004 at 12:57:37PM -0700, Guido van Rossum wrote:
> (I still haven't started using the new email package,
> so I'm reluctant to see my beloved rfc822.py disappearing in the
> future; but I trust its fate is sealed, and I'm happy it gets an
> extension. :-)

I don't know if you followed the subsequent thread, but removing
rfc822 is going to take quite a while.  A 2.5 problem...

Removing statcache.py breaks two modules in Lib/lib-old, cmpcache.py
and dircmp.py.  What should be done: rewrite them to use os.stat
instead of statcache, or just delete them?  The files in lib-old were
last touched six weeks ago when Tim cleaned up the whitespace; before
that they hadn't been touched since 2001 at the latest..

Wow, before Tim's whitespace n11n, lib-old/newdir.py was last touched
in August 1994.  10 years ago!

--amk

From skip at pobox.com  Tue Aug 31 15:29:38 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 31 15:30:00 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <20040831131616.GA15320@rogue.amk.ca>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>
	<20040809211517.GB11199@rogue.amk.ca>
	<20040830192355.GA11836@rogue.amk.ca>
	<ca471dc2040830125729fa790d@mail.gmail.com>
	<20040831131616.GA15320@rogue.amk.ca>
Message-ID: <16692.32066.6049.493875@montanaro.dyndns.org>


    amk> Removing statcache.py breaks two modules in Lib/lib-old,
    amk> cmpcache.py and dircmp.py.  What should be done: rewrite them to
    amk> use os.stat instead of statcache, or just delete them?  

Why not migrate statcache.py to lib-old?

Skip
From amk at amk.ca  Tue Aug 31 15:38:53 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Tue Aug 31 15:39:28 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <16692.32066.6049.493875@montanaro.dyndns.org>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>
	<20040809211517.GB11199@rogue.amk.ca>
	<20040830192355.GA11836@rogue.amk.ca>
	<ca471dc2040830125729fa790d@mail.gmail.com>
	<20040831131616.GA15320@rogue.amk.ca>
	<16692.32066.6049.493875@montanaro.dyndns.org>
Message-ID: <20040831133853.GB15320@rogue.amk.ca>

On Tue, Aug 31, 2004 at 08:29:38AM -0500, Skip Montanaro wrote:
> Why not migrate statcache.py to lib-old?

I don't really see the point of lib-old these days; we now have a
better mechanism for marking modules as outdated.  If people prefer
moving statcache to lib-old, fine; personally, I'd just delete all of
lib-old, too.

--amk

From aahz at pythoncraft.com  Tue Aug 31 15:41:04 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Aug 31 15:41:15 2004
Subject: [Python-Dev] PEP 309 updated slightly
In-Reply-To: <4133B15B.702@blueyonder.co.uk>
References: <20040830192447.4AB841E400A@bag.python.org>
	<4133B15B.702@blueyonder.co.uk>
Message-ID: <20040831134104.GA25907@panix.com>

On Mon, Aug 30, 2004, Peter Harris wrote:
>
> I have updated PEP309 a little, mostly to add the SF patch numbers for 
> the implementation (still waiting for someone to check it in), and to 
> note that the '@' character has been claimed for decorators, although 
> nobody really wanted it for PEP309 anyway.
> 
> I hope there will still be time to squeeze it in before 2.4 beta 1, and 
> if there is any reason why not (apart from there has to be a cut-off 
> point somewhere, which I accept), please someone let me know what work 
> is still needed to make it ready to go in.

What's PEP309?

(People reading python-dev do not all have all PEP numbers memorized,
and it's not reasonable to expect everyone to go look them up when it
would take you only a few moments to save everyone else time.)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy@lion.austin.ibm.com
From skip at pobox.com  Tue Aug 31 15:53:34 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Aug 31 15:54:03 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <20040831133853.GB15320@rogue.amk.ca>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>
	<20040809211517.GB11199@rogue.amk.ca>
	<20040830192355.GA11836@rogue.amk.ca>
	<ca471dc2040830125729fa790d@mail.gmail.com>
	<20040831131616.GA15320@rogue.amk.ca>
	<16692.32066.6049.493875@montanaro.dyndns.org>
	<20040831133853.GB15320@rogue.amk.ca>
Message-ID: <16692.33502.937551.7930@montanaro.dyndns.org>


    >> Why not migrate statcache.py to lib-old?

    amk> I don't really see the point of lib-old these days; 

I thought lib-old was just a convenience so people using defunct modules
could resurrect them more easily.  (I realize people can also grab them from
the CVS attic.)

Skip
From amk at amk.ca  Tue Aug 31 16:03:52 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Tue Aug 31 16:04:30 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/dist/src/Misc NEWS, 1.1125, 1.1126
In-Reply-To: <E1C292N-00084D-Q5@sc8-pr-cvs1.sourceforge.net>
References: <E1C292N-00084D-Q5@sc8-pr-cvs1.sourceforge.net>
Message-ID: <20040831140352.GC15320@rogue.amk.ca>

On Tue, Aug 31, 2004 at 06:51:03AM -0700, akuchling@users.sourceforge.net wrote:
> Add news item.
> +- The mpz, rotor, and xreadlines modules, all deprecated in earlier 
> +  versions of Python, have now been removed.
> +

Well, *that* was messier than I was expecting...  Done now.

I haven't touched the Makefiles for the PC and OS2 ports to remove
these modules; if the maintainers want me to do that, please let me
know.  I did modify the RISC-OS port, because I don't know how
responsive the maintainer is and thought it best to ensure the
Makefile up to date.

--amk
From Paul.Moore at atosorigin.com  Tue Aug 31 16:31:23 2004
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Tue Aug 31 16:31:31 2004
Subject: [Python-Dev] PEP 309 updated slightly
Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060F72@UKDCX001.uk.int.atosorigin.com>

From: Aahz

>On Mon, Aug 30, 2004, Peter Harris wrote:
>>
>> I have updated PEP309 a little,
[...]
>
>> I hope there will still be time to squeeze it in before 2.4 beta 1, and 
>> if there is any reason why not (apart from there has to be a cut-off 
>> point somewhere, which I accept), please someone let me know what work 
>> is still needed to make it ready to go in.

> What's PEP309?

PEP 309 is the "Partial Function Application" class. (Used to be referred
to as "Currying", but that was agreed not to be an accurate name).

Paul


__________________________________________________________________________
This e-mail and the documents attached are confidential and intended 
solely for the addressee; it may also be privileged. If you receive this 
e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin group 
liability cannot be triggered for the message content. Although the 
sender endeavours to maintain a computer virus-free network, the sender 
does not warrant that this transmission is virus-free and will not be 
liable for any damages resulting from any virus transmitted.
__________________________________________________________________________
From alex.nanou at gmail.com  Tue Aug 31 16:51:01 2004
From: alex.nanou at gmail.com (Alex Naanou)
Date: Tue Aug 31 16:51:13 2004
Subject: [Python-Dev] PEP 309 updated slightly
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060F72@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB8803060F72@UKDCX001.uk.int.atosorigin.com>
Message-ID: <36f88922040831075133a98188@mail.gmail.com>

though this might be a bit late, I would like to suggest something a
bit functionally different:
---cut---
class LCurry(object):
   '''
   this is the left curry class.
   '''
   def __new__(cls, func, *args, **kw):
       obj = object.__new__(cls)
       if isinstance(func, LCurry) or isinstance(func, RCurry):
           obj._curry_func = func._curry_func
           obj._curry_args = (func._curry_args[0] + args, func._curry_args[1])
           obj._curry_kw = kw = kw.copy()
           kw.update(func._curry_kw)
       else:
           obj._curry_func = func
           obj._curry_args = (args, ())
           obj._curry_kw = kw.copy()
       return obj
   def __call__(self, *args, **kw):
       self._curry_func(*self._curry_args[0] + args +
self._curry_args[1], **dict(self._curry_kw.items() + kw.items()))

--uncut--

this mainly has one thing different from the reference implementation
in the pep:
1) it is recursive
that is we can curry/partial something more than one and yet avoid the
extra function call per curry level...

IMO this is a worthwhile optimisation....

taken from: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/222061
From barry at python.org  Tue Aug 31 16:59:51 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 31 17:00:00 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <20040830192355.GA11836@rogue.amk.ca>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>
	<20040809211517.GB11199@rogue.amk.ca>
	<20040830192355.GA11836@rogue.amk.ca>
Message-ID: <1093964391.27204.0.camel@geddy.wooz.org>

On Mon, 2004-08-30 at 15:23, A.M. Kuchling wrote:
> > * Leave rfc822, mimetools alone; the stdlib will have to avoid their use 
> >   first, and I'm not going to do that right now.
> > 
> > * Make mimify, MimeWriter raise DeprecationWarnings; nothing in the
> >   stdlib uses them.

+1 on these from me, as the only realistic option.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040831/ade12f57/attachment.pgp
From anthony at interlink.com.au  Tue Aug 31 17:23:52 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Aug 31 17:27:13 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <1093964391.27204.0.camel@geddy.wooz.org>
References: <20040807205703.GA3501@rogue.amk.ca>	<1091914025.1064.48.camel@anthem.wooz.org>	<4115DD5A.2070108@interlink.com.au>	<20040809211517.GB11199@rogue.amk.ca>	<20040830192355.GA11836@rogue.amk.ca>
	<1093964391.27204.0.camel@geddy.wooz.org>
Message-ID: <41349808.9050704@interlink.com.au>

Barry Warsaw wrote:
> On Mon, 2004-08-30 at 15:23, A.M. Kuchling wrote:
> 
>>>* Leave rfc822, mimetools alone; the stdlib will have to avoid their use 
>>>  first, and I'm not going to do that right now.
>>>
>>>* Make mimify, MimeWriter raise DeprecationWarnings; nothing in the
>>>  stdlib uses them.
> 
> 
> +1 on these from me, as the only realistic option.

I'd like to have a really serious effort to make sure we can DW the
first group for 2.5. In particular, the mimetools.Message class is
used for a whole pile of different protocols (anything with rfc2822
style Header: Value lines). To me, it doesn't make much sense that
this lives in a module called 'mimetools'.

But maybe that's just me.

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From barry at python.org  Tue Aug 31 18:42:00 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Aug 31 18:42:05 2004
Subject: [Python-Dev] Removing stuff from 2.4
In-Reply-To: <41349808.9050704@interlink.com.au>
References: <20040807205703.GA3501@rogue.amk.ca>
	<1091914025.1064.48.camel@anthem.wooz.org>
	<4115DD5A.2070108@interlink.com.au>	<20040809211517.GB11199@rogue.amk.ca>
	<20040830192355.GA11836@rogue.amk.ca>
	<1093964391.27204.0.camel@geddy.wooz.org>
	<41349808.9050704@interlink.com.au>
Message-ID: <1093970520.27211.9.camel@geddy.wooz.org>

On Tue, 2004-08-31 at 11:23, Anthony Baxter wrote:

> I'd like to have a really serious effort to make sure we can DW the
> first group for 2.5. In particular, the mimetools.Message class is
> used for a whole pile of different protocols (anything with rfc2822
> style Header: Value lines). To me, it doesn't make much sense that
> this lives in a module called 'mimetools'.
> 
> But maybe that's just me.

Naw, I'm with you.  I'd hoped to get farther along for all this stuff
for 2.4, but I didn't expect to get so distracted <292 winks>. :)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040831/35f9c9a0/attachment-0001.pgp
From nas at arctrix.com  Tue Aug 31 20:41:16 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue Aug 31 20:41:19 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <41343585.6070808@egenix.com>
References: <20040830201816.GA32565@mems-exchange.org>
	<41343585.6070808@egenix.com>
Message-ID: <20040831184116.GA3994@mems-exchange.org>

On Tue, Aug 31, 2004 at 10:23:33AM +0200, M.-A. Lemburg wrote:
> __str__ is indeed allowed to return Unicode objects
> (and has been for quite a while).
[...]
> Now back to your original question: the change you see
> in %-formatting was actually a bug fix. Python 2.3 should
> have exposed the same behavior as 2.4 does now.

I think the note in NEWS is not quite accurate.  It says:

    Let u'%s' % obj try obj.__unicode__() first and fallback to
    obj.__str__().

The change in PyUnicode_Join replaces a PyObject_Str() call with
PyObject_Unicode().  That means that this works in 2.4:

    class A:
        def __str__(self):
            return u'\u1234'

    u'%s' % A()

Perhaps that is unintentional but it seems cleaner to me than adding
a __unicode__ method and not providing a __str__ method.

  Neil
From mal at egenix.com  Tue Aug 31 20:58:31 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue Aug 31 20:58:35 2004
Subject: [Python-Dev] unicode and __str__
In-Reply-To: <20040831184116.GA3994@mems-exchange.org>
References: <20040830201816.GA32565@mems-exchange.org>	<41343585.6070808@egenix.com>
	<20040831184116.GA3994@mems-exchange.org>
Message-ID: <4134CA57.8050100@egenix.com>

Neil Schemenauer wrote:
> On Tue, Aug 31, 2004 at 10:23:33AM +0200, M.-A. Lemburg wrote:
> 
>>__str__ is indeed allowed to return Unicode objects
>>(and has been for quite a while).
> 
> [...]
> 
>>Now back to your original question: the change you see
>>in %-formatting was actually a bug fix. Python 2.3 should
>>have exposed the same behavior as 2.4 does now.
> 
> 
> I think the note in NEWS is not quite accurate.  It says:
> 
>     Let u'%s' % obj try obj.__unicode__() first and fallback to
>     obj.__str__().
> 
> The change in PyUnicode_Join replaces a PyObject_Str() call with

I think you meant PyUnicode_Format().

> PyObject_Unicode().  That means that this works in 2.4:
> 
>     class A:
>         def __str__(self):
>             return u'\u1234'
> 
>     u'%s' % A()
> 
> Perhaps that is unintentional but it seems cleaner to me than adding
> a __unicode__ method and not providing a __str__ method.

This is intended. However, you'll find that calling str(A())
will not give you a Unicode object, but an exception instead.

The reason is that even though the __str__ method may pass
back a Unicode object, PyObject_Str() still assures that
its return value is a string (and that API is used by str()).

Basically, __unicode__ is to unicode() what __str__ is
to str().

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 31 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From raymond.hettinger at verizon.net  Tue Aug 31 21:25:40 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue Aug 31 21:27:12 2004
Subject: [Python-Dev] PEP 309 updated slightly
In-Reply-To: <4133B15B.702@blueyonder.co.uk>
Message-ID: <000401c48f90$6c7fe480$e841fea9@oemcomputer>

> I hope there will still be time to squeeze it in before 2.4 beta 1

Is it the case that partial() cannot perform as well as an equivalent
lambda expression?



Raymond

From scav at blueyonder.co.uk  Tue Aug 31 21:38:46 2004
From: scav at blueyonder.co.uk (Peter Harris)
Date: Tue Aug 31 21:38:51 2004
Subject: [Python-Dev] Re: What is PEP309?
In-Reply-To: <20040831164226.96D151E4008@bag.python.org>
References: <20040831164226.96D151E4008@bag.python.org>
Message-ID: <4134D3C6.8090001@blueyonder.co.uk>

Aahz wrote:

>What's PEP309?
>
>(People reading python-dev do not all have all PEP numbers memorized,
>and it's not reasonable to expect everyone to go look them up when it
>would take you only a few moments to save everyone else time.)
>  
>
OK, sorry about that.

PEP309 is for a class that emulates partial function application (a bit 
like currying in
languages that do it that way, but not exactly).

It lets you make callable wrappers round a callable object, with some 
arguments 'frozen in'.

#e.g.  classes and functions can both be used:
win = Tkinter.Tk()   # my window
blueButn = partial(Tkinter.Button, win, fg='blue')   # factory for blue 
buttons in my window
def callback(name):
    print name,"pressed"
b1 = blueButn(text="Hello", command=partial(callback,"hello"))
b2 = blueButn(text="Goodbye", command=partial(callback,"goodbye"))

The PEP proposes a module 'functional' for stuff (like partial) that 
operates on or returns
other functions.  I expect there are a few things that might belong 
there, but only 'partial' is in
the proposal.

Peter Harris

From scav at blueyonder.co.uk  Tue Aug 31 22:24:29 2004
From: scav at blueyonder.co.uk (Peter Harris)
Date: Tue Aug 31 22:24:33 2004
Subject: [Python-Dev] Right curry considered harmful
In-Reply-To: <20040831164226.96D151E4008@bag.python.org>
References: <20040831164226.96D151E4008@bag.python.org>
Message-ID: <4134DE7D.9020204@blueyonder.co.uk>

Alex Nanouu wrote:

>though this might be a bit late, I would like to suggest something a
>bit functionally different:
>---cut---
>class LCurry(object):
>   '''
>   this is the left curry class.
>   '''
>   def __new__(cls, func, *args, **kw):
>       obj = object.__new__(cls)
>       if isinstance(func, LCurry) or isinstance(func, RCurry):
>           obj._curry_func = func._curry_func
>           obj._curry_args = (func._curry_args[0] + args, func._curry_args[1])
>           obj._curry_kw = kw = kw.copy()
>           kw.update(func._curry_kw)
>       else:
>           obj._curry_func = func
>           obj._curry_args = (args, ())
>           obj._curry_kw = kw.copy()
>       return obj
>   def __call__(self, *args, **kw):
>       self._curry_func(*self._curry_args[0] + args +
>self._curry_args[1], **dict(self._curry_kw.items() + kw.items()))
>
>--uncut--
>
>this mainly has one thing different from the reference implementation
>in the pep:
>1) it is recursive
>that is we can curry/partial something more than one and yet avoid the
>extra function call per curry level...
>
>IMO this is a worthwhile optimisation....
>

I think we'll see if partial() is a useful enough feature to be worth optimising once it
actually makes it into a build and gets used.

Mostly I think nesting many layers of partial(), or optimising the implementation with
that in mind will not be a win for clarity, therefore a net loss however fast you can
make it.

By the way, I think 'right curry' is just a broken idea from the point of view of code readability.  (I know the n'th argument is '2', but 'n' will depend on how many arguments are passed at run time!) The PEP doesn't propose to implement such a monster, or call partial() a curry at all, let alone a 'left' one.

Peter Harris
From alex.nanou at gmail.com  Tue Aug 31 22:59:10 2004
From: alex.nanou at gmail.com (Alex Naanou)
Date: Tue Aug 31 22:59:14 2004
Subject: [Python-Dev] Right curry considered harmful
In-Reply-To: <4134DE7D.9020204@blueyonder.co.uk>
References: <20040831164226.96D151E4008@bag.python.org>
	<4134DE7D.9020204@blueyonder.co.uk>
Message-ID: <36f88922040831135974892728@mail.gmail.com>

On Tue, 31 Aug 2004 21:24:29 +0100, Peter Harris <scav@blueyonder.co.uk> wrote:
> I think we'll see if partial() is a useful enough feature to be worth optimising once it
> actually makes it into a build and gets used.

I for one use the beast quite allot! :)

> Mostly I think nesting many layers of partial(), or optimising the implementation with
> that in mind will not be a win for clarity, therefore a net loss however fast you can
> make it.

well, about clarity, the implementation I posted above is actual code
form the library I am writing (pli.sf.net), I could write a clearer
and *prettier* version with the same functionality if desired...
...why intentionally make an under-designed version when adding some
minor optimisations would not impact either clarity nor efficiency
(IMHO).


> By the way, I think 'right curry' is just a broken idea from the point of view of 
> code readability.  (I know the n'th argument is '2', but 'n' will depend on how 
> many arguments are passed at run time!) The PEP doesn't propose to implement 
> such a monster, or call partial() a curry at all, let alone a 'left' one.

in part I do agree, but I had to implement both as going the other way
would have been a big (and I do mean BIG) loss in both clarity an
flexibility.

---
Alex.
From ncoghlan at iinet.net.au  Tue Aug 31 23:00:43 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue Aug 31 23:02:10 2004
Subject: [Python-Dev] Auto-str and auto-unicode in join
In-Reply-To: <4131BC2A.9060206@egenix.com>
References: <002401c48bb5$765463c0$e841fea9@oemcomputer>	<1f7befae04082615206760c806@mail.gmail.com>	<412E77EF.80007@iinet.net.au>	<1f7befae040828185146444b6d@mail.gmail.com>
	<4131BC2A.9060206@egenix.com>
Message-ID: <4134E6FB.2070104@iinet.net.au>

M.-A. Lemburg wrote:
> Tim Peters wrote:
> 
>> If we were to do auto-str, it would be better to rewrite str.join() as
>> a 1-pass algorithm, using the kind of "double allocated space as
>> needed" gimmick unicode.join uses.  It would be less efficient if
>> auto-promotion to Unicode turns out to be required, but it's hard to
>> measure how little I care about that; it might be faster if auto-str
>> and Unicode promotion aren't needed (as only 1 pass would be needed).
> 
> 
> FWIW, I'm -1 on doing auto-conversion using str()/unicode() in
> .join().

I'm leaning that way myself. The 'auto-str' behaviour gives significant 
speed increases over the ''.join([str(x) for x in seq]) idiom, but the 
semantics become tricky when unicode, strings and things which are 
neither get mixed in the sequence. It also increases the complexity of 
the C code for the join methods.

(Note that the version of auto-str I tried out *doesn't* actually mimic 
the list comprehension idiom - it tries to detect unicode and switch 
join implementations as appropriate, etc, etc.)

Cheers,
Nick.

From nickm at alum.mit.edu  Mon Aug 30 18:50:27 2004
From: nickm at alum.mit.edu (Nick Mathewson)
Date: Wed Sep  1 12:23:40 2004
Subject: [Python-Dev] os.urandom API
In-Reply-To: <000701c48e07$f9cea040$e841fea9@oemcomputer>
Message-ID: <20040830165027.GE29743@totoro.wangafu.net>

On Sun Aug 29 22:37:57 2004, Raymond Hettinger wrote:
> I would like to change the API for the new os.urandom(n) function to
> return a long integer instead of a string.  The former better serves
> more use cases and fits better with existing modules.

With all respect, I disagree.  As a potential user, and as lead
developer of a particularly crypto-heavy Python app
(http://mixminion.net/), I'd have to say an interface that returned a
long integer would not serve my purposes very well at all.

For most crypto apps, you never use the output of your strong entropy
source directly.  Instead, you use your strong entropy source to
generate seeds for (cryptographically strong) PRNGs, to generate keys
for your block ciphers, and so on.  Nearly all of these modules expect
their keys as a sequence of bits, which in Python corresponds more
closely to a character-string than to an arbitrary long.

> In favor of a long integer:
> 
> 1) The call random.seed(os.random(100)) is a likely use case.  If the
> intermediate value is a string, then random.seed() will hash it and
> only use 32 bits.  If the intermediate value is a long integer, all
> bits are used.  In the given example, the latter is clearly what the
> user expects (otherwise, they would only request 4 bytes).

Plausible, but as others indicate, it isn't hard to write:

    random.seed(long(hexlify(os.urandom(100)), 16)

And if you think it's really important, you could change
random.seed(None) to get the default seed in this way, instead of
looking at time.time.

But this isn't the primary use case of cryptographically strong
entropy: The Mersenne Twister algorithm isn't cryptographically
secure.  If the developer wants a cryptographically strong PRNG, she
shouldn't be using random.random().  If she doesn't want a
cryptographically strong PRNG, it's overkill for her to use
os.urandom(), and overkill for her to want more than 32 bits of
entropy anyway.

> 2) Another likely use case is accessing all the tools in the random
> module with a subclass that overrides random() and getrandbits().
> Both can be done easier and faster if os.random() returns long
> integers.  If the starting point is a string, the code gets ugly and
> slow.

As above, this isn't the way people use strong entropy in
well-designed crypto applications.  You use your strong entropy to
seed a strong PRNG, and you plug your strong PRNG into a subclass
overriding random() and getrandbits().

I agree that somebody might decide to just use os.urandom directly as
a shortcut, but such a person isn't likely to care about whether her
code is slow -- a good PRNG should outperform calls to os.urandom by
an order of magnitude or two.

 [....]
> 
> In favor of a string of bytes:
> 
> 1) This form is handy for cyptoweenies to xor with other byte strings
> (perhaps for a one-time pad).


2) By returning the result of the OS's random function directly, we
   make it easier for cryptoweenies to assure themselves that their
   entropy is good.  If it gets massaged into a long, then skeptical
   cryptoweenies will have that much more code to audit.

3) Most crypto libraries don't currently support keying from longs,
   and (as noted in other posts) the long->string conversion isn't as
   easy or clean as the string->long conversion.

yrs,
-- 
Nick Mathewson
(PGP key changed on 15Aug2004; see http://wangafu.net/key.txt)