[Patches] [ python-Patches-941881 ] PEP309 Partial implementation
SourceForge.net
noreply at sourceforge.net
Mon Feb 28 00:05:45 CET 2005
Patches item #941881, was opened at 2004-04-25 20:05
Message generated for change (Comment added) made by loewis
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=941881&group_id=5470
Category: Library (Lib)
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: Hye-Shik Chang (perky)
Assigned to: Nobody/Anonymous (nobody)
Summary: PEP309 Partial implementation
Initial Comment:
This patch implements functional module which is
introduced by PEP309. It has only 'partial' function as
its member in this stage.
Unittest code is copied and modified slightly from
Patch #931010 by Peter Harris.
----------------------------------------------------------------------
>Comment By: Martin v. Löwis (loewis)
Date: 2005-02-28 00:05
Message:
Logged In: YES
user_id=21627
It turns out that the documentation is incorrect, wrt. this
patch, as it mentions a non-existing class functional.Partial.
----------------------------------------------------------------------
Comment By: Raymond Hettinger (rhettinger)
Date: 2005-02-27 21:43
Message:
Logged In: YES
user_id=80475
Exposing the structure members for dynamic alteration exceeds
the PEP specification. While making the tool more flexible, it
slows it down (requiring type checks for every call) and it
encourages strange uses. The notion of giving partial objects a
changable state is at odds with the basis for functional
programming (i.e. statelessness).
The traverse function should use the new Py_VISIT macro.
The PyCallable_Check() is unnecessary as it duplicates the
check already existing in PyObject_Call().
I am not certain whether the PyDict_Copy() is necessary. Calls
to f(**d) already pass a copy of d. Likewise, calling f(a=1) will
create a new dictionary. If so, that dictionary can be updated in-
place with pto->kw rather than creating a new dictionary.
None of these issues are critical. Feel free to go forward with the
patch.
----------------------------------------------------------------------
Comment By: Paul Moore (pmoore)
Date: 2005-02-27 21:43
Message:
Logged In: YES
user_id=113328
To build on Windows (at least with mingw gcc, I don't know
about MSVC) you need to assign members of partial_type which
are symbols in the Python DLL at runtime, not compile time.
The following diff probably explains better:
--- functionalmodule.c.orig 2005-02-27
20:41:41.000000000 +000
+++ functionalmodule.c 2005-02-26 21:41:14.000000000 +0000
@@ -197,7 +197,7 @@
0, /* tp_hash */
(ternaryfunc)partial_call, /* tp_call */
0, /* tp_str */
- PyObject_GenericGetAttr, /* tp_getattro */
+ 0, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC |
@@ -220,7 +220,7 @@
0, /* tp_init */
0, /* tp_alloc */
partial_new, /* tp_new */
- PyObject_GC_Del, /* tp_free */
+ 0, /* tp_free */
};
@@ -239,6 +239,8 @@
int i;
PyObject *m;
char *name;
+ partial_type.tp_getattro = PyObject_GenericGetAttr;
+ partial_type.tp_free = PyObject_GC_Del;
PyTypeObject *typelist[] = {
&partial_type,
NULL
----------------------------------------------------------------------
Comment By: Nick Coghlan (ncoghlan)
Date: 2005-02-26 06:27
Message:
Logged In: YES
user_id=1038590
Moving the discussion to python-dev :)
----------------------------------------------------------------------
Comment By: Steven Bethard (bediviere)
Date: 2005-02-26 04:58
Message:
Logged In: YES
user_id=945502
Oops -- wrong order of association.
... self.args = args[2:] + func.args
... d = func.kw.copy()
... d.update(kw)
should have been
... self.args = func.args + args[2:]
... kw.update(func.kw)
... self.kw = kw
hopefully these errors didn't confuse my intent too much.
----------------------------------------------------------------------
Comment By: Steven Bethard (bediviere)
Date: 2005-02-26 04:53
Message:
Logged In: YES
user_id=945502
Yes, that's much clearer. I don't remember why I couldn't
get it to work this way before (I tried something quite
similar to this), but I'm glad someone could figure out the
simpler way to do it. ;-)
One potential problem with my proposal -- it doesn't work
for nested partials:
py> class C(object):
... pass
...
py> def func(self, arg1, arg2):
... return self, arg1, arg2
...
py> setattr(C, 'a', partial(func, 'a', 'b'))
py> C().a()
(<__main__.C object at 0x01186470>, 'a', 'b')
py> setattr(C, 'a', partial(partial(func, 'a'), 'b'))
py> C().a()
('a', <__main__.C object at 0x01186370>, 'b')
One possible solution would be to merge nested partials, but
I'm not at all certain this solves all the problems:
py> class partial(object):
... def __init__(*args, **kw):
... self = args[0]
... func = args[1]
... if isinstance(func, partial):
... self.fn = func.fn
... self.args = args[2:] + func.args
... d = func.kw.copy()
... d.update(kw)
... self.kw = d
... else:
... self.fn, self.args, self.kw = (args[1],
args[2:], kw)
... def __call__(self, *args, **kw):
... if kw and self.kw:
... d = self.kw.copy()
... d.update(kw)
... else:
... d = kw or self.kw
... return self.fn(*(self.args + args), **d)
... def __get__(self, obj, type=None):
... if obj is None:
... return self
... return partial(self.fn, obj, *self.args, **self.kw)
...
py> class C(object):
... pass
...
py> def func(self, arg1, arg2):
... return self, arg1, arg2
...
py> setattr(C, 'a', partial(func, 'a', 'b'))
py> C().a()
(<__main__.C object at 0x01186BB0>, 'a', 'b')
py> setattr(C, 'a', partial(partial(func, 'a'), 'b'))
py> C().a()
(<__main__.C object at 0x01186650>, 'b', 'a')
On the other hand, merging nested partials does reduce the
number of function calls, so perhaps it's useful in and of
itself...
----------------------------------------------------------------------
Comment By: Nick Coghlan (ncoghlan)
Date: 2005-02-26 04:45
Message:
Logged In: YES
user_id=1038590
If the __get__ method is added to make partial functions
behave like instance methods, the documentation should
probably mention that classmethod() and staticmethod() can
then be used to alter the behaviour:
Py> def f(*args):
... print args
...
Py> class C:
... a = functional.partial(f)
... b = classmethod(functional.partial(f))
... c = staticmethod(functional.partial(f))
...
Py> C().a
<functional.partial object at 0x009E0DD0>
Py> C().a()
(<__main__.C instance at 0x009E6698>,)
Py> C().b()
(<class __main__.C at 0x009DDB40>,)
Py> C().c()
()
(This example used the simpler __get__ implementation I
posted earlier)
----------------------------------------------------------------------
Comment By: Nick Coghlan (ncoghlan)
Date: 2005-02-26 04:27
Message:
Logged In: YES
user_id=1038590
Steve's suggestion does sound good, but can't it simply be
implemented with the PEP implementation and the following
__get__ method?:
def __get__(self, obj, type=None):
if obj is None:
return self
return partial(self.func, obj, *self.args, **self.kwargs)
----------------------------------------------------------------------
Comment By: Nick Coghlan (ncoghlan)
Date: 2005-02-26 04:17
Message:
Logged In: YES
user_id=1038590
Peter Harris already wrote docs in Patch 931007. Presumably
they are still accurate, or perky would have said something.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2005-02-25 21:34
Message:
Logged In: YES
user_id=21627
In this form, the patch cannot be applied, since it lacks
documentation changes. It might be that perky does not
consider himself fluent enough in English to write the
documentation - any other volunteers?
I also think there is value to bediviere's point, even
though the PEP currently specifies partial() differently.
----------------------------------------------------------------------
Comment By: Steven Bethard (bediviere)
Date: 2005-02-19 21:38
Message:
Logged In: YES
user_id=945502
It might be nice if partial objects were usable as
instancemethods, like functions are. Note the following
behavior with the current patch:
>>> import functional
>>> class C(object):
... pass
...
>>> def func(self, arg):
... return arg
...
>>> for item in ['a', 'b', 'c']:
... setattr(C, item, functional.partial(func, item))
...
>>> C().a()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: func() takes exactly 2 arguments (1 given)
If functional.partial was instead defined like:
class partial(object):
def __init__(*args, **kwargs):
self = args[0]
try:
self.func = args[1]
except IndexError:
raise TypeError('expected 2 or more arguments,
got ' %
len(args))
self.obj = ()
self.args = args[2:]
self.kwargs = kwargs
def __call__(self, *args, **kwargs):
if kwargs and self.kwargs:
d = self.kwargs.copy()
d.update(kwargs)
else:
d = kwargs or self.kwargs
return self.func(*(self.obj + self.args + args), **d)
def __get__(self, obj, type=None):
if obj is None:
return self
result = partial(self.func, *self.args, **self.kwargs)
result.obj = (obj,)
return result
where an appropriate __get__ method is provided, then
partial objects behave properly when set as attributes to a
class:
py> def test():
... class C(object):
... pass
... def func(self, arg):
... return arg
... for item in ['a', 'b', 'c']:
... setattr(C, item, functional.partial(func, item))
... c = C()
... return c.a(), c.b(), c.c()
...
py> test()
('a', 'b', 'c')
----------------------------------------------------------------------
Comment By: Nick Coghlan (ncoghlan)
Date: 2004-08-11 02:11
Message:
Logged In: YES
user_id=1038590
I have tested this patch on Windows, and it passes its own
test suite, without affecting any other tests.
However, PCBuild\pythoncore.vcproj & PC\config.c require
modification to allow Python to pick up the new module
correctly.
Patch #1006948 created with the needed changes (also removes
unneeded ODBC references from pythoncore.vcproj as I am
using the free MS toolkits to build here)
My patch definitely needs to be checked by someone with a
copy of Vis Studio 2003!
----------------------------------------------------------------------
Comment By: Paul Moore (pmoore)
Date: 2004-08-03 22:23
Message:
Logged In: YES
user_id=113328
OK, a real need beats my theoretical worries :-) Consider me
convinced.
----------------------------------------------------------------------
Comment By: Bob Ippolito (etrepum)
Date: 2004-05-18 06:05
Message:
Logged In: YES
user_id=139309
I would use partial in situations where speed can matter (imap,
commonly used event handlers, etc.), so a fast C implementation would
be nice to have.
----------------------------------------------------------------------
Comment By: Paul Moore (pmoore)
Date: 2004-04-27 10:03
Message:
Logged In: YES
user_id=113328
Yes, that looks like a significant speed improvement :-) I was
basing my assumptions on the comments made in the PEP.
Sorry.
But I still wonder if having the implementation in Python
wouldn't be better from a maintenance point of view. (As well
as all the arguments about usefulness as sample code, ability
to backport, etc etc, that have come up on python-dev
regarding moving other Python library modules into C...).
----------------------------------------------------------------------
Comment By: Hye-Shik Chang (perky)
Date: 2004-04-27 04:05
Message:
Logged In: YES
user_id=55188
Python-version (function) ... 1.19 2.69
Python-version (class) ... 2.61 2.38
C-version ... 0.50 0.37
(former value is for 100000 instanciations and latter is for
100000 calls.)
And, C version have a facility that changing attributes after
the instantiation that is supported by class version only.
----------------------------------------------------------------------
Comment By: Paul Moore (pmoore)
Date: 2004-04-26 21:30
Message:
Logged In: YES
user_id=113328
Why implement this in C? I can't imagine that the
performance improvement will be that significant. A pure
Python module in the standard library seems to me to be a
far better idea. As the PEP says, "the case for a built-in
coded in C is not very strong". And a Python module is good
self-documentation.
I prefer the function version suggested in the PEP (credited
to Carl Banks) over the class-based one. You need to take a
little care to avoid capturing argument names:
def partial(*args, **kwds):
def callit(*moreargs, **morekwds):
kw = kwds.copy()
kw.update(morekwds)
return args[0](*(args[1:]+moreargs), **kw)
return callit
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=941881&group_id=5470
More information about the Patches
mailing list