I've been thinking about some ideas for reducing the
amount of refcount adjustment that needs to be done,
with a view to making GIL removal easier.
1) Permanent objects
In a typical Python program there are many objects
that are created at the beginning and exist for the
life of the program -- classes, functions, literals,
etc. Refcounting these is a waste of effort, since
they're never going to go away.
So perhaps there could be a way of marking such
objects as "permanent" or "immortal". …
[View More]Any refcount
operation on a permanent object would be a no-op,
so no locking would be needed. This would also have
the benefit of eliminating any need to write to the
object's memory at all when it's only being read.
2) Objects owned by a thread
Python code creates and destroys temporary objects
at a high rate -- stack frames, argument tuples,
intermediate results, etc. If the code is executed
by a thread, those objects are rarely if ever seen
outside of that thread. It would be beneficial if
refcount operations on such objects could be carried
out by the thread that created them without locking.
To achieve this, two extra fields could be added
to the object header: an "owning thread id" and a
"local reference count". (The existing refcount
field will be called the "global reference count"
in what follows.)
An object created by a thread has its owning thread
id set to that thread. When adjusting an object's
refcount, if the current thread is the object's owning
thread, the local refcount is updated without locking.
If the object has no owning thread, or belongs to
a different thread, the object is locked and the
global refcount is updated.
The object is considered garbage only when both
refcounts drop to zero. Thus, after a decref, both
refcounts would need to be checked to see if they
are zero. When decrementing the local refcount and
it reaches zero, the global refcount can be checked
without locking, since a zero will never be written
to it until it truly has zero non-local references
remaining.
I suspect that these two strategies together would
eliminate a very large proportion of refcount-related
activities requiring locking, perhaps to the point
where those remaining are infrequent enough to make
GIL removal practical.
--
Greg
[View Less]
first thing: properties. i prefer this approach to properties (example below).
true, it involves some magic, but the magic is very restricted and contained.
import inspect
def getter(func):
namespace = inspect.stack()[1][0].f_locals
p = namespace.get(func.func_name)
if p is None:
p = property(func)
else:
p = property(func, p.fset, p.fdel)
return p
def setter(func):
namespace = inspect.stack()[1][0].f_locals
p = namespace.get(func.func_name)
if …
[View More]p is None:
p = property(None, func)
else:
p = property(p.fget, func, p.fdel)
return p
class File(object):
def __init__(self):
self._pos = 8
@getter
def position(self):
return self._pos
@setter
def position(self, value):
self._pos = value
f = File()
print f.position
f.position = 11
print f.position
==============================================
btw, in py3k, with class decorators, you could instead use
class File:
@propertyclass
class position:
def get(self):
return self.tell()
def set(self):
return self.tell()
==============================================
second -- metaclasses. there's some notion (Josiah et al) that metaclasses
were introduced to make classes behave as generic attribute containers. this
is not true. using metaclasses like so was not the original intent.
metaclasses are the types of types. they also happened to be executed
before the class body is made into a class, which means they can be used
to tweak class creation, but that's a technicality. using this feature to turn
classes into anything you want is wrong: it's implicit, complex, and mostly
unreadable by the common programmer. i'd guess 80% of python
programmers have never used metaclasses. it's a difficult concept,
and should be used sparsely.
class decorators would cover ~70% of the use-cases for metaclasses,
but for some reason, people insist on complicating even more the
metaclass machinery in py3k. see my protest [1], which was silently ignored.
what you people are asking for is a new language construct which allows
the definition of syntactic namespaces. much like the deceased make pep.
(mis)using classes for this purpose is wrong. it's different -- it's
not a class
anymore. so why is it being defined by a class-clause?
@property
namespace foo:
def get(self):
....
def set(self):
....
-tomer
[1] http://mail.python.org/pipermail/python-3000/2007-June/008330.html
[View Less]
Hi,
Several people have felt the need to be able to 'peek ahead' for the next
value of an iterator without actually retrieving it. I have, and I have a
'peekable' class that I use to wrap around iterators when I need this
feature:
>>> it = iter('python')
>>> it = peekable(it)
>>> it.peek(), it.peek(), it.next(), it.peek(), it.next(), it.next()
('p', 'p', 'p', 'y', 'y', 't')
>>>
There is an example of implementation at:
http://aspn.activestate.com/ASPN/…
[View More]Cookbook/Python/Recipe/304373
A wrapper works fine but a lot of built-in iterators such as
listiterators, tupleiterators, dictionary-***iterators could provide a
peek() method natively, which would be much more efficient and would not
necessitate any buffering. (e.g. in the case of a listiterator, peek()
would be the same as next() but without incrementing the it_index counter)
My idea is the following :
* iterators such as the ones listed above, where providing a peek() method
does not require any buffering or extra bookkeeping, implement this method
natively.
* for the other ones, no peek() method but a user can still wrap them as
in the example above.
* to provide a common interface, define a peekable class like this:
class peekable(object):
def __new__(cls, iterator):
# if the iterator is already 'peekable', return it
# otherwise wrap it
if hasattr(iterator, 'peek'):
return iterator
else:
return object.__new__(cls)
def __init__(self, iterator):
...
def __iter__(self):
return self
def next(self):
...
def peek(self):
...
This means that an iterator would have the option of providing a peek()
method, but wouldn't have to. However all iterators could be made
peekable by writing
it = peekable(it)
--
Arnaud
[View Less]
this is to inform the readers of this group: i created a mirror group
of this
mailing list on google-groups. unlike most mailing list mirrors out
there,
which only allow you to browse the archive, the google group also
allows
you to post to a specific message/thread (so you don't have to start
new topics when you only want to reply, but don't have the original
message).
http://groups.google.com/group/python-ideas
-tomer
Hello!
In the past few months I've been toying around with .NET, C# and
PythonNet. While I still think that C# is too wory (public static
explicit operator Egg(Spam spam) { ... }) C# has one syntax feature I
really like to see in Python.
private float _a
public float a
{
get { return _a; }
set { _a = value; }
}
I think it's a very nice way to define a variable that acts similar to
Python properties. get, set and value are part of the syntax.
Python has no nice way to define a …
[View More]property with set and get. You always
have to use lambda or some private methods.
class Now:
_a = 0.0
@property
def a(self):
"""Read only property
return self._a
def _geta(self):
return self._a
def _seta(self, value):
self._a = value
a = property(_geta, _seta)
It puts a lot of methods into the body of the class that are only used
for properties. I find the C# syntax more intriguing. It puts the getter
and setter into a block of their own and makes reading and understand
the property more easy.
What do you think about this syntax?
class EasyExample:
_a = 0.0
property a:
"""doc string
"""
def get(self) -> float:
return self._a
def set(self, value):
self._a = float(value)
def delete(self):
del self._a
It may be possible to combine the feature with generic methods but I
guess that's not going to be easy.
class ComplexExample:
_a = 0.0
property a:
"""doc string
"""
def get(self) -> float:
return self._a
@generic
def set(self, value:float):
self._a = value
@generic
def set(self, value:int):
self._a = float(value)
@generic
def set(self, value:str):
self._a = float(value)
def delete(self):
del self._a
An alternative syntax. It doesn't look as clean as the other syntax but
has the benefit that the variable is in front of the definition.
class AlternativeSyntax:
_a = 0.0
a = property:
"""doc string
"""
Comments?
Christian
[View Less]