I am wondering if it would be possible to include psutil
(https://pypi.python.org/pypi/psutil ) in the standard library, and if
not, what would be needed.
I am not a developer of it, but I am using psutil at work with good
success. it provides a good deal of services for querying and managing
processes in a cross platform way.
Any thoughts?
Since the introduction of context managers, using the lock object itself
has become easier and safer (in that there's very little chance of
forgetting to release a lock anymore). A big annoyance remains though:
relation between lock and lockee remains informal and there is no
structured way to indicate:
1. whether a state-set is protected by a lock
2. which lock protects the state-set
3. which state-set is protected by a lock
Some languages (e.g. Java) have tried to solve 2 and 3 using …
[View More]intrinsic
locks, however
* that does not solve 1 (and it becomes impossible to informally look
for lock objects lying around and try to find corresponding
state-sets)
* it does not help much when state isn't coalesced in a single object,
and for state hierarchies there is no way to express whether the whole
hierarchy should be protected under the same lock (the root's) or each
leaf should be locked individually. AFAIK intrinsic locks are not
hierarchical themselves
* things get very awkward when using alternate concurrency-management
strategies such as explicit locks for security reason[0], the non-use
of intrinsic locks has to be again documented informally
A fairly small technical change I've been considering to improve this
situation is to store the state-set inside the lock object, and only
yield it through the context manager: that the state-set is protected
by a lock is made obvious, and so is the relation between lock and
state-set. I was delighted to discover that Rust's sync::Mutex[1] and
sync::RWLock[2] follow a very similar strategy of owning the state-set
It's not a panacea, it doesn't fix issues of lock acquisition ordering
for instance, but I think it would go a fairly long way towards making
correct use of locks easier in Python.
The basic changes would be:
* threading.Lock and threading.RLock would now take an optional
`data` parameter
* the parameter would be stored internally and not directly accessible
by users of the lock
* that parameter would be returned by __enter__ and provided to the
current "owner" of the lock
These should cause no forward-compatibility issues, Lock() currently
takes no arguments, and its __enter__ returns no value.
Possible improvements/questions/issues I can see:
* with Lock, the locked state would not be available unless using as
a context manager. RLock could allow getting the protected state
only while locked by the current thread
* as-is, the scheme requires mutable state as it's not possible to
swap the internal state entirely. RLock could allow
state-replacement when locked
* because Python has no ownership concept, it would be possible for
a consumer to keep a reference to the locked state and manipulate
it without locking
I don't consider the third issue to be huge, it could be mitigated by
yielding a proxy to the internal state only valid for the current lock
span. However I do not know if it's possible to create completely
transparent proxies in Python.
The first two issues are slightly more troubling and could be mitigated
by yielding not the state-set alone but a proxy object living only for
the current lock span (or both the state-set and a proxy object) that
proxy would allow getting and setting the state-set, and would error-out
after unlocking.
Lock.acquire() could be altered to return the same proxy (or (state-set,
proxy) pair) however it's currently defined as returning either True or
False so that'd be a backwards- incompatible change. An alternative
would be to add a new acquisition method or a new flag parameter
changing the return value from True to these on acquisition.
A drawback of this additional change is that it would require the lock
object to keep track of the current live proxy(/proxies for rlock?), and
invalidate it(/them) on unlocking, increasing its complexity much more
than just adding a new attribute.
Thoughts?
[0] https://www.securecoding.cert.org/confluence/display/java/LCK00-J.+Use+priv…
[1] http://doc.rust-lang.org/sync/struct.Mutex.html
[2] http://doc.rust-lang.org/sync/struct.RWLock.html
[View Less]
(For the avoidance of all doubt, the following has nothing to do with
dunder methods like int.__add__ etc., but only the functions in the
operator module.)
The operator module contains various function versions of operators,
e.g. operator.add for + , operator.sub for - , etc.
There are also dunder versions of each function, and according to the
documentation:
The function names are those used for special class methods;
variants without leading and trailing __ are also provided …
[View More]for
convenience.
https://docs.python.org/3/library/operator.html
(except that's not quite right, because there are no functions __radd__
etc. matching the special class methods).
The existence of redundant dunder functions came as a complete surprise
to me when I recently discovered it, and I daresay it will come as a
surprise to many people. Having two versions in the operator module has
been known to lead to confusion, e.g.:
https://mail.python.org/pipermail/tutor/2014-October/102877.html
All the third party documentation I've seen for the operator module
refers to the dunderless versions.
I doubt that there is anyone who prefers to write operator.__add__
instead of operator.add, and Terry Reedy has kindly determined that
(aside from tests) the standard library doesn't use any of the dunder
versions.
https://mail.python.org/pipermail/python-list/2014-October/679226.html
I propose a few things:
* institute a policy that, in the event of a new function being added
to the operator module, only the dunderless version will be added;
* change the documentation to make it clear that the dunderless
versions should be used, rather than merely being "convenience"
functions;
* add a prominent note that the dunder versions exist for backwards
compatibility only and should not be used in new code.
At this point, I do NOT propose having the dunder versions of the
functions raise a depreciation warning. I would be content for them to
merely be documented as discouraged, and defer actual depreciation until
Python 5000 or so. (But if there is a strong preference for actual
deprecation, I won't argue against it.)
Thoughts?
--
Steven
[View Less]
This happens to me with some frequency:
result = f(args)
if not result: # of "if result is None:"
raise Exception(...)
What if I could just say?
result = f(args) or raise Exception(...)
I think that's pretty idiomatic. What's more, this would give you a
shorter syntax for things like:
if x <= y:
raise Exception(...)
Which you could write simply as:
x > y or raise Exception(...)
Effectively covering the use case of this proposal
http://article.…
[View More]gmane.org/gmane.comp.python.ideas/26005/
Has this ever been considered?
Javier
[View Less]