On Wed, Sep 30, 2015 at 11:28 AM, Neil Girdhar <mistersheik(a)gmail.com>
wrote:
> I found myself writing:
>
> for vertex, height in zip(
> self.cache.height_to_vertex[height_slice],
> range(height_slice.start, height_slice.stop)):
>
> I would have preferred:
>
> for height, vertex in enumerate(
> self.cache.height_to_vertex)[height_slice]:
>
This does not seem to be an big improvement over
for height, vertex in enumerate(
self.cache.height_to_vertex[height_slice],
height_slice.start):
Hi,
I noticed there is currently no standard solution to submit a job from a
thread to an asyncio event loop.
Here's what the asyncio documentation says about concurrency and
multithreading:
> To schedule a callback from a different thread, the
BaseEventLoop.call_soon_threadsafe() method should be used.
> Example to schedule a coroutine from a different thread:
> loop.call_soon_threadsafe(asyncio.async, coro_func())
The issue with this method is the loss of the coroutine result.
One way to deal with this issue is to connect the asyncio.Future returned
by async (or ensure_future) to a concurrent.futures.Future. It is then
possible to use a subclass of concurrent.futures.Executor to submit a
callback to an asyncio event loop. Such an executor can also be used to set
up communication between two event loops using run_in_executor.
I posted an implementation called LoopExecutor on GitHub:
https://github.com/vxgmichel/asyncio-loopexecutor
The repo contains the loopexecutor module along with tests for several use
cases. The README describes the whole thing (context, examples, issues,
implementation).
It is interesting to note that this executor is a bit different than
ThreadPoolExecutor and ProcessPoolExecutor since it can also submit a
coroutine function. Example:
with LoopExecutor(loop) as executor:
future = executor.submit(operator.add, 1, 2)
assert future.result() == 3
future = executor.submit(asyncio.sleep, 0.1, result=3)
assert future.result() == 3
This works in both cases because submit always cast the given function to a
coroutine. That means it would also work with a function that returns a
Future.
Here's a few topic related to the current implementation that might be
interesting to discuss:
- possible drawback of casting the callback to a coroutine
- possible drawback of concurrent.future.Future using
asyncio.Future._copy_state
- does LoopExecutor need to implement the shutdown method?
- removing the limitation in run_in_executor (can't submit a coroutine
function)
- adding a generic Future connection function in asyncio
- reimplementing wrap_future with the generic connection
- adding LoopExecutor to asyncio (or concurrent.futures)
At the moment, the interaction between asyncio and concurrent.futures only
goes one way. It would be nice to have a standard solution (LoopExecutor or
something else) to make it bidirectional.
Thanks,
Vincent
What are the pros and cons of making enumerate a sequence if its argument
is a sequence?
I found myself writing:
for vertex, height in zip(
self.cache.height_to_vertex[height_slice],
range(height_slice.start, height_slice.stop)):
I would have preferred:
for height, vertex in enumerate(
self.cache.height_to_vertex)[height_slice]:
> On 9/29/2015 9:20 AM, Rob Cliffe wrote:
> Why not
> >
> > def __init__(self, vertices=None, edges=None, weights=None,
> > source_nodes=None):
> > self.vertices = vertices if vertices is not None else []
> > self.edges = edges if edges is not None else []
> > self.weights = weights if weights is not None else {}
> > self.source_nodes = source_nodes if source_nodes is not None else []
>
> I don't understand why not:
>
> self.vertices = vertices or []
> self.edges = edges or []
> self.weights = weights or {}
> self.source_nodes = source_nodes or []
>
>
> Emile
A further virtue of
self.vertices = vertices or []
and the like is that they coerce falsy parameters of the wrong type to the falsy object of the correct type.
E.g. if vertices is '' or 0, self.vertices will be set to [], whereas the ternary expression only tests
for not-None so self.vertices will be set to a probably crazy value.
Hello everyone,
I was wondering how to split a string with multiple separators.
For instance, if I edit some subtitle file and I want the string
'00:02:34,452 --> 00:02:37,927' to become ['00', '02', '34', '452',
'00', '02', '37', '927'] I have to use split too much time and I didn't
find a "clean" way to do it.
I imagined the split function with an iterator as parameter. The string
would be split each time its substring is in the iterator.
Here is the syntax I considered for this :
>>> '00:02:34,452 --> 00:02:37,927'.split([ ':', ' --> ', ',' ])
['00', '02', '34', '452', '00', '02', '37', '927']
Is it a relevant idea ? What do you think about it ?
Regards,
Niilos.
> > A further virtue of
> >
> > self.vertices = vertices or []
> >
> > and the like is that they coerce falsy parameters of the wrong type to the falsy object of the correct type.
> > E.g. if vertices is '' or 0, self.vertices will be set to [], whereas the ternary expression only tests
> >
> > for not-None so self.vertices will be set to a probably crazy value.
>
> Doesn't seem like a virtue to me, seems like it's probably hiding a bug
> in the calling code, which may have other ramifications. Better to have
> the "crazy value" visible and fail faster, so you can go fix that bug.
>
> Carl
I have to agree. It isn't a "virtue", and it's best not to mask such mistakes. But it *is* a... property of the shorter construct that it's more forgiving, and doesn't have the same semantics.
PS -- My first post, and I lost the "Re:" in the Subject, hence this orphan thread which I'm content to see go no further.
I really like PEP 0505. The only thing that does not convince me is the `??` operator. I would like to know what you think of an alternative like `or?`:
a_list = some_list or? []
a_dict = some_dict or? {}
The rationale behind is to let `or` do its job with “truthy” values, while `or?` would require non-None values.
The rest of the PEP looks good to me.
I apologise in advance if this was already proposed and I missed it.
Regards,
Alessio
# The example works with tornado dev verison & python3.5
import tornado
from tornado.httpclient import AsyncHTTPClient
from tornado.concurrent import Future
from tornado.gen import convert_yielded
from functools import wraps
Future.__or__ = Future.add_done_callback
def future(func):
@wraps(func)
def _(*args, **kwds):
return convert_yielded(func(*args, **kwds))
return _
##############
@future
async def ping(url):
httpclient = AsyncHTTPClient()
r = await httpclient.fetch(url)
return r.body.decode('utf-8')
ping("http://baidu.com") | (
lambda r:print(r.result())
)
"""
Maybe python should support arrow syntax for easier use async call ?
Now lambda only can write one line and must have parentheses ...
FOR EXAMPLE
ping("http://baidu.com") | r ->
print(r.result())
print("something else")
I saw some discuss in https://wiki.python.org/moin/AlternateLambdaSyntax
"""
tornado.ioloop.IOLoop.instance().start()
The "str.split" and "str.rsplit" methods have a useful "maxsplit" option,
which lets you set the number of times to split, defaulting to -1 (which is
"unlimited"). The corresponding "os.path.split", however, has no
"maxsplit" option. It can only split once, which splits the last path
segment (the "basename") from the rest (equivalent of "str.rsplit" with
"maxsplit=1").
I think it would be useful if "os.path.split" also had a "maxsplit"
option. This would default to "1" (the current behavior"), but could be
set to any value allowed by "str.split". Using this option would follow
the behavior of "str.rsplit" for that value of "maxsplit".
Hi folks,
Based on the feedback in the recent threads, I've written a draft PEP
that dispenses with the userspace CSPRNG idea, and instead proposes:
* defaulting to using the system RNG for the module level random API
in Python 3.6+
* implicitly switching to the deterministic PRNG if you call
random.seed(), random.getstate() or random.setstate() (this implicit
fallback would trigger a silent-by-default deprecation warning in 3.6,
and a visible-by-default runtime warning after 2.7 goes EOL)
* providing random.system and random.seedable submodules so you can
explicitly opt in to using the one you want without having to manage
your own RNG instances
That approach would provide a definite security improvement over the
status quo, while restricting the compatibility break to a performance
regression in applications that use the module level API without
calling seed(), getstate() or setstate(). It would also allow the
current security warning in the random module documentation to be
moved towards the end of the module, in a section dedicated to
determinism and reproducibility.
The full PEP should be up shortly at
https://www.python.org/dev/peps/pep-0504/, but caching is still a
problem when uploading new PEPs, so if that 404s, try
http://legacy.python.org/dev/peps/pep-0504/
Regards,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
