This blog post from Donald Stufft explains the difference between
a Library and a Application.
The term "Library" is well known - ok
But how to call the container for the libraries, the one thing providing the environment?
Donald Stufft uses the term "Application"
I would like an official term defined by the python docs.
What do you think?
Thomas Guettler http://www.thomas-guettler.de/
On Thu, May 19, 2016 at 11:49 AM, Ilya Kulakov <kulakov.ilya(a)gmail.com>
> I think Python would benefit from providing either a new method (e.g.
> `asyncio.current_event_loop`) or modifying `asyncio.get_event_loop`
> to return current event loop when called from a coroutine. It should, in my
> opinion, reduce necessity of passing event loop
> between coroutines in application's code as it is done in asyncio itself and
> 3rd party libraries (like aiohttp).
I was about to make the same proposal. I can add two practical arguments
First, some coroutines could be simplified. For instance, asyncio.sleep
takes an optional loop argument, defaulting to get_event_loop(). But
get_event_loop is called inside the coroutine, and therefore inside a
running event loop. This loop is actually the only valid choice, and
using anything else should raise an exception. If get_event_loop is
guaranteed to return the current running loop, then there is no reason
to pass it as an argument to asyncio.sleep. Other coroutines could
benefit from that, including wait and wait_for.
It would also help for queues, locks and other task functions (gather,
shield, as_completed, etc.). Here the loop argument is still useful, in
case those objects are declared outside their event loop. However, it
would be really convenient to assume loop=None always works as expected
if the object is created inside a coroutine. In many cases, libraries
tend to forward the loop argument from coroutine to coroutine, just in
case someone writes:
loop = asyncio.new_event_loop()
loop = asyncio.new_event_loop()
`async def coro(*, loop=None)` doesn't really make sense, and could be
discouraged if get_event_loop was guaranteed to return the running event
loop when called inside a coroutine.
I actually proposed this already in one of the pathlib threads on
python-dev, but I decided to repost here, because this is easily seen
as a separate issue. I'll start with some introduction, then moving on
to the actual type hinting part.
In our seemingly never-ending discussions about pathlib support in the
stdlib in various threads, first here on python-ideas, then even more
extensively on python-dev, have perhaps almost converged. The required
changes involve a protocol method, probably named __fspath__, which
any path-like type could implement to return a more, let's say,
"classical" path object such as a str. However, the protocol is
polymorphic and may also return bytes, which has a lot do do with the
fact that the stdlib itself is polymophic and currently accepts str as
well as bytes paths almost everywhere, including the newly-introduced
os.scandir + DirEntry combination. The upcoming improvements will
further allow passing pathlib path objects as well as DirEntry objects
to any stdlib function that take paths.
It came up, for instance here , that the function associated with
the protocol, potentially named os.fspath, will end up needing type
hints. This function takes pathlike objects and turns them into str or
bytes. There are various different scenarios  that can be
considered for code dealing with paths, but let's consider the case of
os.path.* and other traditional python path-related functions.
Currently, it takes str or bytes paths and returns a joined path of
the same type (mixing different types raises an exception).
In the future, it will also accept pathlib objects (underlying type
always str) and DirEntry (underlying type str or bytes) or third-party
path objects (underlying type str or bytes). The function will then
return a pathname of the underlying type.
Currently, it takes a str or bytes and returns the dirname of the same type.
In the future, it will also accept Path and DirEntry and return the
Let's consider the type hint of os.path.dirname at present and in the future:
Currently, one could write
def dirname(p: Union[str, bytes]) -> Union[str, bytes]:
While this is valid, it could be more precise:
pathstring = typing.TypeVar('pathstring', str, bytes)
def dirname(p: pathstring) -> pathstring:
This now contains the information that the return type is the same as
the argument type. The name 'pathstring' may be considered slightly
misleading because "byte strings" are not actually strings in Python
3, but at least it does not advertise the use of bytes as paths, which
is very rarely desirable.
But what about the future. There are two kinds of rich path objects,
those with an underlying type of str and those with an underlying type
of bytes. These should implement the __fspath__() protocol and return
their underlying type. However, we do care about what (underlying)
type is provided by the protocol, so we might want to introduce
something like typing.FSPath[underlying_type]:
FSPath[str] # str-based pathlike, including str
FSPath[bytes] # bytes-based pathlike, including bytes
And now, using the above defined TypeVar pathstring, the future
version of dirname would be type annotated as follows:
def dirname(p: FSPath[pathstring]) -> pathstring:
It's getting late. I hope this made sense :).
asyncio is a great library which with recent changes in 3.5 was made even better.
However there is an inconvenience that bothers me and looks very unnatural: inability to access event loop
that executes a coroutine from within that coroutine.
This is unnatural, because we have `self` and `cls` to access object from a method it is bound to,
we have `current_thread` to get an instance of Thread that currently executes code, but for a coroutine
we have no such method, we cannot get its context of execution.
Current implementation of `get_event_loop` method is not sufficient, it will not work if thread has more
than one event loop.
I think Python would benefit from providing either a new method (e.g. `asyncio.current_event_loop`) or modifying `asyncio.get_event_loop`
to return current event loop when called from a coroutine. It should, in my opinion, reduce necessity of passing event loop
between coroutines in application's code as it is done in asyncio itself and 3rd party libraries (like aiohttp).
This is one of those things which are so easy to implement which makes you
think it is probably not worth adding to the stdlib, but then again, this
is something I've ended up doing and rewriting pretty often over the years.
Real world example:
def __init__(self, *args, **kwargs):
def __enter__(self, *args, **kwargs):
def __exit__(self, *args, **kwargs):
def get_lock(name, bypass_lock=False):
lock_cls = DummyLock if bypass_lock else RedisLock
with get_lock('foo', bypass_lock=True):
Similarly to contextlib.closing and contextlib.suppress, perhaps it would
be nice to have contextlib.DummyContext just because it's something which
is done (I think) fairly often.
On the bikeshedding front, in order to be consistent with
closing(), redirect_stderr(), redirect_stdout() and suppress(), a better
name for this would probably be contextlib.dummy_context.
Extra: the same thing can be achieved by using mock.MagicMock, which
probably makes this proposal useless and kills it entirely. The additional
value is that it would be more explicit/clear/immediate to have this in
contextlib itself as opposed to unittest module, which is kinda weird. But
then again, "there should be (possibly) only one way to do it" so I'm not
OK, I should stop talking with myself. =)
Giampaolo - http://grodola.blogspot.com
Currently the call of function with keyword arguments (say `f(a, b, x=c,
y=d)`) is compiled to following bytecode:
0 LOAD_NAME 0 (f)
3 LOAD_NAME 1 (a)
6 LOAD_NAME 2 (b)
9 LOAD_CONST 0 ('x')
12 LOAD_NAME 3 (c)
15 LOAD_CONST 1 ('y')
18 LOAD_NAME 4 (d)
21 CALL_FUNCTION 514 (2 positional, 2 keyword pair)
For every positional argument its value is pushed on the stack, and for
every keyword argument its name and its value are pushed on the stack.
But keyword arguments are always constant strings! We can reorder the
stack, and push keyword argument values and names separately. And we can
save all keyword argument names for this call in a constant tuple and
load it by one bytecode command.
0 LOAD_NAME 0 (f)
3 LOAD_NAME 1 (a)
6 LOAD_NAME 2 (b)
9 LOAD_NAME 3 (c)
12 LOAD_NAME 4 (d)
15 LOAD_CONST 0 (('x', 'y'))
18 CALL_FUNCTION2 2
1. We save one command for every keyword parameter after the first. This
saves bytecode size and execution time.
2. Since the number of keyword arguments is obtained from tuple's size,
new CALL_FUNCTION opcode needs only the number of positional arguments.
It's argument is simpler and needs less bits (important for wordcode).
1. Increases the number of constants.
The biggest pain with dealing with the peephole optimizer is that it happens after all the complicated flattening and fixup[^1] the compiler does, which means you have to hack up the jump targets as you go along. The future bytecode optimizers that PEP 511 enables will have the same headache.
But this isn't actually necessary. The optimizer could work on a flat array of instructions[^2] instead of an array of bytes, with relocatable jump targets instead of fixed byte offsets, and then the compiler could do the fixup _after_ the optimization.[^3]
It would break the optimizer APIs, but `PyCode_Optimize` isn't public, and the API proposed by PEP 511 is public, but that PEP isn't even finalized, much less accepted yet.
I don't think we need to expose the intermediate representation any farther along than the `PyCode_Optimize` step.[^4] Just moving the optimize one step earlier in the chain solves more than enough to be worth it.
[^1]: If you think numbering the offsets and emitting the jump targets is easy: Every time you fix up a jump, that may require adding an `EXTENDED_ARG`, which means you have to redo any fixups that cross the the current instruction. The compiler does this by just looping until it hasn't added any more `EXTENDED_ARG`s.
[^2]: In case anyone's thinking that wordcode would solve this problem, it doesn't. The `EXTENDED_ARG` jump targets are a much bigger hassle than the 1-or-3-byte-ops, and wordcode not only can't eliminate those, but makes `EXTENDED_ARG` more common.
[^3]: The compiler doesn't actually have exactly what the optimizers would want, but it's pretty close: it has a linked list of block objects, each of which has an array of instruction objects, with jump targets being pointers to blocks. That's actually even better to work with, but too complicated to expose to optimizers. Flattening it would be trivial. Or, if that's too expensive, we could do something almost as simple and much cheaper: convert it in-line to a deque-like linked list of arrays, with jump targets being indices or pointers into that. Or we could just expose the list of blocks as-is, as an opaque thing with a mutable-deque-of-instructions API around it.
[^4]: Later stages--import hooks, optimizing decorators, etc.--have the same pain as the peephole optimizer, but they also have code objects that contain other code objects, and they can be written in Python, and so on, so the same solution isn't feasible there. Of course we could add a function to convert bytecode back to a list of instructions, in some format that can be exposed to Python (in fact, `dis` already does 90% of that), and then another one to convert that back to bytecode and redo the fixup (which is basically the third-party `byteplay` module). But that's almost certainly overkill. (If we wanted that, I'd rather add `byteplay` to the stdlib, port it and `dis` to C, and expose a C API for them. And then we could use that for everything, including the peephole optimizer and PEP 511 optimizers. Which would all be really cool, but I think it's more work than we want to do, and I don't know if we'd actually want something like `byteplay` builtin even if it were easy...)
I propose to add following recommendations in PEP 8 or other documents:
1. It is preferable to pass boolean arguments as keyword arguments (if
this is not the only argument).
2. It is preferable to declare boolean parameters as keyword-only
What are you think about this?