Mailman 3 June 2020 - Python-ideas

Shared Semaphores for synchronisation across unrelated processes
by Vinay Sharma June 9, 2020

June 9, 2020

Hi, Python has integrated shared memory into standard library starting from 3.8 (https://docs.python.org/3/library/multiprocessing.shared_memory.html <https://docs.python.org/3/library/multiprocessing.shared_memory.html>), which provides a user friendly API to access shared memory across unrelated processes using names. But, there are no synchronisation mechanisms present in the standard library to prevent race conditions when shared memory is accessed across unrelated processes. I had … [View More]

4 11

Re: Add close method to queue
by Matthew Einhorn June 9, 2020

June 9, 2020

Accidentally responded off list. On Tue, Jun 9, 2020 at 1:06 PM Андрей <heckad(a)yandex.ru> wrote: > Thank you for your comment. > > > > This feature is very needed for > https://github.com/mosquito/aio-pika/pull/327. > > > > PS How to send response on this message? I don’t see this message on *mail.python.org > <http://mail.python.org>. * > You need to register on python's bug tracker if you want to be able to respond to the bug report there. … [View More] > And where do you find RLock? > It's in the threading package <https://docs.python.org/3.9/library/threading.html#threading.RLock>. You can use a normal Lock, it's just that I needed to be able to call this from __del__ <https://docs.python.org/3/reference/datamodel.html#object.__del__> so I needed a lock that can be re-entered. > > > Отправлено из приложения "Почта > <https://go.microsoft.com/fwlink/?LinkId=550986>" для Windows 10 > > > > *От: *Matthew Einhorn <moiein2000(a)gmail.com> > *Отправлено: *9 июня 2020 г. в 19:41 > *Кому: *Kazantcev Andrey <heckad(a)yandex.ru> > *Тема: *Re: [Python-ideas] Add close method to queue > > > > I was just looking for something similar, but this was already requested > and rejected for lack of use cases here: > https://bugs.python.org/issue29701. > > > > For my own use case, a RLock with a class variable indicating the queue > was closed did the trick. > > > > On Tue, Jun 9, 2020 at 8:25 AM Kazantcev Andrey <heckad(a)yandex.ru> wrote: > > I have a problem with notifying all current subscribers and new > subscribers about the closure of the queue and the reason. For example, I > have a producer that reads messages from websocket or something else and > send this to a queue, and several consumers (I do not know how many). If > any exception occurred, then all current subscribers and subscribers which > will be added later should know about this error. I tried to send an > exception to a queue, but that did not help, because I have several > consumers. Also, this will not protect new consumers. I propose to add a > new close method with exc argument, which will throw an exception when > calling the get method, and also throw an exception for all current > _getters. > _______________________________________________ > Python-ideas mailing list -- python-ideas(a)python.org > To unsubscribe send an email to python-ideas-leave(a)python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/EYVGO… > Code of Conduct: http://python.org/psf/codeofconduct/ > > > [View Less]

1 0

Add close method to queue
by Kazantcev Andrey June 9, 2020

June 9, 2020

I have a problem with notifying all current subscribers and new subscribers about the closure of the queue and the reason. For example, I have a producer that reads messages from websocket or something else and send this to a queue, and several consumers (I do not know how many). If any exception occurred, then all current subscribers and subscribers which will be added later should know about this error. I tried to send an exception to a queue, but that did not help, because I have several … [View More]

2 1

Augment abc.Set API (support named set methods for dictionary view objects)
by ava＠yert.pink ava＠yert.pink June 7, 2020

June 7, 2020

Currently the dictionary view objects behave exactly like sets when set operations defined on them are used, such as the pipe or ampersand operators: >>> d.keys() | [1] {1, 2} dictionary views are even sets by the ABC standard, having support for all the required abstract methods: >>> isinstance(d.keys(), collections.abc.Set) True However, the dict views do not have the equivalent named set methods, such as `intersection` or `issuperset`. It seems that the collections.abc.… [View More]

7 12

PYTHONLOGGING env variable
by Bar Harel June 6, 2020

June 6, 2020

Another idea I've had that may be of use: PYTHONLOGGING environment variable. Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level. Another option would be that -x dev or a different -x logging will initialize basic config. Will be useful mostly for debugging purposes instead of temporarily modifying the code. Kinda surprised it doesn't exist tbh. Bar Harel

9 22

Advanced Debugger Support C-API is incomplete?
by Andrew Pashkin June 6, 2020

June 6, 2020

I want to implement my own version of _PyThread_CurrentFrames() <https://github.com/python/cpython/blob/e838a9324c1719bb917ca81ede8d766b5cb5…>, but it seems like it's not possible using public C-API. The comment <https://github.com/python/cpython/blob/e838a9324c1719bb917ca81ede8d766b5cb5…> in the source code of _PyThread_CurrentFrames() says that lists of interpreters and threads can be mutated even when GIL is held and so HEAD_LOCK() should be used. At the same time Advanced … [View More]

1 1

Automatic notification when your PR test fails or succeeds
by Ram Rachum June 5, 2020

June 5, 2020

Hi guys, I'm not sure whether this is the right mailing list for this, because it's more for the Python development process on GitHub than Python itself. If this belongs somewhere else, let me know. When you open a PR to CPython, it would be cool if there was an option to get email notifications on whether the tests fail or succeeded. For example, I may open a PR as a draft, and only after I get notification on success, mark it as "Ready for review". Right now I just poll it manually, which … [View More]

3 2

asyncio: recursive event loops
by Celelibi June 4, 2020

June 4, 2020

Hello world, I've been recently trying to switch from a library using gevent to a library using asyncio. And do so without contaminating the whole code with the "async" keyword in one go. As expected, calling the synchronous code from a coroutine is pretty straightforward. But the other way around is a bit more tricky. If the function is not running in the thread that runs the event loop, I can use something like this: asyncio.run_coroutine_threadsafe(...).result() Which seems to work, … [View More]despite the documentation saying that "If the future's result isn't yet available, raises InvalidStateError.". But if the function that want to call a coroutine is in the thread that runs the event loop, then there's not much I can do to run a coroutine in the event loop that indirectly called that function. As a somewhat more concrete example, let's consider a class that sends and receive messages: class Client: async def on_msg(self, msg): process(msg) async def asend(self, msg): print("async sending", msg) return len(msg) def send(self, msg): print("sending", msg) # return await self.asend(msg) async def run(self): await self.on_msg("PING") def process(msg): # Some legacy code that indirectly ends up wanting to call: print("Received", msg) print(client.send("PONG")) client = Client() asyncio.run(client.run()) Unless I missed something, it's pretty difficult to have `Client.send` "await" on `Client.asend` without contaminating the whole code with "async" and "await". I only see two solutions right now. Either have `Client.on_msg` call `process` in a new thread and then use the trick above. But running `process` in a separate thread might have some consequences since it doesn't expect it. Or have `Client.send` start a new event loop in a new thread. Which might behave suprisingly when sharing file descriptors with the existing loop. What I'm proposing is to change a bit the behavior of `loop.run_until_complete` as follow: - If the loop isn't running and there's no loop in the current context, keep the current behavior. - If the loop is running, but not in the current context, use the `call_soon_threadsafe` trick above. - If the loop is running in the current context, then loop until the coroutine is done. Additionally, `asyncio.run` could be changed similarly. There's of course a strong caveat that it's easy to create an infinite recursion with this. But I think it's worth it as it would allow a much easier integration of asyncio into existing code. Which (I think) limit its adoption. What do you think about it? Best regards [View Less]

1 0

Optional keyword arguments
by James Lu June 2, 2020

June 2, 2020

Many a python programmer have tired to see code written like: def bar(a1, a2, options=None): if options is None: options = {} ... # rest of function syntax if argument is not passed, evaluate {} and store to options def foo(options:={}): pass syntax if argument is not passed or is None, evaluate {} and store to options* def foo(options?={}): pass The Zen of Python states "there shouldn't be two ways to do the same thing." Thus, only one of ":=" or "?=" should be adopted. They … [View More]

23 151

How to propose a change with tests where the failing test case (current behaviour) is bad or dangerous
by Steve Barnes June 2, 2020

June 2, 2020

There is a long standing issue, (or at least I consider it a poor feature some others consider it compliance with other existing tools), with a couple of python standard libraries that I would like to raise an issue for & PR(s) to address but when it comes to putting tests in place the failing test case can potentially crash the host machine, sometimes in an unrecoverable manner, due to the potential to generate an infinite size file. Is there a way of proposing a change with tests that … [View More]

9 21