On Tue, May 10, 2022 at 1:20 PM Paul Moore  wrote:

> On Tue, 10 May 2022 at 16:31, Barney Stratford
>  wrote:
> >
> > Hello all.
> >
> > It occurs to me that creating threads in Python is more awkward than it
> needs to be. Every time I want to start a new thread, I end up writing the
> same thing over and over again:
> >
> > def target(*args, **kwds):
> >     ...
> > t = threading.Thread(target = target, args = , kwargs=
> )
> > t.start()
> >
> > This becomes especially repetitive when calling a target function that
> only makes sense when run in a new thread, such as a timer.
> >
> > In my own code, I’ve taken to decorating functions that always run in
> new threads. Then I can call the function using the usual function call
> syntax, and have the new thread returned to me. With the decorator, the
> code reads instead:
> >
> > @threaded
> > def target(*args, **kwds):
> >     …
> > t = target(*args, **kwds)
> >
> > This has a couple of advantages. I don’t have to import the threading
> module all over my code. I can use the neater syntax of function calls. The
> function’s definition makes it clear it’s returning a new thread since it’s
> decorated. It gets the plumbing out of the way so I can concentrate more on
> what my code does and less in how it does it.
> >
> > It feels like the right place for this decorator is the standard
> library, so I’ve created PR #91878 for it. @rhettinger suggests that this
> is a bit premature, and that we should discuss it here first. Thoughts?
>
> I like this. I'd probably use it if it existed. Your example isn't
> particularly compelling, though - calling the function "target" makes
> sense when it's the target of the thread, but not so much when it's
> the thread itself. Could you give a motivating example based on real
> code, rather than just the "this is what the syntax would look like"
> as you did above? Ideally showing what it would look like with and
> without the decorator?
>
> Things I *think* i get, but I'd like to see clearly in an example, include:
>
> 1. Does t = target(...) start the thread? I think it does.
> 2. Is it possible to create daemon threads? @threaded(daemon=True)
> seems plausible. I suspect your PR doesn't have this, but it wouldn't
> be difficult to add. But I don't actually know if it's worth adding,
> TBH.
> 3. Can you create multiple threads for the same function? I assume t1,
> t2, t3 = target(arg1), target(arg2), target(arg3) would work.
>
> I'm probably being dense here - I'm pretty sure your proposal actually
> is as simple as it looks (which is a good thing!) and I'm just
> over-complicating things in my head.
>
> Things that come to mind that should be considered here:
1) not every two liner function should be added to the stdlib
and, on the other hand,
2) asyncio features `loop.run_in_executor`
which does essentially the same, with more care for what happens
in "real world" code, but is only usable in async code.

all in all it can be useful for quick and dirty code - and maybe
something more elaborate allowing control on whether to create
daemon threads, or maybe something that will deal with
concurrent.future.executors
as well, and allow access to the function call results

All in all, I find it a nice snippet, but one that is not aligned with
the trend Python has been taking over the last years,
facilitating the writing of "quick and dirty" scripts in contrast
with application code that will take care of all possible corner-cases.

As a good example of what I am talking about, we've seen the contrary
 movement take place, long ago, when
"os.popen" was superseded by "subprocess.*": I think up to today people
had not catched up with the far more complicated API of the latter.

All in all, I'd eventually use this "@threaded" if it was there.



> Paul
>
>    js
  -><-

On Tue, May 10, 2022 at 10:34 AM Barney Stratford < barney_stratford@fastmail.fm> wrote:

...

1. Does t = target(...) start the thread? I think it does. I think it does too. In the commonest use case, immediately after creating a thread, you start it. And if you want to delay the thread but still use the decorator, then you can do that explicitly with some locks. In fact, it’s probably better to explicitly delay execution than have hidden requirements concerning the timing of thread creation and startup.

...

2. Is it possible to create daemon threads? Not at the moment. I did think about this, but felt that simpler is better. Like you say, it’d be easy to add. In fact, I might just go ahead and add it to the PR in a bit. The simplest way to do it is probably to define a second decorator for daemonic threads.

If this is even to be added (i personally lean -1 on it), I suggest intentionally not supporting daemon threads. We should not encourage them to be used, they were a misfeature that in hindsight we should never have created. Daemon threads can lead to very bad surprises upon interpreter finalization - an unfixable problem given how daemon threads are defined to behave.

3. Can you create multiple threads for the same function? I assume t1,

...

t2, t3 = target(arg1), target(arg2), target(arg3) would work. That’s exactly what I had in mind. Make it so that thread creation and function call look exactly alike. You can call a function as many times as you want with whatever args you want, and you can create threads as often as you want with whatever args you want.

There isn’t a single use case where the decorator is particularly compelling; rather, it’s syntactic sugar to hide the mechanism of thread creation so that code reads better.

This is my take as well. I don't like calling code to hide the fact that a thread is being spawned. Use this decorator and if you fail to give the callable a name communicating that it spawns and returns a thread, you will have surprised readers of the calling code. A nicer design pattern is to explicitly manage threads. Use concurrent.futures.ThreadPoolExecutor. Or use the async stuff that Joao mentioned or similar libraries. I think we already provide decent batteries with the threading APIs. -gps

It seems like the consensus is that this is a good idea, but it’s the wrong good idea. Should I cancel the PR or should we try to make it into a better good idea? Cheers, Barney.

On 12May2022 20:17, Barney Stratford wrote:

It seems like the consensus is that this is a good idea, but it’s the wrong good idea. Should I cancel the PR or should we try to make it into a better good idea?

Why not shift slightly? As remarked, having a function automatically spawn threads can be confusing, because spawning a thread has implications for both the thread code itself and for the person calling the function. The caller might find that confusing or complex. Personally, my approach has been a tiny shim function named "bg" in my personal kit, to make it easy to spawn a regular function in a thread: T = bg(func, args....) # returns a running Thread An approach I've also seen is Celery's one of decorating a function with attributes which can dispatch it as a task, roughly: @task def regular_function(.....) ... and it can be dispatched by saying regular_function.defer(......) and variations. Some downsides to decorators include: - only decorated functions can be kicked off "automatically as threads"; that could be a good thing too - the decorator wires in the dispatch mechanism: your decorator spawns a thread, the Celery @task queues to a Celery task queue, and so forth So my personal inclination is to provide an easy to use shim for the caller, not the function. Eg: from cs.threads import bg ..... T = bg(func, args.....) or: from cs.threads import bg as bg_thread from cs.later import bg as bg_later ..... T = bg_thread(func, args......) # run in a Thread ..... R = bg_later(func, args.......) # hand off to a Later, get a Result for collection ...... bg = bg_later # specify the queuing system ...... ... do stuff via bg(), the chosen queuing system ... or whatever other queuing system you might be using. The idea here is to make it easy to submit a function to any of several things rather than decorating the function itself to submit to a now-hardwired thing. Just things to consider.... Cheers, Cameron Simpson

Thinking about it, it's actually a fine design to have a decorator that turns a regular function into one that starts a thread -- from the caller's POV it's no different than having a function that explicitly starts a thread, and it could be a nice shorthand if you do this all the time. (You have to document it in either case.) That said, I don't think we need this 3-line decorator in the stdlib. --Guido On Thu, May 12, 2022 at 10:40 PM Serhiy Storchaka wrote:

10.05.22 18:12, Barney Stratford пише:

...

This has a couple of advantages. I don’t have to import the threading module all over my code. I can use the neater syntax of function calls. The function’s definition makes it clear it’s returning a new thread since it’s decorated. It gets the plumbing out of the way so I an concentrate more on what my code does and less in how it does it.

I do not see advantages. You definitely need to import the threading module or other module depending on the threading module in which you define the decorator. And you need to decorate the function. It will not save you a line of code.

If you need to run a lot of functions in threads, note that a thread has some starting cost. Consider using concurrent.futures.ThreadPoolExecutor.

If you only run few long living threads, the syntax of starting them does not matter, in comparison with the rest of your code.

Also, it looks wrong to me to mix two different things: what code to execute and how to run it. If we need a neater syntax, I would propose:

t = threading.start_thread(func, *args, **kwargs)

But I am not sure that it is worth to add such three-line function in the stdlib.

_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RBIAWP2J... Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...