[Tutor] threading for each line in a large file, and doing it right
Evuraan
evuraan at gmail.com
Tue Apr 24 22:26:47 EDT 2018
Greetings!
Please consider this situation :
Each line in "massive_input.txt" need to be churned by the
"time_intensive_stuff" function, so I am trying to background it.
import threading
def time_intensive_stuff(arg):
# some code, some_conditional
return (some_conditional)
with open("massive_input.txt") as fobj:
for i in fobj:
thread_thingy = thread.Threading(target=time_intensive_stuff, args=(i,) )
thread_thingy.start()
With above code, it still does not feel like it is backgrounding at
scale, I am sure there is a better pythonic way.
How do I achieve something like this bash snippet below in python:
time_intensive_stuff_in_bash(){
# some code
:
}
for i in $(< massive_input.file); do
time_intensive_stuff_in_bash i & disown
:
done
Many thanks in advance,
Thanks!
More information about the Tutor
mailing list