[Tutor] threading for each line in a large file, and doing it right

Evuraan evuraan at gmail.com
Tue Apr 24 22:26:47 EDT 2018


Greetings!

Please consider this situation :
Each line in "massive_input.txt" need to be churned by the
"time_intensive_stuff" function, so I am trying to background it.

import threading

def time_intensive_stuff(arg):
   # some code, some_conditional
   return (some_conditional)

with open("massive_input.txt") as fobj:
   for i in fobj:
      thread_thingy = thread.Threading(target=time_intensive_stuff, args=(i,) )
      thread_thingy.start()


With above code, it still does not feel like it is backgrounding at
scale,  I am sure there is a better pythonic way.

How do I achieve something like this bash snippet below in python:

time_intensive_stuff_in_bash(){
   # some code
  :
}

for i in $(< massive_input.file); do
    time_intensive_stuff_in_bash i & disown
    :
done

 Many thanks in advance,

Thanks!


More information about the Tutor mailing list