Parallel Processing

[Yigit Turgut]

On Jan 8, 4:34 pm, Dave Angel <d... at davea.name> wrote:
> On 01/08/2012 08:23 AM, Yigit Turgut wrote:
>
>
>
>
>
>
>
> > Hi all,
>
> > I am trying to run two functions at the same time with Parallel
> > Processing (pp) as following ;
>
> > import pygame
> > import sys
> > import time
> > import math
> > import pp
>
> > screen = pygame.display.set_mode((0, 0), pygame.FULLSCREEN)
> > timer = pygame.time.Clock()
> > white = True
> > start = time.time()
> > end = time.time() - start
>
> > def test1():
> >    global end
> >    while(end<5):
> >      end = time.time() - start
> >      timer.tick(4) #FPS
> >      screen.fill((255,255,255) if white else (0, 0, 0))
> >      white = not white
> >      pygame.display.update()
>
> > def test2():
> >    global end
> >    while(end2<5):
> >      end2 = time.time() - start
> >      print end
>
> > ppservers = ()
>
> > if len(sys.argv)>  1:
> >      ncpus = int(sys.argv[1])
> >      # Creates jobserver with ncpus workers
> >      job_server = pp.Server(ncpus, ppservers=ppservers)
> > else:
> >      # Creates jobserver with automatically detected number of workers
> >      job_server = pp.Server(ppservers=ppservers)
> > print "Starting PP with", job_server.get_ncpus(), "workers"
>
> > job1 = job_server.submit(test1,test2)
> > result = job1()
>
> > And the output is ;
>
> > Starting PP with 2 workers
> > Traceback (most recent call last):
> >    File "fl.py", line 46, in<module>
> >      job1 = job_server.submit(test1,test2)
> >    File "/usr/lib/python2.6/site-packages/pp.py", line 402, in submit
> >      raise TypeError("args argument must be a tuple")
> > TypeError: args argument must be a tuple
>
> > When I change job1 to just to see if it will run one function only;
> > job1 = job_server.submit(test1)
>
> > I get an output of;
>
> > NameError: global name 'end' is not defined
>
> > end variable is defined as global but I get a NameError. Anyone has an
> > idea what's going on ?
>
> First, please tell us about any external dependencies (eg. imports).
> Pygame may not matter, since there are many people here using it, but
> each of us has to hunt down something that does parallel processing with
> an interface similar to what you're using.  For the next person, the
> most likely candidate out of the dozens available is:http://www.parallelpython.com/
>
> Your problem, as stated in the raise statement, is that the submit
> method is expecting a tuple for its "args argument".  Now I had to go to
> the website to find an example, but it appears that your second argument
> should have been a tuple of the arguments for your function.  See
> *      submit*(self, func, args=(), depfuncs=(), modules=(),
> callback=None, callbackargs=(), group='default', globals=None)
>
> on page:http://www.parallelpython.com/content/view/15/30/#API
>
> So you're passing it two function objects, and the second one is in the
> place where it's expecting a tuple of arguments.  Apparently you should
> be calling submit multiple times, once for each function.
>
> As for the error you get when you do that, please post the full
> traceback. I suspect that the message you did post has a typo in it, or
> that the error occurred when your source code was not as you show it
> here.  You have two variables end and end2, and only the first one is
> ever declared global.  func2() should get a different error when you run
> it; since it tries to use a local end2 before defining it.
>
> I recommend always testing your code with a single thread before trying
> to launch more complex multithread stuff.  Just call func() and func2(),
> and see if they complete, and get reasonable answers.
>
> --
>
> DaveA

There are no imports other than defined on the script, which are;

import pygame
import sys
import time
import math
import pp

You are correct about  trying to pass two functions and second one is
in place where a tuple of arguments supposed to be. But what if these
functions don't have any arguments ? I tested functions test1() and
test2() seperately ; they work. Once I figure out how to run these
functions simultaneously, I will add an argument to test2 and try then
on. My main goal is to simultaneously run two functions, one of them
has one argument the other doesn't. To get familiar with parallel
processing I am experimenting now without arguments and then I will
embed the code to my application. I am experimenting with the
following ;

import pygame
import sys
import time
import math
import pp

screen = pygame.display.set_mode((0, 0), pygame.FULLSCREEN)
timer = pygame.time.Clock()
white = True
start = time.time()
end = time.time() - start

def test1():
  global end
  global white
  while(end<5):
    end = time.time() - start
    timer.tick(4) #FPS
    screen.fill((255,255,255) if white else (0, 0, 0))
    white = not white
    pygame.display.update()

def test2():
  global end
  while(end<5):
    end = time.time() - start
    print end

ppservers = ()
job_server = pp.Server(ppservers=ppservers)
print "Starting pp with", job_server.get_ncpus(), "workers"

job1 = job_server.submit(test1())
job2 = job_server.submit(test2())
result = job1()
result2 = job2()

print "Counting...", result2

job_server.print_stats()

test1() works as expected (job1) but test2() doesn't work and I get
the following traceback ;

Traceback (most recent call last):
  File "fl.py", line 33, in <module>
    job1 = job_server.submit(test1())
  File "/usr/lib/python2.6/site-packages/pp.py", line 458, in submit
    sfunc = self.__dumpsfunc((func, ) + depfuncs, modules)
  File "/usr/lib/python2.6/site-packages/pp.py", line 629, in
__dumpsfunc
    sources = [self.__get_source(func) for func in funcs]
  File "/usr/lib/python2.6/site-packages/pp.py", line 696, in
__get_source
    sourcelines = inspect.getsourcelines(func)[0]
  File "/usr/lib/python2.6/inspect.py", line 678, in getsourcelines
    lines, lnum = findsource(object)
  File "/usr/lib/python2.6/inspect.py", line 519, in findsource
    file = getsourcefile(object) or getfile(object)
  File "/usr/lib/python2.6/inspect.py", line 441, in getsourcefile
    filename = getfile(object)
  File "/usr/lib/python2.6/inspect.py", line 418, in getfile
    raise TypeError('arg is not a module, class, method, '
TypeError: arg is not a module, class, method, function, traceback,
frame, or code object

Error is related to test1 not having an argument.  When I leave it
empty as following ;

job1 = job_server.submit(test1,())

test1 doesn't run. When I do ;

job1 = job_server.submit(test1())

Display works but I get;

TypeError: arg is not a module, class, method, function, traceback,
frame, or code object (complete traceback same as above).

And test2 doesn't work also. But when I do;

job1 = job_server.submit(test1,())
job2 = job_server.submit(test2())

I get test2 working but test1 not working. Obviously related to
argument arrangement in submit.