Using multiprocessing from a Windows service

Volodya orlenko at gmail.com
Thu Feb 5 19:37:18 EST 2009


Hi all,

I think I've found a small bug with multiprocessing package on
Windows. If you try to start a multiprocessing.Process from a Python-
based Windows service, the child process will fail to run. When
running the parent process as a regular Python program, everything
works as expected.

I've tracked the problem down to how main_path is prepared in
multiprocessing.forking.get_preparation_data() (lines 370-377):

def get_preparation_data(name):
    [...skipped a few lines...]
    if not WINEXE:
        main_path = getattr(sys.modules['__main__'], '__file__', None)
        if not main_path and sys.argv[0] not in ('', '-c'):
            main_path = sys.argv[0]
        if main_path is not None:
            if not os.path.isabs(main_path) and \
                                      process.ORIGINAL_DIR is not
None:
                main_path = os.path.join(process.ORIGINAL_DIR,
main_path)
            d['main_path'] = os.path.normpath(main_path)
    return d

When the program is running as a Windows service, but is not packaged
into a single executable, main_path will become the path to the
service executable (typically, pythonservice.exe). When this data
makes it to the child process, the prepare() function will treat
main_path as a path to a python module, and will try to import it.
This causes it to fail.

My quick-and-dirty solution was to check in get_preparation_data() if
main_path ends with '.exe', and if it does, to not pass it at all.
This solves the problem in my case, but perhaps there's a better way
to fix this? Here is my version of get_preparation_data():

def get_preparation_data(name):
    '''
    Return info about parent needed by child to unpickle process
object
    '''
    from .util import _logger, _log_to_stderr

    d = dict(
        name=name,
        sys_path=sys.path,
        sys_argv=sys.argv,
        log_to_stderr=_log_to_stderr,
        orig_dir=process.ORIGINAL_DIR,
        authkey=process.current_process().authkey,
        )

    if _logger is not None:
        d['log_level'] = _logger.getEffectiveLevel()

    if not WINEXE:
        main_path = getattr(sys.modules['__main__'], '__file__', None)
        if not main_path and sys.argv[0] not in ('', '-c'):
            main_path = sys.argv[0]
        if main_path is not None:
            if not os.path.isabs(main_path) and \
                                      process.ORIGINAL_DIR is not
None:
                main_path = os.path.join(process.ORIGINAL_DIR,
main_path)
            if not main_path.endswith('.exe'):
                d['main_path'] = os.path.normpath(main_path)

    return d





More information about the Python-list mailing list