Mailman 3 Get install prefix for module at runtime - Distutils-SIG

15 Sep 2009

      Hi all,

i am pretty new to distutils (since yesterday)  and am struggling with
accessing installed data_files. I want to ship a library that benefits
from some data_files from another package.

My wishes:

    * distribution should go in 'PREFIX/lib/foo' (site-packages?)
    * datafiles should go in 'PREFIX/share/foo'
    * Access to data files from modules

My question is: "How to reliably access data files from a module in
foo?"

A thorough explanation follows. If you can answer the aforementioned
question just think tl;dr and skip it ;-)

Installation Schemes
====================

There seem to be three different installation schemes regarding setting
of this PREFIX. The following examples assume installation of package
'foo' which contains module 'bar' and data file 'baz.data'.

Default install (setup.py install)
-----------------------------------------

* Packages installation path:
    'sys.prefix/lib/pythonX.Y/site-packages/'

* Data files installation path:
    'sys.prefix/share/'

That means that in the aforementioned example the following file system
structure will be created upon installation:

Package:    /usr/lib/python2.5/site-packages/foo/bar.py
Data:       /usr/share/foo/baz.data

Home scheme (setup.py install --home=HOME)
---------------------------------------

* Packages installation path:
    'HOME/lib/python'

* Data files installation path:
    'HOME/share/'

Resulting structure:

Package:    HOME/lib/foo/bar.py
Data        HOME/share/foo/baz.data

Prefix scheme (setup.py --prefix=PREFIX)

* Package installation path:
    'PREFIX/lib/pythonX.Y/site-packages'

* Data installation path:
    'PREFIX/share/'

Resulting structure (for PREFIX==/usr/local)

Package:    /usr/local/lib/python2.5/site-packages/foo/bar.py
Data:       /usr/share/foo/baz.data

Finding Data files
==================

My problem is finding a reliable way to access the data files from
within bar.py.

Approach 1 - using module.__file__
----------------------------------

bar_path_elements = bar.__file__.split(os.path.sep)
prefix = os.path.join(
            os.path.sep,
            *itertools.takewhile(lambda el: el != 'lib'))

data_file_path = os.path.join(prefix, 'share', 'foo')

* Will this work reliably? 

* What if the user specifies a different data directory with
  '--install-data' ?

Approach 2 - writing a build.py file at installation time
---------------------------------------------------------

The last question made me think, that Approach 1 works for the three
standard installation schemes, but fails miserably if the user decides
to do anything fancy. As i do not comprehend all the options you can give
to 'setup.py install' or know about them i am not sure if there are not
many more problems hidden in there.

The solution seems to be to create a build.py file that includes this
information at installation time and access the data there. My problem
is just: HOW?

I tried with the following setup.py script, but am unsure how to access
the correct options:

--- snip ---
# -*- coding: UTF-8 -*-
#!/usr/bin/env python

from __future__ import with_statement

from distutils.core import setup
import sys
import getopt

def create_build_file():
    try:
        opts, args = getopt.getopt(sys.argv[1:], "", [])
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            What to do here????

    except getopt.GetoptError, err:
        pass

    with open('lib/mwdb/build.py', 'w') as build_fp:
        #build_fp.write('# -*- coding: UTF-8 -*-\n')
        #build_fp.write("LIB_PREFIX = '%s'\n"%(options.lib_prefix))
        #build_fp.write("DATA_PREFIX = '%s'\n"%(options.data_prefix))
        #build_fp.write("EXEC_PREFIX = '%s'\n"%(options.exec_prefix))

try:
    if 'install' == sys.argv[1]:
        create_build_file()
except IndexError:
    pass

setup(name='foo',
      version='0.1',
      packages=['foo']
      package_dir = { '':'lib' },
      ...
     )
--- snip ---

The problem i am facing now is: How to parse the options? Do I really have
to recreate the complete distutils option parsing logic - and probably
fail? All that without optparse! How can i get the aforementioned
options? Is the file creation way the correct way to handle this?

I really don't know what to do now and am also wondering why nobody had
that problem before! Are all other libraries shipped with external data
buggy for some installation schemes? How is this normally solved?

with kind regards and thanks for reading all this

    Wolodja Wentland

Get install prefix for module at runtime

Wolodja Wentland

P.J. Eby

Tarek Ziadé

Wolodja Wentland

Wolodja Wentland

Wolodja Wentland

Tarek Ziadé

Tarek Ziadé

Wolodja Wentland

Wolodja Wentland

tags

participants (3)