#!python """\ Easy Install ============ Easy Install is a python module (easy_install) that lets you automatically download, build, install, and manage Python packages. .. contents:: **Table of Contents** Downloading and Installing a Package ------------------------------------ For basic use of ``easy_install``, you need only supply the filename or URL of a source distribution or .egg file (`Python Egg`__). __ http://peak.telecommunity.com/DevCenter/PythonEggs **Example 1**. Download a source distribution, automatically building and installing it:: easy_install http://example.com/path/to/MyPackage-1.2.3.tgz **Example 2**. Install an already-downloaded .egg file:: easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg Easy Install recognizes distutils *source* (not binary) distribution files with extensions of .tgz, .tar, .tar.gz, .tar.bz2, or .zip. And of course it handles already-built .egg distributions. By default, packages are installed to the running Python installation's ``site-packages`` directory, unless you provide the ``-d`` or ``--install-dir`` option to specify an alternative directory. Packages installed to ``site-packages`` are added to an ``easy-install.pth`` file, so that Python will be able to import the package by default. If you do not want this to happen, you should use the ``-m`` or ``--multi`` option, which allows multiple versions of the same package to be selected at runtime. Note that installing to a directory other than ``site-packages`` already implies the ``-m`` option, so if you cannot install to ``site-packages``, please see the `Options`_ section below (under ``--multi``) to find out how to select packages at runtime. Upgrading a Package ------------------- You don't need to do anything special to upgrade a package: just install the new version. If you're using ``-m`` or ``--multi`` (or installing outside of ``site-packages``), the runtime system automatically selects the newest available version of a package. If you're installing to ``site-packages`` and not using ``-m``, installing a package automatically replaces its older version in the ``easy-install.pth`` file, so that Python will import the latest version by default. ``easy_install`` never actually deletes packages (unless you're installing a package with the same name and version number as an existing package), so if you want to get rid of older versions of a package, please see `Uninstalling Packages`_, below. Changing the Active Version (``site-packages`` installs only) ------------------------------------------------------------- If you've upgraded a package, but need to revert to a previously-installed version, you can do so like this:: easy_install PackageName==1.2.3 Where ``1.2.3`` is replaced by the exact version number you wish to switch to. Note that the named package and version must already have been installed to ``site-packages``. If you'd like to switch to the latest version of ``PackageName``, you can do so like this:: easy_install PackageName This will activate the latest installed version. Uninstalling Packages --------------------- If you have replaced a package with another version, then you can just delete the package(s) you don't need by deleting the PackageName-versioninfo.egg file or directory (found in the installation directory). If you want to delete the currently installed version of a package (or all versions of a package), you should first run:: easy_install -m PackageName This will ensure that Python doesn't continue to search for a package you're planning to remove. After you've done this, you can safely delete the .egg files or directories. Options ------- ``--zip, -z`` Enable installing the package as a zip file. This can significantly increase Python's overall import performance if you're installing to ``site-packages`` and not using the ``--multi`` option, because Python process zipfile entries on ``sys.path`` much faster than it does directories. So, if you don't use this option, and you install a lot of packages, some of them may be slower to import. But, this option is disabled by default, unless you're installing from an already-built binary zipfile (``.egg`` file). This is to avoid problems when using packages that dosn't support running from a zip file. Such packages usually access data files in their package directories using the Python ``__file__`` or ``__path__`` attribute, instead of the ``pkg_resources`` API. So, if you find that a package doesn't work properly when used with this option, you may want to suggest to the author that they switch to using the ``pkg_resources`` resource API, which will allow their package to work whether it's installed as a zipfile or not. (Note: this option only affects the installation of newly-built packages that are not already installed in the target directory; if you want to convert an existing installed version from zipped to unzipped or vice versa, you'll need to delete the existing version first.) ``--multi-version, -m`` "Multi-version" mode. Specifying this option prevents ``easy_install`` from adding an ``easy-install.pth`` entry for the package being installed, and if an entry for any version the package already exists, it will be removed upon successful installation. In multi-version mode, no specific version of the package is available for importing, unless you use ``pkg_resources.require()`` to put it on ``sys.path``. This can be as simple as:: from pkg_resources import require require("SomePackage", "OtherPackage", "MyPackage") which will put the latest installed version of the specified packages on ``sys.path`` for you. (For more advanced uses, like selecting specific versions and enabling optional dependencies, see the ``pkg_resources`` API doc.) Note that if you install to a directory other than ``site-packages``, this option is automatically in effect, because ``.pth`` files can only be used in ``site-packages`` (at least in Python 2.3 and 2.4). So, if you use the ``--install-dir`` or ``-i`` options, you must also use ``require()`` to enable packages at runtime ``--install-dir=DIR, -d DIR`` Set the installation directory. It is up to you to ensure that this directory is on ``sys.path`` at runtime, and to use ``pkg_resources.require()`` to enable the installed package(s) that you need. """ import sys, os.path, pkg_resources, re, zipimport, zipfile, tarfile, shutil from distutils.sysconfig import get_python_lib from shutil import rmtree # must have, because it can be called from __del__ from pkg_resources import * class Installer: """Manage a download/build/install process""" pth_file = None def __init__(self, instdir=None, zip_ok=False, multi=None, tmpdir=None): if tmpdir is None: from tempfile import mkdtemp tmpdir = mkdtemp(prefix="easy_install-") self.tmpdir = tmpdir site_packages = get_python_lib() if instdir is None or self.samefile(site_packages,instdir): instdir = site_packages self.pth_file = PthDistributions( os.path.join(instdir,'easy-install.pth') ) elif multi is None: multi = True elif not multi: # explicit false, raise an error raise RuntimeError( "Can't do single-version installs outside site-packages" ) self.instdir = instdir self.zip_ok = zip_ok self.multi = multi def close(self): if os.path.isdir(self.tmpdir): rmtree(self.tmpdir,True) def __del__(self): self.close() def samefile(self,p1,p2): if hasattr(os.path,'samefile') and ( os.path.exists(p1) and os.path.exists(p2) ): return os.path.samefile(p1,p2) return ( os.path.normpath(os.path.normcase(p1)) == os.path.normpath(os.path.normcase(p2)) ) def download(self, spec): """Locate and/or download or `spec`, returning a local filename""" if isinstance(spec,Requirement): pass else: scheme = URL_SCHEME(spec) if scheme: # It's a url, download it to self.tmpdir return self._download_url(scheme.group(1), spec) elif os.path.exists(spec): # Existing file or directory, just return it return spec else: try: spec = Requirement.parse(spec) except ValueError: raise RuntimeError( "Not a URL, existing file, or requirement spec: %r" % (spec,) ) # process a Requirement dist = AvailableDistributions().best_match(spec,[]) if dist is not None and dist.path.endswith('.egg'): return dist.path return self.download(self._find_package(spec)) def install_eggs(self, dist_filename): # .egg dirs or files are already built, so just return them if dist_filename.lower().endswith('.egg'): return self.install_egg(dist_filename,True) # Anything else, try to extract and build if os.path.isfile(dist_filename): self._extract_file(dist_filename) # Find the setup.py file from glob import glob setup_script = os.path.join(self.tmpdir, 'setup.py') if not os.path.exists(setup_script): setups = glob(os.path.join(self.tmpdir, '*', 'setup.py')) if not setups: raise RuntimeError( "Couldn't find a setup script in %s" % dist_filename ) if len(setups)>1: raise RuntimeError( "Multiple setup scripts in %s" % dist_filename ) setup_script = setups[0] self._run_setup(setup_script) for egg in glob( os.path.join(os.path.dirname(setup_script),'dist','*.egg') ): self.install_egg(egg, self.zip_ok) def _extract_zip(self,zipname,extract_dir): import zipfile z = zipfile.ZipFile(zipname) try: for info in z.infolist(): name = info.filename # don't extract absolute paths or ones with .. in them if name.startswith('/') or '..' in name: continue target = os.path.join(extract_dir,name) if name.endswith('/'): # directory ensure_directory(target) else: # file ensure_directory(target) data = z.read(info.filename) f = open(target,'wb') try: f.write(data) finally: f.close() del data finally: z.close() def _extract_tar(self,tarobj): try: tarobj.chown = lambda *args: None # don't do any chowning! for member in tarobj: if member.isfile() or member.isdir(): name = member.name # don't extract absolute paths or ones with .. in them if not name.startswith('/') and '..' not in name: tarobj.extract(member,self.tmpdir) finally: tarobj.close() def _run_setup(self, setup_script): from setuptools.command import bdist_egg sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) old_dir = os.getcwd() save_argv = sys.argv[:] save_path = sys.path[:] try: os.chdir(os.path.dirname(setup_script)) try: sys.argv[:] = [setup_script, '-q', 'bdist_egg'] sys.path.insert(0,os.getcwd()) execfile(setup_script, {'__file__':setup_script, '__name__':'__main__'} ) except SystemExit, v: if v.args and v.args[0]: raise RuntimeError( "Setup script exited with %s" % (v.args[0],) ) finally: os.chdir(old_dir) sys.path[:] = save_path sys.argv[:] = save_argv def _find_package(self, req): # TODO: search here for a distro to download, matching Requirement # 'req' and return the package URL or filename raise DistributionNotFound(spec) def install_egg(self, egg_path, zip_ok): import shutil destination = os.path.join(self.instdir, os.path.basename(egg_path)) ensure_directory(destination) if not self.samefile(egg_path, destination): if os.path.isdir(destination): shutil.rmtree(destination) elif os.path.isfile(destination): os.unlink(destination) if zip_ok: if egg_path.startswith(self.tmpdir): shutil.move(egg_path, destination) else: shutil.copy2(egg_path, destination) elif os.path.isdir(egg_path): shutil.move(egg_path, destination) else: os.mkdir(destination) self._extract_zip(egg_path, destination) if self.pth_file is not None: if os.path.isdir(destination): dist = Distribution.from_filename( destination, metadata=PathMetadata( destination, os.path.join(destination,'EGG-INFO') ) ) else: metadata = EggMetadata(zipimport.zipimporter(destination)) dist = Distribution.from_filename(destination,metadata=metadata) # remove old map(self.pth_file.remove, self.pth_file.get(dist.key,())) if not self.multi: self.pth_file.add(dist) # add new self.pth_file.save() def _download_url(self, scheme, url): # Determine download filename from urlparse import urlparse name = filter(None,urlparse(url)[2].split('/'))[-1] while '..' in name: name = name.replace('..','.').replace('\\','_') filename = os.path.join(self.tmpdir,name) if scheme=='svn' or scheme.startswith('svn+'): return self._download_svn(url, filename) # Download the file from urllib import FancyURLopener, URLopener class _opener(FancyURLopener): http_error_default = URLopener.http_error_default try: filename,headers = _opener().retrieve( url,filename ) except IOError,v: if v.args and v.args[0]=='http error': raise RuntimeError( "Download error: %s %s" % v.args[1:3] ) else: raise if headers['content-type'].lower().startswith('text/html'): return self._download_html(url, headers, filename) # and return its filename return filename def _extract_file(self, dist_filename): if zipfile.is_zipfile(dist_filename): self._extract_zip(dist_filename, self.tmpdir) else: try: tar = tarfile.open(dist_filename) except tarfile.TarError: raise RuntimeError( "Not a valid tar or zip archive: %s" % dist_filename ) else: self._extract_tar(tar) def _download_html(self, url, headers, filename): sf_url = url.startswith('http://prdownloads.') file = open(filename) for line in file: if line.strip(): # Check if it is a subversion index page: if re.search(r'Revision \d+:', line): file.close() os.unlink(filename) return self._download_svn(url, filename) elif sf_url and re.search(r'^<HTML><HEAD>', line, re.I): continue elif sf_url and re.search(r'\s*<TITLE>Select a Mirror for File: ', line): # Sourceforge mirror page: return self._download_sourceforge(url, file.read()) else: break # not an index page file.close() raise RuntimeError("Unexpected HTML page found at "+url) def _download_svn(self, url, filename): os.system("svn checkout -q %s %s" % (url, filename)) return filename def _download_sourceforge(self, source_url, sf_page): """ Return a (randomly-selected) (scheme URL) for downloading the package, given the SourceForge mirror-selection page. """ import random import urlparse import urllib mirror_regex = re.compile(r'HREF=(/.*?\?use_mirror=[^>]*)') urls = [m.group(1) for m in mirror_regex.finditer(sf_page)] if not urls: raise RuntimeError( "URL appears to be a Sourceforge mirror page, but no URLs found") url = urlparse.urljoin(source_url, random.choice(urls)) f = urllib.urlopen(url) mirror_page = f.read() f.close() match = re.search(r'<META HTTP-EQUIV="refresh" content=".*?URL=(.*?)"', mirror_page) if not match: raise RuntimeError( 'No META HTTP-EQUIV="refresh" found in Sourceforge page at %s' % url) download_url = match.group(1) scheme = URL_SCHEME(download_url) print download_url return self._download_url(scheme.group(1), download_url) class PthDistributions(AvailableDistributions): """A .pth file with Distribution paths in it""" dirty = False def __init__(self, filename): self.filename = filename; self._load() AvailableDistributions.__init__( self, list(yield_lines(self.paths)), None, None ) def _load(self): self.paths = [] if os.path.isfile(self.filename): self.paths = [line.rstrip() for line in open(self.filename,'rt')] while self.paths and not self.paths[-1].strip(): self.paths.pop() def save(self): """Write changed .pth file back to disk""" if self.dirty: data = '\n'.join(self.paths+['']) f = open(self.filename,'wt') f.write(data) f.close() self.dirty = False def add(self,dist): """Add `dist` to the distribution map""" if dist.path not in self.paths: self.paths.append(dist.path); self.dirty = True AvailableDistributions.add(self,dist) def remove(self,dist): """Remove `dist` from the distribution map""" while dist.path in self.paths: self.paths.remove(dist.path); self.dirty = True AvailableDistributions.remove(self,dist) URL_SCHEME = re.compile('([-+.a-z0-9]{2,}):',re.I).match def main(argv, factory=Installer): from optparse import OptionParser parser = OptionParser(usage = "usage: %prog [options] url [url...]") parser.add_option("-d", "--install-dir", dest="instdir", default=None, help="install package to DIR", metavar="DIR") parser.add_option("-z", "--zip", action="store_true", dest="zip_ok", default=False, help="install package as a zipfile") parser.add_option("-m", "--multi-version", action="store_true", dest="multi", default=None, help="make apps have to require() a version") (options, args) = parser.parse_args() try: if not args: parser.error("No urls, filenames, or requirements specified") for spec in args: inst = factory(options.instdir, options.zip_ok, options.multi) try: print "Downloading", spec downloaded = inst.download(spec) print "Installing", os.path.basename(downloaded) inst.install_eggs(downloaded) finally: inst.close() except RuntimeError, v: print >>sys.stderr,"error:",v sys.exit(1) if __name__ == '__main__': main(sys.argv[1:])