[Python-checkins] r72680 - peps/trunk/pep-0376.txt

tarek.ziade python-checkins at python.org
Sat May 16 18:02:06 CEST 2009


Author: tarek.ziade
Date: Sat May 16 18:02:06 2009
New Revision: 72680

Log:
changes after feedback from python-dev

Modified:
   peps/trunk/pep-0376.txt

Modified: peps/trunk/pep-0376.txt
==============================================================================
--- peps/trunk/pep-0376.txt	(original)
+++ peps/trunk/pep-0376.txt	Sat May 16 18:02:06 2009
@@ -22,28 +22,41 @@
 Definitions
 ===========
 
-A **project** is a Python application composed of one or many Python packages.
-It is distributed using a `setup.py` script with Distutils and/or Setuptools.
-
-Once installed, one or several **packages** are added in Python's site-packages.
+A **project** is a Python application composed of one or several files, which can
+be Python modules, extensions or data. It is distributed using a `setup.py` script 
+with Distutils and/or Setuptools. The `setup.py` script indicates where each 
+elements should be installed.
+
+Once installed, the elements are located in various places in the system, like:
+
+- in Python's site-packages (Python modules, Python modules organized into packages, 
+  Extensions, etc.)
+- in Python's `include` directory.
+- in Python's `bin` or `Script` directory.
+- etc.
 
 Rationale
 =========
 
 There are two problems right now in the way projects are installed in Python:
 
-- There are too many ways to install a project in Python.
+- There are too many ways to do it.
 - There is no API to get the metadata of installed projects.
 
 How projects are installed
 --------------------------
 
-Right now, when a project is installed in Python, every package its contains 
-is installed in the `site-packages` directory with the Distutils `install` 
-command.
+Right now, when a project is installed in Python, every elements its contains 
+is installed in various directories. 
+
+The pure Python code for instance is  installed in the `purelib` directory,
+which is located in the Python  installation in `lib\python2.6\site-packages`
+for example under unix-like systems or Mac OS X, and in `Lib/site-packages` 
+under Windows. This is done with the Distutils `install` command, which calls
+various subcommands.
 
 The `install_egg_info` subcommand is called during this process, in order to 
-create an `.egg-info` file in the `site-packages` directory.
+create an `.egg-info` file in the `purelib` directory.
 
 For example, if the `zlib` project (which contains one package) is installed,
 two elements will be installed in `site-packages`::
@@ -51,8 +64,8 @@
  - zlib
  - zlib-2.5.2-py2.4.egg-info
 
-Where `zlib` is the package, and `zlib-2.5.2-py2.4.egg-info` is
-a file containing the package metadata as described in PEP 314.
+Where `zlib` is a Python package, and `zlib-2.5.2-py2.4.egg-info` is
+a file containing the project metadata as described in PEP 314 [#pep314]_.
 
 This file corresponds to the file called `PKG-INFO`, built by 
 the `sdist` command.
@@ -63,11 +76,13 @@
 
 - `easy_install` creates an `EGG-INFO` directory inside an `.egg` directory, 
   and adds a `PKG-INFO` file inside this directory. The `.egg` directory 
-  contains in that case the packages of the project.
-
-- `pip` creates an `.egg-info` directory inside the site-packages directory
-  and adds a `PKG-INFO` file inside it. Packages are installed in 
-  site-packages directory in a regular way.
+  contains in that case all the elements of the project that are supposed to
+  be installed in `site-packages`, and is placed in the `site-packages` 
+  directory.
+
+- `pip` creates an `.egg-info` directory inside the `site-packages` directory
+  and adds a `PKG-INFO` file inside it. Elements of the project are then
+  installed in various places like Distutils does.
 
 They both add other files in the `EGG-INFO` or `.egg-info` directory, and
 create or modify `.pth` files.
@@ -76,16 +91,19 @@
 ---------------------
 
 Distutils doesn't provide any `uninstall` command. If you want to uninstall
-a project, you have to be a power user and remove the various package 
-directories from the right `site-packages` directory, then look over the right 
-`pth` files. And this method differs, depending on the tools you are using.
-
-The worst issue is that you depend on the way the packager created his package.
-When you call `python setup.py install`, it will not be installed the same way
-depending on the tool used by the packager (mainly Distutils or Setuptools).
+a project, you have to be a power user and remove the various elements that
+were installed. Then look over the `.pth` file to clean them if necessary.
+
+And the process differs, depending on the tools you have used to install the
+project, and if the project's `setup.py` uses Distutils or Setuptools.
 
-But there's common behavior: files are copied in your installation.
-And there's a way to keep track of theses file, so to remove them.
+Under some circumstances, you might not be able to know for sure that you 
+have removed everything, or that you didn't break another project by
+removing a file that was shared among the two projects.
+
+But there's common behavior: when you install a project, files are copied 
+in your system. And there's a way to keep track of theses files, so to remove 
+them.
 
 What this PEP proposes
 ----------------------
@@ -101,51 +119,58 @@
 =============================
 
 The first change would be to make `.egg-info` a directory and let it
-hold the `PKG-INFO` file built by the `write_pkg_file` method.
+hold the `PKG-INFO` file built by the `write_pkg_file` method of 
+the `Distribution` class in Distutils.
 
-This change will not impact Python itself, because this file is not
-used anywhere yet in the standard library. So there's no need of
-deprecation.
+This change will not impact Python itself, because `egg-info` files are not
+used anywhere yet in the standard library besides Distutils. 
 
 Although it will impact the `setuptools` and `pip` projects, but given 
-the fact that they already work with a directory that contains a 
-`PKG-INFO` file, the change will be small.
+the fact that they already work with a directory that contains a `PKG-INFO` 
+file, the change will have no deep consequences.
 
-For example, if the `zlib` package is installed, two elements
-will be installed in `site-packages`::
+For example, if the `zlib` package is installed, the elements that
+will be installed in `site-packages` will become::
 
     - zlib
     - zlib-2.5.2.egg-info/
         PKG-INFO
 
-The Python version will also be removed from the .egg-info directory
-name. To be able to implement this change, the impacted code in Distutils
-is the `install_egg_info` command, and the various third-party projects.
+The Python version will also be removed from the `.egg-info` directory
+name.
 
 Adding a RECORD in the .egg-info directory
 ==========================================
 
 A `RECORD` file will be added inside the `.egg-info` directory at installation
-time.
+time. The `RECORD` file will hold the list of installed files. These correspond 
+to the files listed by the `record` option of the `install` command, and will 
+always be generated. This will allow uninstallation, as explained later in this 
+PEP. This RECORD file is inspired from PEP 262 FILES [#pep262]_.
+
+The RECORD format
+-----------------
+
+The `RECORD` file is composed of records, one line per installed file.
+Each record is composed of three elements separated by a `;` character:
+
+- the file's full **path**
 
-- the `RECORD` file will hold the list of installed files. These
-  correspond to the files listed by the `record` option of the `install` 
-  command, and will always be generated. This will allow uninstallation, as
-  explained later in this PEP.
-
-The `install` command will record by default installed files in the
-RECORD file, using these rules:
-
-- if the installed file is located in a directory in `site-packages`,
-  it will be a '/'-separated relative path, no matter what is the target 
-  system. This makes this information cross-compatible and allows simple
-  installation to be relocatable.
+ - if the installed file is located in a directory in `site-packages`,
+   it will be a '/'-separated relative path, no matter what is the target 
+   system. This makes this information cross-compatible and allows simple
+   installation to be relocatable.
 
-- if the installed file is located elsewhere in the system, a 
-  '/'-separated absolute path is used.
+ - if the installed file is located elsewhere in the system, a 
+   '/'-separated absolute path is used.
 
-This will require changing the way the `install` command writes the record
-file, so the old `record` behavior will be deprecated.
+- the **MD5** hash of the file, encoded in hex. Notice that `pyc` and `pyo`
+  generated files will not have a hash.
+
+- the file's size in bytes
+
+Example
+-------
 
 Back to our `zlib` example, we will have::
 
@@ -154,6 +179,21 @@
         PKG-INFO
         RECORD
 
+And the RECORD file will contain::
+
+    zlib/include/zconf.h;b690274f621402dda63bf11ba5373bf2;9544
+    zlib/include/zlib.h;9c4b84aff68aa55f2e9bf70481b94333;66188
+    zlib/lib/libz.a;e6d43fb94292411909404b07d0692d46;91128
+    zlib/share/man/man3/zlib.3;785dc03452f0508ff0678fba2457e0ba;4486
+    zlib-2.5.2.egg-info/PKG-INFO;6fe57de576d749536082d8e205b77748;195
+    zlib-2.5.2.egg-info/RECORD
+
+Notice that:
+
+- the `RECORD` file can't contain a hash of itself and is just mentioned here
+- `zlib` and `zlib-2.5.2.egg-info` are located in `site-packages` so the file 
+  paths are relative to it.
+
 New functions in pkgutil
 ========================
 
@@ -178,16 +218,31 @@
 
   Uses `get_egg_info` to get the `PKG-INFO` file, and returns a 
   `DistributionMetadata` instance that contains the metadata.
-  This will require a small change in `DistributionMetadata` (see #4908).
 
-- get_egg_info_file(project_name, filename) -> file object or None
+- get_files(project_name) -> iterator of (path, hash, size, other_projects)
+
+  Uses `get_egg_info` to get the `RECORD` file, and returns an iterator.
+
+  Each returned element is a tuple `(path, hash, size, other_projects)` where
+  ``path`, ``hash``, ``size`` are the values found in the RECORD file.
 
-  Uses `get_egg_info` and gets any file inside the directory,
-  pointed by filename.
+  `other_projects` is a tuple containing the name of the projects that are 
+  also referring to this file in their own RECORD file (same path).
+
+  If `other_projects` is empty, it means that the file is only referred by the
+  current project. In other words, it can be removed if the project is removed.
+
+- get_egg_info_file(project_name, path) -> file object or None
+
+  Uses `get_egg_info` and gets any element inside the directory,
+  pointed by its relative path. `get_egg_info_file` will perform
+  an `os.path.join` on `get_egg_info(project_name)` and `path` to build the 
+  whole path.
 
 Let's use it with our `zlib` example::
 
-    >>> from pkgutil import get_egg_info, get_metadata, get_egg_info_file
+    >>> from pkgutil import (get_egg_info, get_metadata, get_egg_info_file, 
+    ...                      get_files)
     >>> get_egg_info('zlib')
     '/opt/local/lib/python2.6/site-packages/zlib-2.5.2.egg-info'
     >>> metadata = get_metadata('zlib')
@@ -197,6 +252,16 @@
     some
     ...
     files
+    >>> for path, hash, size, other_projects in get_files('zlib'):
+    ...     print '%s %s %d %s' % (path, hash, size, ','.join(other_projects))
+    ...
+    zlib/include/zconf.h b690274f621402dda63bf11ba5373bf2 9544
+    zlib/include/zlib.h 9c4b84aff68aa55f2e9bf70481b94333 66188
+    zlib/lib/libz.a e6d43fb94292411909404b07d0692d46 91128 
+    zlib/share/man/man3/zlib.3 785dc03452f0508ff0678fba2457e0ba 4486 
+    zlib-2.5.2.egg-info/PKG-INFO 6fe57de576d749536082d8e205b77748 195 
+    zlib-2.5.2.egg-info/RECORD None None 
+
 
 Adding an Uninstall function
 ============================
@@ -204,14 +269,13 @@
 Distutils provides a very basic way to install a project, which is running
 the `install` command over the `setup.py` script of the distribution.
 
-Distutils will provide a very basic `uninstall` command that will remove
-all files listed in the `RECORD` file of a project, as long as they are not 
-mentioned in another `RECORD` file and as long as the package is installed
-using the standard described earlier.
-
-This command will be added in ``distutils.util`` and will take the name
-of the project to uninstall as its argument. A call to uninstall will return a 
-list of uninstalled files::
+Distutils will provide a very basic ``uninstall`` function, that will be added 
+in ``distutils.util`` and will take the name of the project to uninstall as 
+its argument. ``uninstall`` will use ``pkgutil.get_files`` and remove all 
+unique files, as long as their hash didn't change. Then it will remove
+directories where it removed the last elements.
+
+``uninstall`` will return a list of uninstalled files::
 
     >>> from distutils.util import uninstall
     >>> uninstall('zlib')
@@ -220,23 +284,28 @@
 
 If the project is not found, a ``DistutilsUninstallError`` will be raised.
 
-To make it a reference API for third-party projects that wish to provide
-an `uninstall feature`. The ``uninstall`` function can also be invoked with a 
-second callable argument, that will be invoked for each file to be removed.
-If this callable returns `True`, the file will be removed.
+To make it a reference API for third-party projects that wish to control 
+how `uninstall` works, a second callable argument can be used. It will be 
+called for each file that is removed. If the callable returns `True`, the 
+file will be removed. If it returns False, it will be left alone.
 
 Examples::
 
     >>> def _remove_and_log(path):
     ...     logging.info('Removing %s' % path)
     ...     return True
+    ...
     >>> uninstall('zlib', _remove_and_log)
 
     >>> def _dry_run(path):
     ...     logging.info('Removing %s (dry run)' % path)
     ...     return False
+    ...
     >>> uninstall('zlib', _dry_run)
 
+Of course, a third-party tool can use ``pkgutil.get_files`` for a maximum 
+control, to implement their own uninstall feature.
+
 Backward compatibility and roadmap
 ==================================
 
@@ -244,10 +313,20 @@
 version of Distutils, and will also work with existing third-party tools.
 
 Although, a backport of the new Distutils for 2.5, 2.6, 3.0 and 3.1 will be
-provided so people can benefit from the new features.
+provided so people can benefit from these new features.
 
 The plan is to integrate them for Python 2.7 and Python 3.2
 
+References
+==========
+
+.. [#pep262]
+   http://www.python.org/dev/peps/pep-0262
+
+.. [#pep314]
+   http://www.python.org/dev/peps/pep-0314
+
+
 Aknowledgments
 ==============
 


More information about the Python-checkins mailing list