[Python-checkins] CVS: python/nondist/peps pep-0273.txt,1.5,1.6

Barry Warsaw bwarsaw@users.sourceforge.net
Tue, 13 Nov 2001 12:25:45 -0800


Update of /cvsroot/python/python/nondist/peps
In directory usw-pr-cvs1:/tmp/cvs-serv17128

Modified Files:
	pep-0273.txt 
Log Message:
Jim's latest revision.


Index: pep-0273.txt
===================================================================
RCS file: /cvsroot/python/python/nondist/peps/pep-0273.txt,v
retrieving revision 1.5
retrieving revision 1.6
diff -C2 -d -r1.5 -r1.6
*** pep-0273.txt	2001/11/02 20:44:57	1.5
--- pep-0273.txt	2001/11/13 20:25:43	1.6
***************
*** 13,17 ****
  Abstract
      This PEP adds the ability to import Python modules
!     *.py, *.py[co] and packages from zip archives.
  
  
--- 13,19 ----
  Abstract
      This PEP adds the ability to import Python modules
!     *.py, *.py[co] and packages from zip archives.  The
!     same code is used to speed up normal directory imports
!     provided os.listdir is available.
  
  
***************
*** 35,43 ****
      import all Python library files from an archive.
  
-     Reading compressed zip archives requires the zlib module.  An
-     import of zlib will be attempted prior to any other imports.  If
-     zlib is not available at that time, only uncompressed archives
-     will be readable, even if zlib subsequently becomes available.
- 
  
  Subdirectory Equivalence
--- 37,40 ----
***************
*** 45,51 ****
      The zip archive must be treated exactly as a subdirectory tree so
      we can support package imports based on current and future rules.
!     Zip archive files must be created with relative path names.  That
!     is, archive file names are of the form: file1, file2, dir1/file3,
!     dir2/dir3/file4.
  
      Suppose sys.path contains "/A/B/SubDir" and "/C/D/E/Archive.zip",
--- 42,47 ----
      The zip archive must be treated exactly as a subdirectory tree so
      we can support package imports based on current and future rules.
!     All zip data is taked from the Central Directory, the data must be
!     correct, and brain dead zip files are not accommodated.
  
      Suppose sys.path contains "/A/B/SubDir" and "/C/D/E/Archive.zip",
***************
*** 94,120 ****
      easy.
  
  
  zlib
  
      Compressed zip archives require zlib for decompression.  Prior to
!     any other imports, we attempt an import of zlib, and set a flag if
!     it is available.  All compressed files are invisible unless this
!     flag is true.
! 
!     It could happen that zlib was available later.  For example, the
!     import of site.py might add the correct directory to sys.path so a
!     dynamic load succeeds.  But compressed files will still be
!     invisible.  It is unknown if it can happen that importing site.py
!     can cause zlib to appear, so maybe we're worrying about nothing.
!     On Windows and Linux, the early import of zlib succeeds without
!     site.py.
! 
!     The problem here is the confusion caused by the reverse.  Either a
!     zip file satisfies imports or it doesn't.  It is silly to say that
!     site.py needs to be uncompressed, and that maybe imports will
!     succeed later.  If you don't like this, create uncompressed zip
!     archives or make sure zlib is available, for example, as a
!     built-in module.  Or we can write special search logic during zip
!     initialization.
  
  
--- 90,103 ----
      easy.
  
+     This same mechanism is used to speed up directory (non-zip) imports.
+     See below.
  
+ 
  zlib
  
      Compressed zip archives require zlib for decompression.  Prior to
!     any other imports, we attempt an import of zlib.  Import of
!     compressed files will fail with a message "missing zlib" unless
!     zlib is available.
  
  
***************
*** 134,146 ****
      Python versions on the same machine.
  
!     I propose that there is one name added to sys.path, and the
!     file name is "python%s%s.zip" % (sys.version[0], sys.version[2]).
!     For example, python22.zip.  This is the same on all platforms.
!     On Unix, the directory is sys.prefix + "/lib".  So for prefix
!     /usr/local, the path /usr/local/lib/python2.2/ is already on
!     sys.path, and /usr/local/lib/python22.zip would be added.
!     On Windows, the directory is the directory of sys.executable.
!     The zip archive name is always inserted as the second item
!     in sys.path.  The first is the directory of the main.py (thanks Tim).
  
  Implementation
--- 117,179 ----
      Python versions on the same machine.
  
!     We add one name to sys.path.  On Unix, the directory is
!     sys.prefix + "/lib", and the file name is
!     "python%s%s.zip" % (sys.version[0], sys.version[2]).
!     So for Python 2.2 and prefix /usr/local, the path
!     /usr/local/lib/python2.2/ is already on sys.path, and
!     /usr/local/lib/python22.zip would be added.
!     On Windows, the file is the full path to python22.dll, with
!     "dll" replaced by "zip".  The zip archive name is always inserted
!     as the second item in sys.path.  The first is the directory of the
!     main.py (thanks Tim).
! 
! 
! Directory Imports
! 
!     The static Python dictionary used to speed up zip imports can be
!     used to speed up normal directory imports too.  For each item in
!     sys.path that is not a zip archive, we call os.listdir, and add
!     the directory contents to the dictionary.  Then instead of calling
!     fopen() in a double loop, we just check the dictionary.  This
!     greatly speeds up imports.  If os.listdir doesn't exist, the
!     dictionary is not used.
! 
! 
! Benchmarks
! 
!     Case  Original 2.2a3    Using os.listdir   Zip Uncomp  Zip Compr
!     ---- -----------------  -----------------  ----------  ----------
!       1  3.2 2.5 3.2->1.02  2.3 2.5 2.3->0.87  1.66->0.93  1.5->1.07
!       2  2.8 3.9 3.0->1.32  Same as Case 1.
!       3  5.7 5.7 5.7->5.7   2.1 2.1 2.1->1.8   1.25->0.99  1.19->1.13
!       4  9.4 9.4 9.3->9.35  Same as Case 3.
! 
!     Case 1: Local drive C:, sys.path has its default value.
!     Case 2: Local drive C:, directory with files is at the end of sys.path.
!     Case 3: Network  drive, sys.path has its default value.
!     Case 4: Network  drive, directory with files is at the end of sys.path.
! 
!     Benchmarks were performed on a Pentium 4 clone, 1.4 GHz, 256 Meg.
!     The machine was running Windows 2000 with a Linux/Samba network server.
!     Times are in seconds, and are the time to import about 100 Lib modules.
!     Case 2 and 4 have the "correct" directory moved to the end of sys.path.
!     "Uncomp" means uncompressed zip archive, "Compr" means compressed.
! 
!     Initial times are after a re-boot of the system; the time after
!     "->" is the time after repeated runs.  Times to import from C:
!     after a re-boot are rather highly variable for the "Original" case,
!     but are more realistic.
! 
! 
! Custom Imports
! 
!     The logic demonstrates the ability to import using default searching
!     until a needed Python module (in this case, os) becomes available.
!     This can be used to bootstrap custom importers.  For example, if
!     "importer()" in __init__.py exists, then it could be used for imports.
!     The "importer()" can freely import os and other modules, and these
!     will be satisfied from the default mechanism.  This PEP does not
!     define any custom importers, and this note is for information only.
! 
  
  Implementation