[Python-checkins] r58166 - in sandbox/trunk/import_in_py: Py3K Py3K/Py3K_TODO Py3K/_importlib.py Py3K/importlib.py Py3K/tests Py3K/tests/__init__.py Py3K/tests/builtin_frozen_help.py Py3K/tests/ext_help.py Py3K/tests/mock_importlib.py Py3K/tests/py_help.py Py3K/tests/test_builtin_importer.py Py3K/tests/test_chained_importer.py Py3K/tests/test_frozen_importer.py Py3K/tests/test_fs_importer.py Py3K/tests/test_fs_loader.py Py3K/tests/test_import_mechanics.py Py3K/tests/test_regression.py Py3K_TODO
brett.cannon
python-checkins at python.org
Mon Sep 17 00:20:59 CEST 2007
Author: brett.cannon
Date: Mon Sep 17 00:20:58 2007
New Revision: 58166
Added:
sandbox/trunk/import_in_py/Py3K/
sandbox/trunk/import_in_py/Py3K/Py3K_TODO
- copied, changed from r57667, sandbox/trunk/import_in_py/Py3K_TODO
sandbox/trunk/import_in_py/Py3K/_importlib.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/importlib.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/
sandbox/trunk/import_in_py/Py3K/tests/__init__.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/builtin_frozen_help.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/ext_help.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/mock_importlib.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/py_help.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/test_builtin_importer.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/test_chained_importer.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/test_frozen_importer.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/test_fs_importer.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/test_fs_loader.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/test_import_mechanics.py (contents, props changed)
sandbox/trunk/import_in_py/Py3K/tests/test_regression.py (contents, props changed)
Removed:
sandbox/trunk/import_in_py/Py3K_TODO
Log:
Start a Py3K version of importlib. As of right now it doesn't work as there is
at least one test failing because of integer division issues. Plus I know that
_r_long fails in the face of a bytecode timestamp since that is being passed
bytes instead of a string.
Copied: sandbox/trunk/import_in_py/Py3K/Py3K_TODO (from r57667, sandbox/trunk/import_in_py/Py3K_TODO)
==============================================================================
--- sandbox/trunk/import_in_py/Py3K_TODO (original)
+++ sandbox/trunk/import_in_py/Py3K/Py3K_TODO Mon Sep 17 00:20:58 2007
@@ -7,7 +7,8 @@
* Put into 2.6 stdlib.
- Py3K-specific
* Port to Py3K.
- + Consider it a fork of code as semantics can change compared to 2.6 code.
+ + Write script that will copy certain files (i.e., all files that have
+ no Py3K-specific modifications beyond 2to3) and pass through 2to3.
* Bootstrap in as default import implementation.
* Fix C API to call importlib.
* Remove C implementation.
Added: sandbox/trunk/import_in_py/Py3K/_importlib.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/_importlib.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,976 @@
+"""Implementation of import in Python source code.
+
+Direct imports are not allowed because of dependency issues. Any required
+modules must be injected from an outside source.
+
+Default parameter values cannot be from a module as any injected module will
+come after the module has been imported.
+
+====================
+References on import
+====================
+* Language reference
+ http://docs.python.org/ref/import.html
+* __import__ function
+ http://docs.python.org/lib/built-in-funcs.html
+* Packages
+ http://www.python.org/doc/essays/packages.html
+* PEP 235: Import on Case-Insensitive Platforms
+ http://www.python.org/dev/peps/pep-0235
+* PEP 275: Import Modules from Zip Archives
+ http://www.python.org/dev/peps/pep-0273
+* PEP 302: New Import Hooks
+ http://www.python.org/dev/peps/pep-0302/
+* PEP 328: Imports: Multi-line and Absolute/Relative
+ http://www.python.org/dev/peps/pep-0328
+
+============
+Known Issues
+============
+* runpy ('-m' command-line option for Python) does not work.
+ + Requires get_code to be implemented for loaders.
+ + Uses pkgutil.get_loader which fakes a loader if __loader__ is not defined.
+ + New loaders do define __loader__ but not get_code, and thus dies on an
+ AttributeError.
+ + Possible fix
+ - Implement optional interface for loaders.
+ - Do not have new loaders set __loader__.
+ * Might happen because of security issues.
+* warnings and stack level.
+ + Affected tests
+ - test___all__
+ - test_imageop
+ + 'warnings' assumes that the import code does not show up in the call
+ stack.
+ + Because import now implemented in Python, import does show up in the
+ call stack.
+ + Trick of specifying going back in the call stack two levels so warning
+ shows up in the caller will cause the warning no longer holds true.
+ + Possible fixes
+ - Special module deprecation function.
+ - Code in warnings.warn to handle special import case.
+ - Flag on warnings.warn that warning is for an import and ignore stack
+ level argument.
+ * Could also infer from type of warning.
+* test_pkg
+ + Old-style test that compares output.
+ + Setting of __loader__ leads to different output.
+
+"""
+from __future__ import with_statement
+# The injected modules are 'imp', 'sys', 'marshal', and '_os' (a.k.a. 'posix',
+# 'nt' & 'os2').
+
+
+# XXX Could also expose Modules/getpath.c:joinpath()
+def _path_join(*args):
+ """Replacement for os.path.join so as to remove dependency on os module."""
+ return path_sep.join(args)
+
+def _path_exists(path):
+ """Replacement for os.path.exists to help remove dependency on 'os'."""
+ try:
+ _os.stat(path)
+ except OSError:
+ return False
+ else:
+ return True
+
+
+def _path_is_mode_type(path, mode):
+ """Test whether the path is the specified mode type."""
+ try:
+ stat_info = _os.stat(path)
+ except OSError:
+ return False
+ return (stat_info.st_mode & 0o170000) == mode
+
+# XXX Could also expose Modules/getpath.c:isfile()
+def _path_isfile(path):
+ """Replacement for os.path.isfile."""
+ return _path_is_mode_type(path, 0o100000)
+
+# XXX Could also expose Modules/getpath.c:isdir()
+def _path_isdir(path):
+ """Replacement for os.path.isdir."""
+ return _path_is_mode_type(path, 0o040000)
+
+def _path_without_ext(path, ext_type):
+ """Replacement for os.path.splitext()[0]."""
+ for suffix in suffix_list(ext_type):
+ if path.endswith(suffix):
+ return path[:-len(suffix)]
+ else:
+ raise ValueError("path is not of the specified type")
+
+def _path_absolute(path):
+ """Replacement for os.path.abspath."""
+ try:
+ return _os._getfullpathname(path)
+ except AttributeError:
+ if path.startswith('/'):
+ return path
+ else:
+ return _path_join(_os.getcwd(), path)
+
+
+class _BuiltinFrozenBaseImporter(object):
+
+ """Base class for meta_path importers for built-in and frozen modules.
+
+ Subclasses must provide the _find and _load methods.
+
+ """
+
+ def find_module(self, fullname, path=None):
+ """See if a built-in or frozen module can be imported based on the
+ specified name."""
+ if self._find(fullname):
+ return self
+ else:
+ return None
+
+ def load_module(self, fullname):
+ """Load a built-in or frozen module.
+
+ 'imp' code for loading a built-in or frozen module handles the setting
+ of a module in sys.modules before initializing the module. This means
+ that we do not need to perform that step as outlined by PEP 302.
+
+ """
+ try:
+ return sys.modules[fullname]
+ except KeyError:
+ mod = self._load(fullname)
+ if not mod:
+ raise ImportError("expected built-in module not loaded")
+ return mod
+
+
+class BuiltinImporter(_BuiltinFrozenBaseImporter):
+
+ """sys.meta_path class for importing built-in modules.
+
+ XXX Possible optimization is to bail out in find_module() if 'path' is set
+ to a value.
+
+ """
+
+ def __init__(self):
+ self._find = imp.is_builtin
+ self._load = imp.init_builtin
+
+
+class FrozenImporter(_BuiltinFrozenBaseImporter):
+
+ """sys.meta_path class for importing frozen modules."""
+
+ def __init__(self):
+ self._find = imp.is_frozen
+ self._load = imp.init_frozen
+
+
+class ChainedImporter(object):
+
+ """Importer that sequentially calls other importers."""
+
+ def __init__(self, *importers):
+ self._importers = importers
+
+ def find_module(self, fullname, path=None):
+ for importer in self._importers:
+ result = importer.find_module(fullname, path)
+ if result:
+ return result
+ else:
+ return None
+
+
+# XXX Make generic and push fs code into FileImporter?
+def chaining_fs_path_hook(*path_hooks):
+ """Create a path hook that sequentially asks other path hooks if they can
+ handle a sys.path entry, returning a chained importer for those that said
+ they could."""
+ def chained_fs_path_hook(path_entry):
+ """Check which of the associated importers are able to handle the
+ filesystem path entry."""
+ absolute_path = _path_absolute(path_entry)
+ if not _path_isdir(absolute_path):
+ raise ImportError("only directories are supported")
+ accepted = []
+ for path_hook in path_hooks:
+ try:
+ accepted.append(path_hook(absolute_path))
+ except ImportError:
+ continue
+ if not accepted:
+ raise ImportError("no path hooks could handle %s" % path_entry)
+ return ChainedImporter(*accepted)
+ return chained_fs_path_hook
+
+
+def check_name(method):
+ """Decorator to verify that the module being requested matches the one the
+ loader can handle."""
+ def inner(self, name, *args, **kwargs):
+ if self._name != name:
+ raise ImportError("loader cannot handle %s" % name)
+ return method(self, name, *args, **kwargs)
+ inner.__name__ = method.__name__
+ inner.__doc__ = method.__doc__
+ inner.__dict__.update(method.__dict__)
+ return inner
+
+
+class _ExtensionFileLoader(object):
+
+ """Provide the ability to load an extension module."""
+
+ def __init__(self, name, path, is_pkg):
+ """Initialize the loader, ignoring the is_pkg argument."""
+ self._name = name
+ self._path = path
+ if is_pkg:
+ raise ValueError("extension modules cannot be packages")
+
+ @check_name
+ def load_module(self, fullname):
+ """Load an extension module."""
+ assert self._name == fullname
+ try:
+ module = imp.load_dynamic(fullname, self._path)
+ module.__loader__ = self
+ return module
+ except:
+ # If an error occurred, don't leave a partially initialized module.
+ if fullname in sys.modules:
+ del sys.modules[fullname]
+ raise
+
+ @check_name
+ def is_package(self, fullname):
+ """Return False as an extension module can never be a package."""
+ return False
+
+ @check_name
+ def get_code(self, fullname):
+ """Return None as an extension module cannot create just a code
+ object."""
+ return None
+
+ @check_name
+ def get_source(self, fullname):
+ """Return None as extension modules have no source code."""
+ return None
+
+
+def suffix_list(suffix_type):
+ """Return a list of suffixes for a specific type of file."""
+ return [suffix[0] for suffix in imp.get_suffixes()
+ if suffix[2] == suffix_type]
+
+
+def init_module(loader, code_object, name, path, is_pkg):
+ """Return an initialized module.
+
+ If initialization fails and the module did not already exist, then remove
+ it from sys.modules to prevent a partially initialized module from being
+ left around. If this action is for a reload (i.e., the module already
+ existed), leave the original module state in place.
+
+ """
+ module = sys.modules.get(name)
+ is_reload = True if module is not None else False
+ if not is_reload:
+ module = imp.new_module(name)
+ sys.modules[name] = module
+ else:
+ original_values = {}
+ modified_attrs = ['__loader__', '__name__', '__file__']
+ for attr in modified_attrs:
+ try:
+ original_values[attr] = getattr(module, attr)
+ except AttributeError:
+ pass
+ module.__loader__ = loader
+ module.__name__ = name
+ module.__file__ = path
+ if is_pkg:
+ module.__path__ = [module.__file__.rsplit(path_sep, 1)[0]]
+ try:
+ exec(code_object, module.__dict__)
+ except:
+ if not is_reload:
+ del sys.modules[name]
+ else:
+ for attr in modified_attrs:
+ if attr in original_values:
+ setattr(module, attr, original_values[attr])
+ else:
+ delattr(module, attr)
+ raise
+ return module
+
+
+def handle_code(loader, name, source_path, bytecode_path):
+ """Return the code object and the location of the code used to create the
+ code object in a two-item sequence.
+
+ The loader is expected to implement:
+
+ + get_bytecode(name:str) -> str
+ Read bytecode.
+ Only called if bytecode_path is true.
+ + mod_time(name:str) -> int
+ Return the last modification time for the source of the specified
+ module.
+ Only called if both source_path and bytecode_path are true.
+ + get_source(name:str) -> str
+ Return the source code for the module.
+ Only called if source_path is true.
+ + write_bytecode(name:str, timestamp:int, data:str) -> bool
+ Write out bytecode for the module with the specified timestamp,
+ returning a boolean based on whether any actual storage of the bytecode
+ was done.
+
+ """
+ # Request paths instead of just booleans since 'compile' needs it for
+ # source.
+ if not source_path and not bytecode_path:
+ raise ValueError("neither source nor bytecode was specified as "
+ "available")
+ source_timestamp = None
+ # Try to use bytecode if it is available.
+ if bytecode_path:
+ magic, pyc_timestamp, bytecode = loader.get_bytecode(name)
+ try:
+ # Verify that the magic number is valid.
+ if imp.get_magic() != magic:
+ raise ImportError("bad magic number")
+ # Verify that the bytecode is not stale.
+ if source_path:
+ # Keep in a variable so that if bytecode turns out to be bad
+ # then the value can be reused when re-generating bytecode.
+ source_timestamp = loader.mod_time(name)
+ if pyc_timestamp < source_timestamp:
+ raise ImportError("bytcode is stale")
+ # Bytecode seems fine, so try to use it.
+ try:
+ return marshal.loads(bytecode), bytecode_path
+ except ValueError:
+ # XXX Is it useful to bail out when the bytecode is bad? Why
+ # not use the source if possible and issue a warning that the
+ # bytecode was bad?
+ # Since bad bytecode halts the import entirely, having the
+ # source code is useless. Signal this fact by setting
+ # bytecode_path to None.
+ bytecode_path = None
+ raise ImportError('Non-code object found')
+ except ImportError:
+ # If the exception is from the bytecode being corrupt, let it
+ # propagate.
+ if not source_path or bytecode_path is None:
+ raise
+ # Use the source.
+ source = loader.get_source(name)
+ code_object = compile(source, source_path, 'exec')
+ # Generate bytecode and write it out.
+ if source_timestamp is None: # May have from bytecode verification.
+ source_timestamp = loader.mod_time(name)
+ data = marshal.dumps(code_object)
+ loader.write_bytecode(name, source_timestamp, data)
+ return code_object, source_path
+
+
+class _PyFileLoader(object):
+ # XXX Still smart to have this as a separate class? Or would it work
+ # better to integrate with PyFileImporter? Could cache _is_pkg info.
+ # FileImporter can be changed to return self instead of a specific loader
+ # call. Otherwise _base_path can be calculated on the fly without issue if
+ # it is known whether a module should be treated as a path or package to
+ # minimize stat calls. Could even go as far as to stat the directory the
+ # importer is in to detect changes and then cache all the info about what
+ # files were found (if stating directories is platform-dependent).
+
+ """Load a Python source or bytecode file."""
+
+ _handler = handle_code
+ _module_init = init_module
+
+ def __init__(self, name, path, is_pkg):
+ self._name = name
+ self._is_pkg = is_pkg
+ # Figure out the base path based on whether it was source or bytecode
+ # that was found.
+ try:
+ self._base_path = _path_without_ext(path, imp.PY_SOURCE)
+ except ValueError:
+ self._base_path = _path_without_ext(path, imp.PY_COMPILED)
+
+ def _find_path(self, ext_type):
+ """Find a path from the base path and the specified extension type that
+ exists, returning None if one is not found."""
+ for suffix in suffix_list(ext_type):
+ path = self._base_path + suffix
+ if _path_exists(path):
+ return path
+ else:
+ return None
+
+
+ def _source_path(self):
+ """Return the path to an existing source file for the module, or None
+ if one cannot be found."""
+ # Not a property so that it is easy to override.
+ return self._find_path(imp.PY_SOURCE)
+
+ def _bytecode_path(self):
+ """Return the path to a bytecode file, or None if one does not
+ exist."""
+ # Not a property for easy overriding.
+ return self._find_path(imp.PY_COMPILED)
+
+ @check_name
+ def load_module(self, fullname):
+ """Load a Python source or bytecode file."""
+ try:
+ code_object, path = self._handler(fullname, self._source_path(),
+ self._bytecode_path())
+ except ValueError:
+ raise ImportError("loader cannot handle %s" % fullname)
+ return self._module_init(code_object, fullname, path, self._is_pkg)
+
+ @check_name
+ def mod_time(self, name):
+ """Return the modification time of the source for the specified module.
+
+ Raise ValueError if no source exists or ImportError (like get_source)
+ if the loader cannot handle the module.
+
+ """
+ source_path = self._source_path()
+ if source_path:
+ return int(_os.stat(source_path).st_mtime)
+ else:
+ raise ValueError('no source for %s' % name)
+
+ @check_name
+ def get_source(self, fullname):
+ """Return the source for the specified module (using universal
+ newlines), or None if it is not available.
+
+ Raises ImportError if the loader cannot handle the module.
+
+ """
+ source_path = self._source_path()
+ if source_path:
+ return open(source_path, 'U').read()
+ elif self._bytecode_path():
+ return None
+ else:
+ raise ImportError('no source or bytecode available')
+
+ @check_name
+ def get_bytecode(self, name):
+ """Return the magic number, timestamp, and the module bytecode for the
+ module.
+
+ Raises ImportError (just like get_source) if the laoder cannot handle
+ the module.
+
+ """
+ try:
+ with open(self._bytecode_path(), 'rb') as bytecode_file:
+ data = bytecode_file.read()
+ return data[:4], _r_long(data[4:8]), data[8:]
+ except AttributeError:
+ return None
+
+ @check_name
+ def write_bytecode(self, name, timestamp, data):
+ """Write out 'data' for the specified module using the specific
+ timestamp, returning a boolean
+ signifying if the write-out actually occurred.
+
+ Raises ImportError (just like get_source) if the specified module
+ cannot be handled by the loader.
+
+ """
+ bytecode_path = self._bytecode_path()
+ if not bytecode_path:
+ bytecode_path = self._base_path + suffix_list(imp.PY_COMPILED)[0]
+ try:
+ with open(bytecode_path, 'wb') as bytecode_file:
+ bytecode_file.write(imp.get_magic())
+ bytecode_file.write(_w_long(timestamp))
+ bytecode_file.write(data)
+ return True
+ except IOError as exc:
+ if exc.errno == EACCES:
+ return False
+ else:
+ raise
+
+ def get_data(self, path):
+ """Return the data from path as raw bytes."""
+ return open(path, 'rb').read()
+
+ @check_name
+ def is_package(self, fullname):
+ """Return a boolean based on whether the module is a package.
+
+ Raises ImportError (like get_source) if the loader cannot handle the
+ package.
+
+ """
+ return self._is_pkg
+
+ @check_name
+ def get_code(self, fullname):
+ """Return a code object for the module.
+
+ Raise ImportError (like get_source) if the load cannot handle the
+ specified module.
+
+ """
+ try:
+ return self._handler(fullname, self._source_path(),
+ self._bytecode_path())[0]
+ except ValueError:
+ raise ImportError("cannot get the code for %s" % fullname)
+
+
+class FileImporter(object):
+
+ """Base class for file importers.
+
+ Subclasses are expected to define the following attributes:
+
+ * _suffixes
+ Sequence of file suffixes whose order will be followed.
+ * _possible_package
+ True if importer should check for packages.
+ * _loader
+ A callable that takes the module name, a file path, and whether
+ the path points to a package and returns a loader for the module found
+ at that path.
+
+ """
+
+ def __init__(self, path_entry):
+ """Initialize an importer for the passed-in sys.path entry (which is
+ assumed to have already been verified as an existing directory).
+
+ Can be used as an entry on sys.path_hook.
+
+ """
+ self._path_entry = path_entry
+
+ def find_module(self, fullname, path=None):
+ tail_module = fullname.rsplit('.', 1)[-1]
+ package_directory = None
+ if self._possible_package:
+ for ext in self._suffixes:
+ package_directory = _path_join(self._path_entry, tail_module)
+ init_filename = '__init__' + ext
+ package_init = _path_join(package_directory, init_filename)
+ if (_path_isfile(package_init) and
+ _case_ok(self._path_entry, tail_module) and
+ _case_ok(package_directory, init_filename)):
+ return self._loader(fullname, package_init, True)
+ for ext in self._suffixes:
+ file_name = tail_module + ext
+ file_path = _path_join(self._path_entry, file_name)
+ if (_path_isfile(file_path) and
+ _case_ok(self._path_entry, file_name)):
+ return self._loader(fullname, file_path, False)
+ else:
+ # Raise a warning if it matches a directory w/o an __init__ file.
+ if (package_directory is not None and
+ _path_isdir(package_directory) and
+ _case_ok(self._path_entry, tail_module)):
+ warnings.warn("Not importing directory %s: missing __init__"
+ % package_directory,
+ ImportWarning)
+ return None
+
+
+class ExtensionFileImporter(FileImporter):
+
+ """Importer for extension files."""
+
+ _possible_package = False
+ _loader = _ExtensionFileLoader
+
+ def __init__(self, path_entry):
+ # Assigning to _suffixes here instead of at the class level because
+ # imp is not imported at the time of class creation.
+ self._suffixes = suffix_list(imp.C_EXTENSION)
+ super(ExtensionFileImporter, self).__init__(path_entry)
+
+
+class PyFileImporter(FileImporter):
+
+ """Importer for source/bytecode files."""
+
+ _possible_package = True
+ _loader = _PyFileLoader
+
+ def __init__(self, path_entry):
+ # Lack of imp during class creation means _suffixes is set here.
+ # Make sure that Python source files are listed first! Needed for an
+ # optimization by the loader.
+ self._suffixes = suffix_list(imp.PY_SOURCE)
+ self._suffixes += suffix_list(imp.PY_COMPILED)
+ super(PyFileImporter, self).__init__(path_entry)
+
+
+class ImportLockContext(object):
+
+ """Context manager for the import lock."""
+
+ def __enter__(self):
+ """Acquire the import lock."""
+ imp.acquire_lock()
+
+ def __exit__(self, exc_type, exc_value, exc_traceback):
+ """Release the import lock regardless of any raised exceptions."""
+ imp.release_lock()
+
+
+class Import(object):
+
+ """Class that implements the __import__ interface.
+
+ Backwards compatibility is maintained by extending sys.meta_path
+ interally (for handling built-in and frozen modules) and providing a
+ default path hooks entry (for extension modules, .py, and .pyc
+ files). Both are controlled during instance initialization.
+
+ """
+
+ def __init__(self, default_path_hook=None,
+ extended_meta_path=None):
+ """Store a default path hook entry and a sequence to internally extend
+ sys.meta_path by (passing in None uses default importers)."""
+ if extended_meta_path is None:
+ self.extended_meta_path = BuiltinImporter(), FrozenImporter()
+ else:
+ self.extended_meta_path = extended_meta_path
+ self.default_path_hook = default_path_hook
+ if self.default_path_hook is None:
+ # Create a handler to deal with extension modules, .py, and .pyc
+ # files. Built-in and frozen modules are handled by sys.meta_path
+ # entries.
+ importers = [ExtensionFileImporter, PyFileImporter]
+ self.default_path_hook = chaining_fs_path_hook(*importers)
+
+ def _search_meta_path(self, name, path=None):
+ """Check the importers on sys.meta_path for a loader along with the
+ extended meta path sequence stored within this instance.
+
+ The extended sys.meta_path entries are searched after the entries on
+ sys.meta_path.
+
+ """
+ for entry in (tuple(sys.meta_path) + self.extended_meta_path):
+ loader = entry.find_module(name, path)
+ if loader:
+ return loader
+ else:
+ raise ImportError("No module named %s" % name)
+
+ def _sys_path_importer(self, path_entry):
+ """Return the importer for the specified path, from
+ sys.path_importer_cache if possible.
+
+ If None is stored in sys.path_importer_cache then use the default path
+ hook.
+
+ """
+ try:
+ # See if an importer is cached.
+ importer = sys.path_importer_cache[path_entry]
+ # If None was returned, use default importer factory.
+ if importer is None:
+ # XXX Would it break backwards-compatibility to set the importer
+ # in sys.path_importer_cache, replacing the None entry and not
+ # rely on None being there?
+ return self.default_path_hook(path_entry)
+ else:
+ return importer
+ except KeyError:
+ # No cached importer found; try to get a new one from
+ # sys.path_hooks.
+ for importer_factory in sys.path_hooks:
+ try:
+ importer = importer_factory(path_entry)
+ # XXX Going to break backwards-compatibility by storing
+ # an instance of the default importer? None still handled
+ # properly so shouldn't be any different than some other
+ # importer being stored.
+ sys.path_importer_cache[path_entry] = importer
+ return importer
+ except ImportError:
+ continue
+ else:
+ # No importer factory on sys.path_hooks works; use the default
+ # importer factory.
+ try:
+ importer = self.default_path_hook(path_entry)
+ sys.path_importer_cache[path_entry] = importer
+ return importer
+ except ImportError:
+ sys.path_importer_cache[path_entry] = None
+ raise ImportError("no importer found for %s" % path_entry)
+
+ def _search_std_path(self, name, path=None):
+ """Check sys.path or 'path' (depending if 'path' is set) for the
+ named module and return its loader."""
+ if path:
+ search_paths = path
+ else:
+ search_paths = sys.path
+ for entry in search_paths:
+ try:
+ importer = self._sys_path_importer(entry)
+ except ImportError:
+ continue
+ loader = importer.find_module(name)
+ if loader:
+ return loader
+ else:
+ raise ImportError("No module named %s" % name)
+
+ def module_from_cache(self, name):
+ """Try to return the named module from sys.modules.
+
+ Return False if the module is not in the cache.
+ """
+ if name in sys.modules:
+ return sys.modules[name]
+ else:
+ return False
+
+ def post_import(self, module):
+ """Perform any desired post-import processing on the module."""
+ return module
+
+ def _import_module(self, name, path=None):
+ """Import the specified module with no handling of parent modules.
+
+ If None is set for a value in sys.modules (to signify that a relative
+ import was attempted and failed) then ImportError is raised.
+
+ """
+ cached_module = self.module_from_cache(name)
+ if cached_module is not False:
+ if cached_module is None:
+ raise ImportError("relative import redirect")
+ else:
+ return cached_module
+ try:
+ # Attempt to find a loader on sys.meta_path.
+ loader = self._search_meta_path(name, path)
+ except ImportError:
+ # sys.meta_path search failed. Attempt to find a loader on
+ # sys.path. If this fails then module cannot be found.
+ loader = self._search_std_path(name, path)
+ # A loader was found. It is the loader's responsibility to have put an
+ # entry in sys.modules.
+ return self.post_import(loader.load_module(name))
+
+ def _import_full_module(self, name):
+ """Import a module along with its parent modules and set into
+ sys.modules."""
+ # Import the module (importing its parent modules first).
+ name_parts = name.split('.')
+ current_name_parts = []
+ parent_module = None
+ path_list = None
+ # XXX Any perk to being optimistic and assuming parent modules were
+ # imported already, and thus search in the reverse order for the first module
+ # to be found?
+ for name_part in name_parts:
+ current_name_parts.append(name_part)
+ # Recreating name every time around the loop is sub-optimal, but
+ # does away with base case of a non-dotted name.
+ current_name = ".".join(current_name_parts)
+ if parent_module:
+ try:
+ # XXX Better to use __path__ as found in 'globals' argument
+ # in __call__? Possibly different if module in sys.modules
+ # resolves to a different one from where the import call
+ # was made. Not sure if this worth worrying about, though,
+ # since relative imports could easily negate use of
+ # __path__ and sys.modules is totally reasonable to
+ # consider the canonical modules.
+ path_list = parent_module.__path__
+ except AttributeError:
+ pass
+ module = self._import_module(current_name, path_list)
+ if parent_module:
+ setattr(parent_module, name_part, module)
+ parent_module = module
+
+ def _classic_resolve_name(self, name, caller_name, caller_is_package):
+ """Return the absolute name of the module specified in 'name' as based
+ on the name of the caller and whether it is a package."""
+ if caller_is_package:
+ base_name = caller_name
+ else:
+ if '.' not in caller_name:
+ # Importing in a top-level module, so name is already absolute.
+ return name
+ else:
+ base_name = caller_name.rsplit('.', 1)[0]
+ return base_name + '.' + name
+
+ def _resolve_name(self, name, caller_name, caller_is_package, level):
+ """Return the absolute name of the module specified by 'name' based on
+ where the module is being imported and the requested change in dotted
+ name level.
+
+ Absolute imports always take the form of ``from <name> import ...``.
+ This means that the number of leading dots in '<name>' becomes 'level'
+ and the rest of '<name>' becomes 'name'. This leads to the possibility
+ that 'name' is the empty string, representing the current package.
+
+ """
+ if caller_is_package:
+ level -= 1
+ if caller_name.count('.') < level:
+ raise ImportError("attempted relative import beyond top-level "
+ "pacakge")
+ base_name = caller_name.rsplit('.', level)[0]
+ if name:
+ return base_name + '.' + name
+ else:
+ return base_name
+
+ def _return_module(self, absolute_name, relative_name, fromlist):
+ """Return the proper module based on what module was requested (and its
+ absolute module name), who is requesting it, and whether any specific
+ attributes were specified.
+
+ The semantics of this method revolve around 'fromlist'. When it is
+ empty, the module up to the first dot is to be returned. When the
+ module being requested is an absolute name this is simple (and
+ relative_name is an empty string). But if the requested module was
+ a relative import (as signaled by relative_name having a non-false
+ value), then the name up to the first dot in the relative name resolved
+ to an absolute name is to be returned.
+
+ When fromlist is not empty and the module being imported is a package,
+ then the values
+ in fromlist need to be checked for. If a value is not a pre-existing
+ attribute a relative import is attempted. If it fails then suppressed
+ the failure silently.
+
+ """
+ if not fromlist:
+ if relative_name:
+ absolute_base = absolute_name.rpartition(relative_name)[0]
+ relative_head = relative_name.split('.', 1)[0]
+ to_return = absolute_base + relative_head
+ else:
+ to_return = absolute_name.split('.', 1)[0]
+ return sys.modules[to_return]
+ # When fromlist is not empty, return the actual module specified in
+ # the import.
+ else:
+ module = sys.modules[absolute_name]
+ if hasattr(module, '__path__') and hasattr(module, '__name__'):
+ # When fromlist has a value and the imported module is a
+ # package, then if a name in fromlist is not found as an
+ # attribute on module, try a relative import to find it.
+ # Failure is fine and the exception is suppressed.
+ check_for = list(fromlist)
+ if '*' in check_for and hasattr(module, '__all__'):
+ check_for.extend(module.__all__)
+ for item in check_for:
+ if item == '*':
+ continue
+ if not hasattr(module, item):
+ resolved_name = self._resolve_name(item, module.__name__,
+ True, 1)
+ try:
+ self._import_full_module(resolved_name)
+ except ImportError:
+ pass
+ return module
+
+ def __call__(self, name, globals={}, locals={}, fromlist=[], level=-1):
+ """Import a module.
+
+ The 'name' argument is the name of the module to be imported (e.g.,
+ 'foo' in ``import foo`` or ``from foo import ...``).
+
+ 'globals' and 'locals' are the global and local namespace dictionaries
+ of the module where the import statement appears. 'globals' is used to
+ introspect the __path__ and __name__ attributes of the module making
+ the call. 'local's is ignored.
+
+ 'fromlist' lists any specific objects that are to eventually be put
+ into the namespace (e.g., ``from for.bar import baz`` would have 'baz'
+ in the fromlist, and this includes '*'). An entry of '*' will lead to
+ a check for __all__ being defined on the module. If it is defined then
+ the values in __all__ will be checked to make sure that all values are
+ attributes on the module, attempting a module import relative to 'name'
+ to set that attribute.
+
+ When 'name' is a dotted name, there are two different situations to
+ consider for the return value. One is when the fromlist is empty.
+ In this situation the import statement imports and returns the name up
+ to the first dot. All subsequent names are imported but set as
+ attributes as needed on parent modules. When fromlist is not empty
+ then the module represented by the full dotted name is returned.
+
+ 'level' represents how to handle possible relative imports. If 'level'
+ is set to -1 then the module name could be either relative or absolute.
+ A value of 0 is for absolute module names., Any positive value
+ represents the number of dots listed in the relative import statement
+ (e.g. has a value of 2 for ``from .. import foo``).
+
+ """
+ if not name and level < 1:
+ raise ValueError("Empty module name")
+ is_pkg = True if '__path__' in globals else False
+ caller_name = globals.get('__name__')
+ with ImportLockContext():
+ if level and caller_name:
+ # Handle the classic style of import: relative first, then
+ # absolute.
+ if level == -1:
+ relative_name = self._classic_resolve_name(name, caller_name,
+ is_pkg)
+ imported_name = relative_name
+ try:
+ # Try a relative import.
+ self._import_full_module(imported_name)
+ except ImportError:
+ # If the relative import fails (or is redirected), try an
+ # absolute import.
+ imported_name = name
+ self._import_full_module(imported_name)
+ # Redirection entry for resolved relative name to instead
+ # redirect to the absolute import.
+ sys.modules[relative_name] = None
+ # If using absolute imports with a relative path, only attempt with
+ # the fully-resolved module name.
+ else:
+ imported_name = self._resolve_name(name, caller_name, is_pkg,
+ level)
+ # This call will also handle setting the attribute on the
+ # package.
+ self._import_full_module(imported_name)
+ # Absolute module import of a top-level module.
+ else:
+ imported_name = name
+ self._import_full_module(name)
+ relative_name = '' if imported_name == name else name
+ return self._return_module(imported_name, relative_name, fromlist)
+
+# XXX Eventually replace with a proper __all__ value (i.e., don't expose os
+# replacements but do expose _ExtensionFileLoader, etc. for testing).
+__all__ = list(globals().keys())
Added: sandbox/trunk/import_in_py/Py3K/importlib.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/importlib.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,139 @@
+"""Implementation of Python's import machinery in Python source code.
+
+The Import class implements the semantics of import. This means that an
+instance of this class can be called to properly check sys.modules,
+sys.meta_path, sys.path, etc. to import the requested module.
+
+The importing of built-in, frozen, extension, .py, and .pyc files are all
+handled by implementing importers and loaders as specified by PEP 302. This
+leads to the ability to easily control imports based on the type of module.
+There is also a clear distinction between how the module is stored and how it
+is formatted (e.g., a Python source file does not require it be stored as a
+file on a filesystem).
+
+To help with the distinction between how a module is stored compared to its
+format the idea of handlers is introduced. A handler implements a specific
+interface while specifying the type of module it handles (usually by the what
+one would expect as a file extension if the module was on a filesystem). This
+allows a PEP 302 importer/loader to represent how a module is stored while the
+handler deals with how the module is formatted.
+
+A handler is expected to implement the handle_code method only. The handler
+for Python source and bytecode modules goes farther and defines an extensive
+interface that is designed so that alternatives on module formatting can be
+supported easily without needing to write a new importer/loader. The Python
+source and bytecode handler also expects loaders to define a more extensive
+interface to allow for different backend stores (e.g., databases) to use the
+handler without modification. All of this helps with code reuse and possible
+errors from the complicated relationship between Python source and bytecode
+modules.
+"""
+import _importlib
+
+# XXX Temporary functions that should eventually be removed.
+import os
+
+def _set__import__():
+ """Set __import__ to an instance of Import."""
+ global original__import__
+ original__import__ = __import__
+ __builtins__['__import__'] = Import()
+
+
+def _reset__import__():
+ """Set __import__ back to the original implementation (assumes
+ _set__import__ was called previously)."""
+ __builtins__['__import__'] = original__import__
+
+
+def _w_long(x):
+ """Convert a 32-bit integer to little-endian.
+
+ XXX Temporary until marshal's long functions are exposed.
+
+ """
+ x = int(x)
+ bytes = []
+ bytes.append(x & 0xFF)
+ bytes.append((x >> 8) & 0xFF)
+ bytes.append((x >> 16) & 0xFF)
+ bytes.append((x >> 24) & 0xFF)
+ return ''.join(chr(x) for x in bytes)
+
+
+def _r_long(bytes):
+ """Convert 4 bytes in little-endian to an integer.
+
+ XXX Temporary until marshal's long function are exposed.
+
+ """
+ x = ord(bytes[0])
+ x |= ord(bytes[1]) << 8
+ x |= ord(bytes[2]) << 16
+ x |= ord(bytes[3]) << 24
+ return x
+
+
+def _case_ok(directory, file_name):
+ """Verify that file_name (as found in 'directory') has the proper case.
+
+ The path is assumed to already exist.
+
+ XXX Temporary until imp's case_ok function can be exposed.
+
+ XXX Better to roll this into a single function some how so that existence
+ check can be part of case check and thus cut down on stat calls?
+
+ """
+ # If platform is not case-sensitive *or* the environment variable
+ # PYTHONCASEOK is defined, then os.path.exists already handled the case by
+ # either doing a case-sensitive check or from the user saying he does not
+ # want case-sensitivity, respectively.
+ if sys.platform not in ('win32', 'mac', 'riscos', 'darwin', 'cygwin',
+ 'os2emx') or os.environ.get('PYTHONCASEOK'):
+ return True
+ directory_contents = os.listdir(directory)
+ if file_name in directory_contents:
+ return True
+ else:
+ return False
+
+
+
+# Required built-in modules.
+try:
+ import posix as _os
+except ImportError:
+ try:
+ import nt as _os
+ except ImportError:
+ try:
+ import os2 as _os
+ except ImportError:
+ raise ImportError('posix, nt, or os2 module required for importlib')
+_importlib._os = _os
+import imp, sys, marshal
+_importlib.imp = imp
+_importlib.sys = sys
+_importlib.marshal = marshal
+
+
+# XXX These all need to either go away or become built-in modules
+# (<cough>Neal</cough>).
+from errno import EACCES
+from os import sep
+import warnings
+
+_importlib._r_long = _r_long #XXX Expose original from marshal.
+_importlib._w_long = _w_long #XXX Expose original from marshal.
+_importlib._case_ok = _case_ok #XXX Expose original from imp.
+# For os.path.join replacement; pull from Include/osdefs.h:SEP .
+_importlib.path_sep = sep
+# For allowing silent failure of .pyc creation when permission is denied.
+_importlib.EACCES = EACCES
+
+_importlib.warnings = warnings
+
+del _importlib
+
+from _importlib import *
Added: sandbox/trunk/import_in_py/Py3K/tests/__init__.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/__init__.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,14 @@
+if __name__ == '__main__':
+ import os
+ import os.path
+
+ abs_test_directory = os.path.split(__file__)[0]
+ rel_test_directory = os.path.split(abs_test_directory)[1]
+ for filename in os.listdir(abs_test_directory):
+ if not filename.startswith('test_') or not filename.endswith('.py'):
+ continue
+ full_filename = os.path.join(abs_test_directory, filename)
+ module_name = os.path.splitext(filename)[0]
+ module = __import__(rel_test_directory + '.' + module_name)
+ getattr(module, module_name).test_main()
+ print('~' * 20)
Added: sandbox/trunk/import_in_py/Py3K/tests/builtin_frozen_help.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/builtin_frozen_help.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,67 @@
+import sys
+import unittest
+
+class BuiltinFrozen_Tester(unittest.TestCase):
+
+ """Common test cases for built-in and frozen modules.
+
+ Expected attributes:
+ * self.importer
+ The meta_path importer to test.
+ * self.module_name
+ Name of a module that the importer can import.
+ * self.bad_module_names
+ Sequence of module names that cannot be imported.
+
+ """
+
+ def test_find_module_basic(self):
+ # Make sure expected modules can be found.
+ self.failUnless(self.importer.find_module(self.module_name))
+
+ def test_find_module_failures(self):
+ # Modules of the wrong type should not be found.
+ for module_name in self.bad_module_names:
+ self.failUnlessEqual(self.importer.find_module(module_name), None)
+
+ def test_find_module_arg_count(self):
+ # Cover proper number of arguments.
+ self.failUnlessRaises(TypeError, self.importer.find_module)
+ self.failUnlessRaises(TypeError, self.importer.find_module,
+ self.module_name, None, None)
+
+ def test_load_module_prev_import_check(self):
+ # If a module is already in sys.modules it should be returned without
+ # being re-initialized.
+ self.failUnlessEqual(self.importer.load_module(self.module_name),
+ sys.modules[self.module_name])
+
+ def test_load_module_new_import(self):
+ # Make sure importing of a module that was not done before works
+ # properly.
+ # Do not forget to put any removed module back into sys.modules!
+ # Certain modules like 'sys' have some attributes that are set
+ # only once during interpreter initialization and are never set
+ # again afterwards.
+ mod = None
+ try:
+ if self.module_name in sys.modules:
+ mod = sys.modules[self.module_name]
+ del sys.modules[self.module_name]
+ loaded_module = self.importer.load_module(self.module_name)
+ self.failUnlessEqual(self.module_name, loaded_module.__name__)
+ finally:
+ if mod:
+ sys.modules[self.module_name] = mod
+
+ def test_load_module_ImportError(self):
+ self.failUnlessRaises(ImportError,
+ self.importer.load_module,
+ self.bad_module_names[0])
+
+ def test_load_module_arg_count(self):
+ self.failUnlessRaises(TypeError, self.importer.load_module)
+ self.failUnlessRaises(TypeError, self.importer.load_module,
+ self.module_name, None, None)
+
+
Added: sandbox/trunk/import_in_py/Py3K/tests/ext_help.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/ext_help.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,21 @@
+import imp
+import os
+import sys
+
+
+def find_ext_location(name):
+ possible_suffixes = [suffix[0] for suffix in imp.get_suffixes()
+ if suffix[2] == imp.C_EXTENSION]
+ path_entry = None
+ filename = None
+ try:
+ for path in sys.path:
+ for suffix in possible_suffixes:
+ filename = name + suffix
+ if os.path.exists(os.path.join(path, name + suffix)):
+ path_entry = path
+ raise StopIteration
+ except StopIteration:
+ return (path_entry, filename)
+ else:
+ raise ValueError('%s could not be found on sys.path' % name)
Added: sandbox/trunk/import_in_py/Py3K/tests/mock_importlib.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/mock_importlib.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,100 @@
+import sys
+import marshal
+import imp
+from test import test_support
+from importlib import _w_long
+
+
+def log_call(method):
+ """Log method calls to self.log."""
+ def log_and_call(self, *args, **kwargs):
+ self.log.append(method.__name__)
+ return method(self, *args, **kwargs)
+ return log_and_call
+
+
+class MockModule(object):
+
+ """A mock module."""
+
+ def __init__(self, name=None, file_path=None, pkg_list=None, __all__=None):
+ if name is not None:
+ self.__name__ = name
+ if file_path is not None:
+ self.__file__ = file_path
+ if pkg_list is not None:
+ self.__path__ = pkg_list
+ if __all__ is not None:
+ self.__all__ = __all__
+
+
+# Mock Importers (with optional path_hooks support).
+
+class ErrorImporter(object):
+
+ """Mock importer to have a guaranteed error point."""
+
+ def find_module(self, fullname, path=None):
+ self.find_request = fullname, path
+ raise ImportError
+
+ @classmethod
+ def set_on_sys_path(cls):
+ error_entry = '<error>'
+ sys.path.append(error_entry)
+ ins = cls()
+ sys.path_importer_cache[error_entry] = ins
+ return ins
+
+
+class PassImporter(object):
+
+ """Mock importer that always pass on importing a module."""
+
+ def __call__(self, path_entry):
+ """Always pass when asked to create an importer."""
+ raise ImportError
+
+ def find_module(self, fullname, path=None):
+ self.find_request = fullname, path
+ return None
+
+ @classmethod
+ def set_on_sys_path(cls):
+ pass_entry = '<pass>'
+ sys.path.append(pass_entry)
+ ins = cls()
+ sys.path_importer_cache[pass_entry] = ins
+ return ins
+
+
+class SucceedImporter(object):
+
+ """Mock importer that always succeed by returning 'self'."""
+
+ def __init__(self):
+ self.path_entries = []
+ self.loaded_modules = []
+
+ def __call__(self, path_entry):
+ self.path_entries.append(path_entry)
+ return self
+
+ def find_module(self, fullname, path=None):
+ self.find_request = fullname, path
+ return self
+
+ def load_module(self, fullname):
+ self.load_request = fullname
+ module = MockModule(fullname, '<succeed importer>')
+ self.loaded_modules.append(module)
+ sys.modules[fullname] = module
+ return module
+
+ @classmethod
+ def set_on_sys_path(cls):
+ succeed_entry = '<success>'
+ sys.path.append(succeed_entry)
+ ins = cls()
+ sys.path_importer_cache[succeed_entry] = ins
+ return ins
Added: sandbox/trunk/import_in_py/Py3K/tests/py_help.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/py_help.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,180 @@
+from __future__ import with_statement
+
+import marshal
+import os.path
+import py_compile
+import shutil
+import sys
+import tempfile
+import unittest
+
+
+class TestPyPycFiles(unittest.TestCase):
+
+ """Base class to help in generating a fresh source and bytecode file.
+
+ Structure of created files:
+ * directory/
+ + py_path [module_name + py_ext]
+ + pyc_path [module_name +pyc_ext]
+
+ """
+
+ def setUp(self, faked_names=True):
+ """Generate the path to a temporary file to test with.
+
+ If faked_names is true then all names are non-standard compared to
+ normal Python files.
+
+ """
+ # Set needed info for file paths.
+ if faked_names:
+ self.module_name = '<test_module>'
+ self.py_ext = '.source'
+ self.pyc_ext = '.bytecode'
+ else:
+ self.module_name = '_test_module'
+ self.py_ext = '.py'
+ self.pyc_ext = '.pyc' if __debug__ else '.pyo'
+ try:
+ del sys.modules[self.module_name]
+ except KeyError:
+ pass
+ self.directory = tempfile.gettempdir()
+ self.test_attr = ('test_attr', None)
+ self.py_path = os.path.join(self.directory,
+ self.module_name+self.py_ext)
+ self.pyc_path = os.path.join(self.directory,
+ self.module_name+self.pyc_ext)
+ # Create source file.
+ self.source = '%s = %r' % self.test_attr
+ with open(self.py_path, 'w') as py_file:
+ py_file.write(self.source)
+ # Create bytecode file.
+ py_compile.compile(self.py_path, self.pyc_path, 'exec')
+ code = compile(self.source, self.pyc_path, 'exec')
+ self.bytecode = marshal.dumps(code)
+ sys.path.insert(0, self.directory)
+
+ def tearDown(self):
+ """If the temporary path was used, make sure to clean up."""
+ if self.directory in sys.path:
+ sys.path.remove(self.directory)
+ if os.path.exists(self.py_path):
+ os.remove(self.py_path)
+ if os.path.exists(self.pyc_path):
+ os.remove(self.pyc_path)
+
+ def verify_module(self, module, file_path=None):
+ """Verify that the module is correct."""
+ if file_path:
+ self.failUnlessEqual(module.__name__, self.module_name)
+ self.failUnlessEqual(module.__file__, file_path)
+ self.failUnless(hasattr(module, self.test_attr[0]))
+ self.failUnlessEqual(getattr(module, self.test_attr[0]),
+ self.test_attr[1])
+
+
+class TestPyPycPackages(TestPyPycFiles):
+
+ """Create a testing package.
+
+ Structure of created files (on top of files created in superclasses):
+ * self.directory/
+ + top_level_module_path [top_level_module_name + py_ext]
+ + pkg_path/ [pkg_name]
+ - pkg_init_path [ '__init__' + py_ext]
+ - pkg_module_path [module_name + py_ext]
+ - sub_pkg_path/ [sub_pkg_name]
+ * sub_pkg_init_path ['__init__' + py_ext]
+ * sub_pkg_module_path [module_name + py_ext]
+
+ """
+
+ def setUp(self, faked_names=True):
+ TestPyPycFiles.setUp(self, faked_names)
+ self.top_level_module_name = 'top_level_' + self.module_name
+ self.top_level_module_path = os.path.join(self.directory,
+ self.top_level_module_name+self.py_ext)
+ with open(self.top_level_module_path, 'w') as top_level_file:
+ top_level_file.write(self.source)
+ if faked_names:
+ self.pkg_name = '<test_pkg>'
+ else:
+ self.pkg_name = '_test_pkg'
+ self.pkg_path = os.path.join(self.directory, self.pkg_name)
+ try:
+ os.mkdir(self.pkg_path)
+ except OSError:
+ self.tearDown()
+ os.mkdir(self.pkg_path)
+ self.pkg_init_path = os.path.join(self.pkg_path,
+ '__init__'+self.py_ext)
+ self.pkg_init_pyc_path = os.path.join(self.pkg_path,
+ '__init__'+self.pyc_ext)
+ with open(self.pkg_init_path, 'w') as pkg_file:
+ pkg_file.write(self.source)
+ self.pkg_module_name = '.'.join([self.pkg_name, self.module_name])
+ self.pkg_module_path = os.path.join(self.pkg_path,
+ self.module_name+self.py_ext)
+ with open(self.pkg_module_path, 'w') as module_file:
+ module_file.write(self.source)
+ self.sub_pkg_tail_name = 'sub_pkg'
+ self.sub_pkg_name = '.'.join([self.pkg_name, self.sub_pkg_tail_name])
+ self.sub_pkg_path = os.path.join(self.pkg_path, self.sub_pkg_tail_name)
+ os.mkdir(self.sub_pkg_path)
+ self.sub_pkg_init_path = os.path.join(self.sub_pkg_path,
+ '__init__'+self.py_ext)
+ with open(self.sub_pkg_init_path, 'w') as subpkg_file:
+ subpkg_file.write(self.source)
+ self.sub_pkg_module_name = '.'.join([self.sub_pkg_name,
+ self.module_name])
+ self.sub_pkg_module_path = os.path.join(self.sub_pkg_path,
+ self.module_name+self.py_ext)
+ with open(self.sub_pkg_module_path, 'w') as submodule_file:
+ submodule_file.write(self.source)
+
+ def tearDown(self):
+ TestPyPycFiles.tearDown(self)
+ os.remove(self.top_level_module_path)
+ pyc_path = (os.path.splitext(self.top_level_module_path)[0] +
+ self.pyc_ext)
+ if os.path.exists(pyc_path):
+ os.remove(pyc_path)
+ shutil.rmtree(self.pkg_path)
+
+ def verify_package(self, module, actual_name=None):
+ self.failUnless(hasattr(module, self.test_attr[0]))
+ self.failUnlessEqual(getattr(module, self.test_attr[0]),
+ self.test_attr[1])
+ if module.__name__ == self.pkg_name:
+ self.failUnless(module.__file__ in
+ [self.pkg_init_path, self.pkg_init_pyc_path])
+ self.failUnlessEqual(module.__path__, [self.pkg_path])
+ # Module in top-level package.
+ if actual_name and self.pkg_module_name in actual_name:
+ self.failUnless(hasattr(module, self.module_name))
+ sub_module = getattr(module, self.module_name)
+ self.failUnlessEqual(sub_module.__name__, self.pkg_module_name)
+ self.failUnlessEqual(sub_module.__file__, self.pkg_module_path)
+ self.verify_module(sub_module)
+ # Package within top-level package.
+ if actual_name and self.sub_pkg_name in actual_name:
+ self.failUnless(hasattr(module, self.sub_pkg_tail_name))
+ sub_pkg = getattr(module, self.sub_pkg_tail_name)
+ self.failUnlessEqual(sub_pkg.__name__, self.sub_pkg_name)
+ self.failUnlessEqual(sub_pkg.__file__, self.sub_pkg_init_path)
+ self.failUnlessEqual(sub_pkg.__path__, [self.sub_pkg_path])
+ self.verify_module(sub_pkg)
+ if actual_name == self.sub_pkg_module_name:
+ self.failUnless(hasattr(sub_pkg, self.module_name))
+ sub_module = getattr(sub_pkg, self.module_name)
+ self.failUnlessEqual(sub_module.__name__,
+ self.sub_pkg_module_name)
+ self.failUnlessEqual(sub_module.__file__,
+ self.sub_pkg_module_path)
+ self.verify_module(sub_module)
+ if module.__name__ == self.pkg_module_name:
+ self.failUnlessEqual(module.__file__, self.pkg_module_path)
+
+
Added: sandbox/trunk/import_in_py/Py3K/tests/test_builtin_importer.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/test_builtin_importer.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,22 @@
+import importlib
+
+from tests.builtin_frozen_help import BuiltinFrozen_Tester
+
+from test.test_support import run_unittest
+
+class BuiltinImporterTests(BuiltinFrozen_Tester):
+
+ """Test the built-in module importer."""
+
+ importer = importlib.BuiltinImporter()
+ module_name = 'sys'
+ bad_module_names = ('tokenize', 'time', '__hello__')
+
+
+def test_main():
+ run_unittest(BuiltinImporterTests)
+
+
+
+if __name__ == '__main__':
+ test_main()
Added: sandbox/trunk/import_in_py/Py3K/tests/test_chained_importer.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/test_chained_importer.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,126 @@
+import importlib
+from tests import mock_importlib
+
+import os
+from test import test_support
+import unittest
+
+
+class ChainingPathHook(unittest.TestCase):
+
+ """Tests for importlib.chaining_fs_path_hook."""
+
+ def test_single_hook(self):
+ # A single hook argument should work as expected.
+ # Also tests acceptance of a relative path.
+ hook = importlib.chaining_fs_path_hook(mock_importlib.SucceedImporter())
+ self.assert_(hook('.'))
+ hook = importlib.chaining_fs_path_hook(mock_importlib.PassImporter())
+ self.assertRaises(ImportError, hook, '.')
+
+ def test_path_acceptance(self):
+ # Valid paths should be accepted, but non-existent paths should be
+ # rejected outright.
+ succeed_hook = mock_importlib.SucceedImporter()
+ hook = importlib.chaining_fs_path_hook(succeed_hook)
+ self.assert_(hook('.'))
+ self.assertRaises(ImportError, hook, '_nonexistentdirectory')
+
+ def test_multiple_hooks(self):
+ # Multiple hooks should work.
+ succeed_hook = mock_importlib.SucceedImporter()
+ pass_hook = mock_importlib.PassImporter()
+ hook = importlib.chaining_fs_path_hook(pass_hook, succeed_hook)
+ self.assert_(hook('.'))
+ hook = importlib.chaining_fs_path_hook(succeed_hook, pass_hook)
+ self.assert_(hook('.'))
+ hook = importlib.chaining_fs_path_hook(pass_hook, pass_hook)
+ self.assertRaises(ImportError, hook, '.')
+
+ def test_order_preservation(self):
+ # The order of importers to call should be maintained.
+ succeed1 = mock_importlib.SucceedImporter()
+ succeed2 = mock_importlib.SucceedImporter()
+ hook = importlib.chaining_fs_path_hook(succeed1, succeed2)
+ importer = hook('.')
+ self.assert_(importer._importers[0] is succeed1)
+ self.assert_(importer._importers[1] is succeed2)
+
+ def test_absolute_path(self):
+ # The path hooks that are called should be passed an absolute path.
+ succeed = mock_importlib.SucceedImporter()
+ hook = importlib.chaining_fs_path_hook(succeed)
+ hook('.')
+ absolute_path = os.path.abspath('.')
+ self.assertEquals(os.path.normpath(succeed.path_entries[0]), absolute_path)
+
+ def test_reject_file(self):
+ # A file should not be accepted.
+ succeed = mock_importlib.SucceedImporter()
+ hook = importlib.chaining_fs_path_hook(succeed)
+ with open(test_support.TESTFN, 'w') as test_file:
+ test_file.write('# For testing chained path hook.')
+ try:
+ self.assertRaises(ImportError, hook, test_support.TESTFN)
+ finally:
+ test_support.unlink(test_support.TESTFN)
+
+ def test_reject_bad_dir(self):
+ # A non-existent directory should be rejected.
+ succeed = mock_importlib.SucceedImporter()
+ hook = importlib.chaining_fs_path_hook(succeed)
+ self.assertRaises(ImportError, hook, '_asdfasdfasdf')
+
+
+class ChainedImporterTests(unittest.TestCase):
+
+ """Tests for importlib.ChainedImporter."""
+
+ def test_single_importer(self):
+ # There should be no issue with a single importer.
+ succeed = mock_importlib.SucceedImporter()
+ importer = importlib.ChainedImporter(succeed)
+ self.assert_(importer.find_module('blah') is succeed)
+ self.assertEqual(succeed.find_request[0], 'blah')
+ pass_ = mock_importlib.PassImporter()
+ importer = importlib.ChainedImporter(pass_)
+ self.assert_(importer.find_module(pass_) is None)
+
+ def test_multiple_importers(self):
+ # Test that multiple importers work.
+ succeed = mock_importlib.SucceedImporter()
+ pass_ = mock_importlib.PassImporter()
+ importer = importlib.ChainedImporter(succeed, pass_)
+ self.assert_(importer.find_module('blah') is succeed)
+ self.assert_(succeed.find_request[0], 'blah')
+ importer = importlib.ChainedImporter(pass_, succeed)
+ self.assert_(importer.find_module('blah') is succeed)
+ self.assert_(succeed.find_request[0], 'blah')
+
+ def test_order(self):
+ # Make sure that the importers are called in the proper order.
+ succeed = mock_importlib.SucceedImporter()
+ pass_ = mock_importlib.PassImporter()
+ assert not hasattr(pass_, 'find_request')
+ assert not hasattr(succeed, 'find_request')
+ importer = importlib.ChainedImporter(pass_, succeed)
+ importer.find_module('blah')
+ self.assert_(hasattr(pass_, 'find_request'))
+ self.assertEqual(pass_.find_request[0], 'blah')
+ self.assert_(hasattr(succeed, 'find_request'))
+ self.assertEqual(succeed.find_request[0], 'blah')
+ succeed = mock_importlib.SucceedImporter()
+ succeed2 = mock_importlib.SucceedImporter()
+ importer = importlib.ChainedImporter(succeed, succeed2)
+ importer.find_module('blah')
+ self.assert_(hasattr(succeed, 'find_request'))
+ self.assertEqual(succeed.find_request[0], 'blah')
+ self.assert_(not hasattr(succeed2, 'find_request'))
+
+
+def test_main():
+ test_support.run_unittest(ChainingPathHook, ChainedImporterTests)
+
+
+if __name__ == '__main__':
+ test_main()
Added: sandbox/trunk/import_in_py/Py3K/tests/test_frozen_importer.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/test_frozen_importer.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,61 @@
+import importlib
+
+from tests.builtin_frozen_help import BuiltinFrozen_Tester
+
+import io
+import sys
+from test.test_support import run_unittest
+
+
+class FrozenImporterTests(BuiltinFrozen_Tester):
+
+ """Test the frozen module importer.
+
+ Python include the __hello__ and __phello__ frozen modules (as defined in
+ Python/frozen.c). Both do output to the screen, though, which should be
+ suppressed during testing.
+
+ """
+
+ importer = importlib.FrozenImporter()
+ module_name = '__hello__'
+ bad_module_names = ('tokenize', 'time', 'sys')
+
+ def setUp(self):
+ """Importing the testing frozen modules outputs to stdout. Redirect
+ stdout to block that."""
+ self._orig_stdout = sys.stdout
+ sys.stdout = io.StringIO()
+
+ def tearDown(self):
+ """Reset stdout to what it is by default."""
+ sys.stdout = self._orig_stdout
+
+
+class FrozenPackageInitTests(FrozenImporterTests):
+
+ """Tests the frozen module import for package __init__ modules."""
+
+ module_name = '__phello__'
+
+
+class FrozenPackageModuleTests(FrozenImporterTests):
+
+ """Tests the frozen module import for modules in packages."""
+
+ module_name = '__phello__.spam'
+
+class FrozenPackageInitTests(FrozenImporterTests):
+
+ """Tests the frozen module import for package __init__ modules."""
+
+ module_name = '__phello__'
+
+
+def test_main():
+ run_unittest(FrozenImporterTests, FrozenPackageInitTests, FrozenPackageModuleTests)
+
+
+
+if __name__ == '__main__':
+ test_main()
Added: sandbox/trunk/import_in_py/Py3K/tests/test_fs_importer.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/test_fs_importer.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,210 @@
+from __future__ import with_statement
+
+import importlib
+
+from tests import mock_importlib
+from tests.ext_help import find_ext_location
+from tests.py_help import TestPyPycPackages
+
+import imp
+import os
+import os.path
+import py_compile
+import sys
+import tempfile
+from test import test_support
+import unittest
+import warnings
+
+
+class PyFileImporterTests(TestPyPycPackages):
+
+ """Test the PyFileImporterTests (and thus also FileImporter).
+
+ No need to check for searching any deeper than a package (e.g., don't need
+ to look for a sub-module or sub-package) since the import machinery will
+ always have an importer for the containing package directory.
+
+ """
+
+ def setUp(self):
+ TestPyPycPackages.setUp(self, faked_names=False)
+ self.importer = importlib.PyFileImporter(self.directory)
+ self.importer._loader = lambda name, path, is_pkg: path
+
+ def tearDown(self):
+ TestPyPycPackages.tearDown(self)
+
+ def test_py_top_level(self):
+ # A top-level source module should work.
+ test_support.unlink(self.pyc_path)
+ found = self.importer.find_module(self.module_name)
+ self.assertEqual(found, self.py_path)
+
+ def test_failure(self):
+ os.unlink(self.py_path)
+ os.unlink(self.pyc_path)
+ found = self.importer.find_module(self.module_name)
+ self.assert_(found is None)
+
+ def test_pyc_top_level(self):
+ # A top-level bytecode module should work.
+ test_support.unlink(self.py_path)
+ found = self.importer.find_module(self.module_name)
+ self.assertEqual(found, self.pyc_path)
+
+ def test_py_package(self):
+ # A Python source package should be found.
+ # TestPyPycPackages, by default, does not compile the __init__ file.
+ if self.pkg_init_path[-1] not in ('c', 'o'):
+ assert not os.path.exists(self.pkg_init_path + 'c')
+ assert not os.path.exists(self.pkg_init_path + 'o')
+ else:
+ assert False
+ found = self.importer.find_module(self.pkg_name)
+ self.assertEqual(found, self.pkg_init_path)
+
+ def test_pyc_package(self):
+ # A bytecode package should be found.
+ py_compile.compile(self.pkg_init_path)
+ test_support.unlink(self.pkg_init_path)
+ expected = self.pkg_init_path + ('c' if __debug__ else 'o')
+ found = self.importer.find_module(self.pkg_name)
+ self.assertEqual(found, expected)
+
+ def test_file_type_order(self):
+ # The order of the search should be preserved with source files being
+ # found first.
+ assert os.path.exists(self.py_path)
+ assert self.py_ext in self.importer._suffixes
+ assert os.path.exists(self.pyc_path)
+ assert self.pyc_ext in self.importer._suffixes
+ py_suffixes = importlib.suffix_list(imp.PY_SOURCE)
+ found = self.importer.find_module(self.module_name)
+ for suffix in py_suffixes:
+ if found.endswith(suffix):
+ break
+ else:
+ self.fail("Python source files not searched for before bytecode "
+ "files")
+
+ def test_missing__init__warning(self):
+ # An ImportWarning should be raised if a directory matches a module
+ # name but no __init__ file exists.
+ test_support.unlink(self.pkg_init_path)
+ with test_support.catch_warning() as w:
+ warnings.simplefilter('always', ImportWarning)
+ found = self.importer.find_module(self.pkg_name)
+ self.assert_(found is None)
+ self.assert_(issubclass(w.category, ImportWarning))
+ self.assert_(str(w.message).endswith("missing __init__"))
+
+ def test_package_before_module(self):
+ # A package should always be found before a module with the same name.
+ # This should not vary based on whether one is source and another is
+ # bytecode, etc.
+ module_path = os.path.join(self.directory, self.pkg_name + self.py_ext)
+ with open(module_path, 'w') as module_file:
+ module_file.write('# Testing packcage over module import.')
+ try:
+ # Source/source.
+ found = self.importer.find_module(self.pkg_name)
+ self.assert_('__init__' in found)
+ # Source/bytecode.
+ py_compile.compile(self.pkg_init_path)
+ os.unlink(self.pkg_init_path)
+ found = self.importer.find_module(self.pkg_name)
+ self.assert_('__init__' in found)
+ # Bytecode/bytecode.
+ py_compile.compile(module_path)
+ os.unlink(module_path)
+ found = self.importer.find_module(self.pkg_name)
+ self.assert_('__init__' in found)
+ # Bytecode/source.
+ # tearDown will remove the package __init__ file.
+ with open(self.pkg_init_path, 'w') as pkg_file:
+ pkg_file.write('# testing package/module import preference.')
+ os.unlink(self.pkg_init_path + ('c' if __debug__ else 'o'))
+ found = self.importer.find_module(self.pkg_name)
+ self.assert_('__init__' in found)
+ finally:
+ test_support.unlink(module_path)
+ test_support.unlink(os.path.join(self.directory,
+ self.pkg_name + self.pyc_ext))
+
+ def test_module_case_sensitivity(self):
+ # Case-sensitivity should always matter as long as PYTHONCASEOK is not
+ # set.
+ name_len = len(self.top_level_module_name)
+ bad_case_name = (self.top_level_module_name[:name_len/2].upper() +
+ self.top_level_module_name[name_len/2:].lower())
+ env_guard = test_support.EnvironmentVarGuard()
+ env_guard.unset('PYTHONCASEOK')
+ with env_guard:
+ self.failUnless(not self.importer.find_module(bad_case_name))
+ if sys.platform not in ('win32', 'mac', 'darwin', 'cygwin', 'os2emx',
+ 'riscos'):
+ return
+ env_guard = test_support.EnvironmentVarGuard()
+ env_guard.set('PYTHONCASEOK', '1')
+ with env_guard:
+ assert os.environ['PYTHONCASEOK']
+ self.failUnless(self.importer.find_module(bad_case_name))
+
+ def test_package_case_sensitivity(self):
+ # Case-sensitivity should always matter as long as PYTHONCASEOK is not
+ # set.
+ name_len = len(self.pkg_name)
+ bad_case_name = (self.pkg_name[:name_len/2].upper() +
+ self.pkg_name[name_len/2:].lower())
+ bad_init_name = os.path.join(self.directory, self.pkg_name,
+ '__INit__.py')
+ env_guard = test_support.EnvironmentVarGuard()
+ env_guard.unset('PYTHONCASEOK')
+ with env_guard:
+ self.failUnless(not self.importer.find_module(bad_case_name))
+ os.unlink(self.pkg_init_path)
+ with open(bad_init_name, 'w') as init_file:
+ init_file.write('# Test case-sensitivity of imports.')
+ self.failUnless(not self.importer.find_module(self.pkg_name))
+ if sys.platform not in ('win32', 'mac', 'darwin', 'cygwin', 'os2emx',
+ 'riscos'):
+ return
+ os.unlink(bad_init_name)
+ with open(self.pkg_init_path, 'w') as init_file:
+ init_file.write('# Used for testing import.')
+ env_guard = test_support.EnvironmentVarGuard()
+ env_guard.set('PYTHONCASEOK', '1')
+ with env_guard:
+ assert os.environ['PYTHONCASEOK']
+ self.failUnless(self.importer.find_module(bad_case_name))
+ with open(bad_init_name, 'w') as init_file:
+ init_file.write('# Used to test case-insensitivity of import.')
+ self.failUnless(self.importer.find_module(self.pkg_name))
+
+
+class ExtensionFileImporterTests(unittest.TestCase):
+
+ def test_basic(self):
+ # Finding an extension module should work.
+ search_for = 'datetime'
+ extension_dir, filename = find_ext_location(search_for)
+ importer = importlib.ExtensionFileImporter(extension_dir)
+ importer._loader = lambda name, path, is_pkg: path
+ found = importer.find_module(search_for)
+ self.assert_(found is not None)
+ self.assert_(search_for in found)
+
+ def test_failure(self):
+ search_for = 'asdfsdfasdffd'
+ importer = importlib.ExtensionFileImporter('.')
+ found = importer.find_module(search_for)
+ self.assert_(found is None)
+
+
+def test_main():
+ test_support.run_unittest(PyFileImporterTests, ExtensionFileImporterTests)
+
+
+if __name__ == '__main__':
+ test_main()
Added: sandbox/trunk/import_in_py/Py3K/tests/test_fs_loader.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/test_fs_loader.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,403 @@
+import importlib
+
+from tests import mock_importlib
+from tests.ext_help import find_ext_location
+from tests.py_help import TestPyPycPackages
+
+import imp
+import os
+import py_compile
+import sys
+from test import test_support
+import unittest
+
+
+class LoaderBasics(unittest.TestCase):
+
+ def reload_test(self, loader, name):
+ # If a module already exists in sys.modules then it should be reused
+ # and be re-initialized.
+ if name not in sys.modules:
+ loader.load_module(name)
+ cached = sys.modules[name]
+ loaded = loader.load_module(name)
+ self.assert_(cached is loaded)
+
+ def basic_test(self, loader, name, path):
+ # A module, after being loaded, should appear in sys.modules.
+ if name in sys.modules:
+ del sys.modules[name]
+ loaded = loader.load_module(name)
+ self.assert_(loaded is sys.modules[name])
+ self.assertEqual(loaded.__name__, name)
+ self.assert_(loaded.__file__.startswith(path),
+ "%s does not start with %s" % (loaded.__file__, path))
+ self.assertEqual(loaded.__loader__, loader)
+
+ def ImportError_on_bad_name(self, loader, bad_name, extra_methods=[]):
+ to_test = ['load_module', 'is_package', 'get_code', 'get_source']
+ to_test += extra_methods
+ for method_name in to_test:
+ method = getattr(loader, method_name)
+ self.assertRaises(ImportError, method, bad_name)
+
+
+class ExtensionFileLoaderTests(LoaderBasics):
+
+ testing_module = '_testcapi'
+ testing_location = find_ext_location(testing_module)
+ testing_path = os.path.join(*testing_location)
+
+ def setUp(self):
+ self.loader = importlib._ExtensionFileLoader(self.testing_module,
+ self.testing_path, False)
+
+ def test_basic(self):
+ # Should be able to import an extension module.
+ self.basic_test(self.loader, self.testing_module, self.testing_path)
+
+ def test_reload(self):
+ # A module that is already in sys.modules should be reused and be
+ # re-initialized.
+ self.reload_test(self.loader, self.testing_module)
+
+ def test_ImportError_on_bad_name(self):
+ # ImportError should be raised when a method is called with a module
+ # name it cannot handle.
+ self.ImportError_on_bad_name(self.loader, self.testing_module + 'asdf')
+
+ def test_is_package(self):
+ # Should always be False.
+ self.assert_(not self.loader.is_package(self.testing_module))
+
+ def test_get_code(self):
+ # Should always be None.
+ self.assert_(not self.loader.get_code(self.testing_module))
+
+ def test_get_source(self):
+ # Should always be None.
+ self.assert_(not self.loader.get_source(self.testing_module))
+
+
+class BasicPyFileLoaderTests(LoaderBasics, TestPyPycPackages):
+
+ """Very basic tests for the source loader."""
+
+ def setUp(self):
+ TestPyPycPackages.setUp(self, faked_names=False)
+
+ def test_basic(self):
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ self.basic_test(loader, self.module_name, self.py_path)
+
+ def test_pkg_basic(self):
+ loader = importlib._PyFileLoader(self.pkg_name, self.pkg_init_path,
+ True)
+ self.basic_test(loader, self.pkg_name, self.pkg_init_path)
+
+ def test_reload(self):
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ self.reload_test(loader, self.module_name)
+
+ def test_ImportError_on_bad_name(self):
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ extra_methods = ['write_bytecode', 'get_bytecode', 'mod_time']
+ self.ImportError_on_bad_name(loader, self.module_name + 'sdfasdf',
+ extra_methods)
+
+ def test_no_stale_module_on_failure(self):
+ # A failure during loading should not leave a partially initialized
+ # module in sys.modules.
+ def fail_loading(loader, name, *args):
+ sys.modules[name] = 'Should not exist'
+ raise ImportError('initial failed load')
+ if self.module_name in sys.modules:
+ del sys.modules[self.module_name]
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ loader._handler = fail_loading
+ self.assertRaises(ImportError, loader.load_module, self.module_name)
+ self.assert_(self.module_name not in sys.modules)
+
+ def test_file_deletion_post_init(self):
+ # Loading from source (with no bytecode), deleting the source (because
+ # bytecode was generated), and then loading again should work.
+ test_support.unlink(self.pyc_path)
+ if self.module_name in sys.modules:
+ del sys.modules[self.module_name]
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ found = loader.load_module(self.module_name)
+ self.verify_package(found, self.module_name)
+ del sys.modules[self.module_name]
+ os.unlink(self.py_path)
+ assert os.path.exists(self.pyc_path)
+ found = loader.load_module(self.module_name)
+ self.verify_package(found, self.module_name)
+
+ def test_fails_with_no_files_post_init(self):
+ # The loader should fail gracefully if the files it would have used to
+ # load the module have been removed.
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ test_support.unlink(self.py_path)
+ test_support.unlink(self.pyc_path)
+ self.assertRaises(ImportError, loader.load_module, self.module_name)
+
+ def test_reload_failure(self):
+ # If a module is reloaded and something happens during loading, the
+ # module should be left in place and not cleared out.
+ test_support.unlink(self.pyc_path)
+ with open(self.py_path, 'w') as source_file:
+ source_file.write('x = 42/0')
+ mock_module = mock_importlib.MockModule(self.module_name)
+ mock_module.__file__ = 'blah'
+ if hasattr(mock_module, '__loader__'):
+ delattr(mock_module, '__loader__')
+ sys.modules[self.module_name] = mock_module
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ self.assertRaises(ZeroDivisionError, loader.load_module,
+ self.module_name)
+ self.assert_(self.module_name in sys.modules)
+ self.assertEquals(mock_module.__file__, 'blah')
+ self.assert_(not hasattr(mock_module, '__loader__'))
+
+
+def log_call(instance, method_name):
+ """Log a method call."""
+ method = getattr(instance, method_name)
+ if not hasattr(instance, '_log'):
+ instance._log = []
+ def logger(*args, **kwargs):
+ instance._log.append(method_name)
+ return method(*args, **kwargs)
+ setattr(instance, method_name, logger)
+
+
+class PyFileLoaderLoadingTests(TestPyPycPackages):
+
+ """Test that the source loader uses the proper file.
+
+ Make sure all tests are run both against a top-level module and a package
+ where appropriate.
+
+ """
+
+ def setUp(self):
+ TestPyPycPackages.setUp(self, faked_names=False)
+
+ def test_pyc_over_py(self):
+ # If a bytecode file is good, don't even bother with the source
+ # (top-level or package).
+ for name, path, is_pkg in [(self.module_name, self.py_path, False),
+ (self.pkg_name, self.pkg_init_path, True)]:
+ if name in sys.modules:
+ del sys.modules[name]
+ py_compile.compile(path)
+ loader = importlib._PyFileLoader(name, path, is_pkg)
+ log_call(loader, 'get_source')
+ found = loader.load_module(name)
+ self.assert_('get_source' not in loader._log)
+ self.verify_package(found, name)
+
+ def test_only_good_pyc(self):
+ # Should be able to load even if only bytecode is available (top-level
+ # or package).
+ to_test = [(self.module_name, self.py_path, self.pyc_path, False),
+ (self.pkg_name, self.pkg_init_path, self.pkg_init_pyc_path,
+ True)]
+ for name, source_path, bytecode_path, is_pkg in to_test:
+ if name in sys.modules:
+ del sys.modules[name]
+ py_compile.compile(source_path)
+ os.unlink(source_path)
+ loader = importlib._PyFileLoader(name, bytecode_path, is_pkg)
+ log_call(loader, 'mod_time')
+ log_call(loader, 'get_source')
+ found = loader.load_module(name)
+ self.assert_('mod_time' not in loader._log)
+ self.assert_('get_source' not in loader._log)
+ self.verify_package(found, name)
+
+ def test_only_py(self):
+ # Having only source should be fine (top-level or package).
+ to_test = [(self.module_name, self.py_path, self.pyc_path, False),
+ (self.pkg_name, self.pkg_init_path, self.pkg_init_pyc_path,
+ True)]
+ for name, source_path, bytecode_path, is_pkg in to_test:
+ if name in sys.modules:
+ del sys.modules[name]
+ test_support.unlink(bytecode_path)
+ loader = importlib._PyFileLoader(name, source_path, is_pkg)
+ log_call(loader, 'get_bytecode')
+ log_call(loader, 'write_bytecode')
+ found = loader.load_module(name)
+ self.assert_('get_bytecode' not in loader._log)
+ self.assert_(os.path.exists(bytecode_path))
+ self.verify_package(found, name)
+ # Make sure generated bytecode is good.
+ del sys.modules[name]
+ os.unlink(source_path)
+ assert os.path.exists(bytecode_path)
+ loader = importlib._PyFileLoader(name, bytecode_path, is_pkg)
+ log_call(loader, 'mod_time')
+ log_call(loader, 'get_bytecode')
+ found = loader.load_module(name)
+ self.assert_('mod_time' not in loader._log)
+ self.assert_('get_bytecode' in loader._log)
+ self.verify_package(found, name)
+
+ def test_stale_pyc(self):
+ # If a bytecode file when compared to its source then regenerate the
+ # bytecode.
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ log_call(loader, 'write_bytecode')
+ with open(self.pyc_path, 'rb') as bytecode_file:
+ data = bytecode_file.read()
+ timestamp = importlib._r_long(data[4:8])
+ with open(self.pyc_path, 'wb') as bytecode_file:
+ bytecode_file.write(data[:4])
+ bytecode_file.write(importlib._w_long(timestamp-1))
+ bytecode_file.write(data[8:])
+ found = loader.load_module(self.module_name)
+ self.assert_('write_bytecode' in loader._log)
+ self.verify_package(found, self.module_name)
+ source_mtime = os.stat(self.py_path).st_mtime
+ bytecode_mtime = os.stat(self.pyc_path).st_mtime
+ self.assert_(bytecode_mtime >= source_mtime)
+ del sys.modules[self.module_name]
+ loader = importlib._PyFileLoader(self.module_name, self.pyc_path,
+ False)
+ log_call(loader, 'get_bytecode')
+ log_call(loader, 'get_source')
+ found = loader.load_module(self.module_name)
+ self.assert_('get_bytecode' in loader._log)
+ self.assert_('get_source' not in loader._log)
+ self.verify_package(found, self.module_name)
+
+ def test_bad_magic(self):
+ # If the magic cookie for bytecode is bad then raise an exception (no
+ # source), or regenerate the bytecode.
+ def change_magic():
+ with open(self.pyc_path, 'rb') as bytecode_file:
+ data = bytecode_file.read()
+ assert imp.get_magic() != '0' * 4
+ with open(self.pyc_path, 'wb') as bytecode_file:
+ bytecode_file.write('0' * 4)
+ bytecode_file.write(data[4:])
+ # With source.
+ change_magic()
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ log_call(loader, 'write_bytecode')
+ found = loader.load_module(self.module_name)
+ self.assert_('write_bytecode' in loader._log)
+ self.verify_package(found, self.module_name)
+ source_mtime = os.stat(self.py_path).st_mtime
+ bytecode_mtime = os.stat(self.pyc_path).st_mtime
+ self.assert_(source_mtime <= bytecode_mtime)
+ del sys.modules[self.module_name]
+ loader = importlib._PyFileLoader(self.module_name, self.pyc_path,
+ False)
+ log_call(loader, 'get_bytecode')
+ log_call(loader, 'get_source')
+ found = loader.load_module(self.module_name)
+ self.assert_('get_bytecode' in loader._log)
+ self.assert_('get_source' not in loader._log)
+ self.verify_package(found, self.module_name)
+ # Without source.
+ del sys.modules[self.module_name]
+ os.unlink(self.py_path)
+ change_magic()
+ loader = importlib._PyFileLoader(self.module_name, self.pyc_path,
+ False)
+ self.assertRaises(ImportError, loader.load_module, self.module_name)
+
+ def test_malformed_bytecode(self):
+ # Invalid bytecode triggers an exception, source or not.
+ source_mtime = int(os.stat(self.py_path).st_mtime)
+ with open(self.pyc_path, 'wb') as bytecode_file:
+ bytecode_file.write(imp.get_magic())
+ bytecode_file.write(importlib._w_long(source_mtime))
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ self.assertRaises(Exception, loader.load_module, self.module_name)
+
+
+class PEP302PyFileInterface(TestPyPycPackages):
+
+ """Test the optional extensions from PEP 302."""
+
+ def setUp(self):
+ TestPyPycPackages.setUp(self, faked_names=False)
+
+ def test_get_source(self):
+ # Return the source when available, None if there is at least bytecode,
+ # and raise ImportError if there is no change of loading the module.
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ source = loader.get_source(self.module_name)
+ self.assertEqual(source, self.source)
+ os.unlink(self.py_path)
+ self.assert_(loader.get_source(self.module_name) is None)
+ os.unlink(self.pyc_path)
+ self.assertRaises(ImportError, loader.get_source, self.module_name)
+
+ def test_get_data(self):
+ loader = importlib._PyFileLoader(self.module_name, self.pyc_path,
+ False)
+ data = loader.get_data(self.pyc_path)
+ with open(self.pyc_path, 'rb') as bytecode_file:
+ original = bytecode_file.read()
+ self.assertEqual(data, original)
+
+ def test_is_package(self):
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ self.assert_(not loader.is_package(self.module_name))
+ loader = importlib._PyFileLoader(self.pkg_name, self.pkg_init_path,
+ True)
+ self.assert_(loader.is_package(self.pkg_name))
+
+ def get_code_test(self, loader):
+ attr_name, attr_value = self.test_attr
+ code_object = loader.get_code(loader._name)
+ ns = {}
+ exec(code_object, ns)
+ self.assert_(attr_name in ns)
+ self.assertEqual(ns[attr_name], attr_value)
+
+ def test_get_code(self):
+ # Source and bytecode.
+ assert os.path.exists(self.py_path) and os.path.exists(self.pyc_path)
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ self.get_code_test(loader)
+ # Source only.
+ os.unlink(self.pyc_path)
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ self.get_code_test(loader)
+ # Bytecode only.
+ py_compile.compile(self.py_path, doraise=True)
+ os.unlink(self.py_path)
+ loader = importlib._PyFileLoader(self.module_name, self.pyc_path,
+ False)
+ self.get_code_test(loader)
+ # Bad magic number in bytecode.
+ with open(self.pyc_path, 'rb') as bytecode_file:
+ data = bytecode_file.read()
+ with open(self.pyc_path, 'wb') as bytecode_file:
+ bytecode_file.write('0' * 4)
+ bytecode_file.write('0' * 4)
+ bytecode_file.write(data[8:])
+ loader = importlib._PyFileLoader(self.module_name, self.pyc_path,
+ False)
+ self.assertRaises(ImportError, loader.get_code, self.module_name)
+ # Nothing available.
+ os.unlink(self.pyc_path)
+ assert not os.path.exists(self.py_path)
+ assert not os.path.exists(self.pyc_path)
+ loader = importlib._PyFileLoader(self.module_name, self.py_path, False)
+ self.assertRaises(ImportError, loader.get_code, self.module_name)
+
+
+def test_main():
+ test_support.run_unittest(ExtensionFileLoaderTests, BasicPyFileLoaderTests,
+ PyFileLoaderLoadingTests,
+ PEP302PyFileInterface)
+
+
+if __name__ == '__main__':
+ test_main()
Added: sandbox/trunk/import_in_py/Py3K/tests/test_import_mechanics.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/test_import_mechanics.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,513 @@
+import importlib
+
+from tests import mock_importlib
+
+import sys
+from test.test_support import run_unittest
+import unittest
+
+
+class ImportHelper(unittest.TestCase):
+
+ """Common helper methods for testing the Importer class."""
+
+ def setUp(self):
+ """Store a copy of the 'sys' attribute pertaining to imports."""
+ # Don't backup sys.modules since dict is cached.
+ # Don't clear sys.modules as it can wreak havoc
+ # (e.g., losing __builtin__).
+ self.old_sys_modules = sys.modules.copy()
+ self.old_meta_path = sys.meta_path[:]
+ sys.meta_path = []
+ self.old_sys_path = sys.path[:]
+ sys.path = []
+ self.old_path_hooks = sys.path_hooks[:]
+ sys.path_hooks = []
+ self.old_path_importer_cache = sys.path_importer_cache.copy()
+ sys.path_importer_cache.clear()
+ self.default_importer = mock_importlib.PassImporter()
+ self.importer = importlib.Import(self.default_importer, tuple())
+
+ def tearDown(self):
+ """Restore backup of import-related attributes in 'sys'."""
+ sys.modules.clear()
+ sys.modules.update(self.old_sys_modules)
+ sys.meta_path = self.old_meta_path
+ sys.path = self.old_sys_path
+ sys.path_hooks = self.old_path_hooks
+ sys.path_importer_cache = self.old_path_importer_cache
+
+ def clear_sys_modules(*modules):
+ for module in modules:
+ try:
+ del sys.modules[module]
+ except KeyError:
+ pass
+
+
+class ImportHelper2(ImportHelper):
+
+ """Create mock modules."""
+
+ def setUp(self):
+ ImportHelper.setUp(self)
+ self.parent_name = '<parent>'
+ self.child_name = '<child>'
+ self.full_child_name = '.'.join([self.parent_name, self.child_name])
+ self.parent_module = mock_importlib.MockModule(self.parent_name)
+ self.child_module = mock_importlib.MockModule(self.full_child_name)
+ setattr(self.parent_module, self.child_name, self.child_module)
+
+class ImportNameResolutionTests(ImportHelper2):
+
+ """Test the absolute name resolution for relative imports."""
+
+ def test_classic_relative_import_in_package(self):
+ # Importing from within a package's __init__ file should lead to a
+ # resolved import name of the package name tacked on to the name of the
+ # module being imported.
+ resolved_name = self.importer._classic_resolve_name(self.child_name,
+ self.parent_name,
+ True)
+ self.failUnlessEqual(resolved_name, self.full_child_name)
+
+ def test_classic_relative_import_in_module(self):
+ # Importing within a module in a package should lead to the importer's
+ # module name being removed and replaced with the name of what is to be
+ # imported.
+ calling_from = self.parent_name + '.' + '<calling from>'
+ resolved_name = self.importer._classic_resolve_name(self.child_name,
+ calling_from,
+ False)
+ self.failUnlessEqual(resolved_name, self.full_child_name)
+
+ def test_relative_import_in_package_for_a_module(self):
+ # Trying to import a single level within a package within it's __init__
+ # module should stay within the package.
+ # ``from .child_name import ...`` in a package.
+ resolved_name = self.importer._resolve_name(self.child_name,
+ self.parent_name, True, 1)
+ self.failUnlessEqual(resolved_name, self.full_child_name)
+
+ def test_relative_import_in_module_for_a_module(self):
+ # Importing from within a module in a package should try to import from
+ # within the same directory as the module requesting the import.
+ # ``from .child_name import ...`` in a package module.
+ calling_from = self.parent_name + '.' + '<calling from>'
+ resolved_name = self.importer._resolve_name(self.child_name,
+ calling_from, False, 1)
+ self.failUnlessEqual(resolved_name, self.full_child_name)
+
+ def test_relative_import_deep_in_package(self):
+ # Calling from a deep point in the package should still work.
+ depth = 10
+ name_extension = (str(x) for x in range(10))
+ calling_from = self.parent_name + '.' + '.'.join(name_extension)
+ resolved_name = self.importer._resolve_name(self.child_name,
+ calling_from, False, depth)
+ self.failUnlessEqual(resolved_name, self.full_child_name)
+
+ def test_attempt_to_escape_out_of_package_init(self):
+ # Attempting to go too high out of a package in its __init__ file
+ # should raise ImportError.
+ # ``from ..child_name import ...`` in a top-level package.
+ self.failUnlessRaises(ImportError, self.importer._resolve_name,
+ self.child_name, self.parent_name, True, 2)
+
+ def test_attempt_to_escape_out_of_package_module(self):
+ # Attempting to go too high in the package from a module should raise
+ # ImportError.
+ # ``from ..child_name import ...`` in a top-level package module.
+ calling_from = self.parent_name + '.' + '<calling from>'
+ self.failUnlessRaises(ImportError, self.importer._resolve_name,
+ self.child_name, calling_from, False, 2)
+
+ def test_relative_import_in_top_level(self):
+ # Attempting a relative import in a top-level location should raise
+ # ImportError.
+ # ``from .child_name import ...`` outside of a package.
+ self.failUnlessRaises(ImportError, self.importer._resolve_name,
+ self.child_name, self.parent_name, False, 1)
+
+ def test_relative_import_in_package_init(self):
+ # ``from . import ...`` in a package.
+ resolved_name = self.importer._resolve_name('', self.parent_name, True,
+ 1)
+ self.failUnlessEqual(resolved_name, self.parent_name)
+
+ def test_relative_import_in_package_module(self):
+ # ``from . import ...`` in a package module.
+ resolved_name = self.importer._resolve_name('', self.full_child_name,
+ False, 1)
+ self.failUnlessEqual(resolved_name, self.parent_name)
+
+ def test_relative_import_redirection(self):
+ # Having a relative module name resolve to a name that has a value of
+ # None in sys.modules should redirect to import an absolute import for
+ # the specified name.
+ module_name = '<to import>'
+ pkg_name = '<pkg>'
+ resolved_relative_name = module_name + '.' + pkg_name
+ expected_module = mock_importlib.MockModule(module_name)
+ sys.modules[resolved_relative_name] = None
+ sys.modules[module_name] = expected_module
+ importing_globals = {'__name__':pkg_name, '__path__':['some path']}
+ imported = self.importer(module_name, importing_globals, level=-1)
+ self.failUnless(imported is expected_module)
+
+
+class ImportFromListTests(ImportHelper2):
+
+ """Test conditions based on fromlist."""
+
+ def test_fromlist_relative_import(self):
+ # Any items specified in fromlist while importing a package needs to be
+ # checked as to whether it is a pre-existing attribute or should be
+ # considered a declaration for a relative import.
+ module_name = '<module name>'
+ pkg_name = '<pkg name>'
+ full_module_name = pkg_name + '.' + module_name
+ # Already have package imported.
+ pkg_module = mock_importlib.MockModule(pkg_name, pkg_list=['some path'])
+ sys.modules[pkg_name] = pkg_module
+ # Make sure implicit import succeeds.
+ succeed = mock_importlib.SucceedImporter()
+ sys.meta_path.append(succeed)
+ # Import the package with a fromlist of the module.
+ module = self.importer._return_module(pkg_name, '',
+ fromlist=[module_name])
+ self.failUnless(hasattr(module, module_name))
+ fromlist_module = getattr(module, module_name)
+ self.failUnlessEqual(fromlist_module.__name__, full_module_name)
+
+ def test_fromlist_nonexistent(self):
+ # If something listed in a fromlist does not exist the import
+ # should still succeed.
+ pkg_name = '<pkg name>'
+ pkg_module = mock_importlib.MockModule(pkg_name, pkg_list=['some path'])
+ sys.modules[pkg_name] = pkg_module
+ nonexistent_attr = 'asdfsdfd'
+ module = self.importer._return_module(pkg_name, '',
+ fromlist=[nonexistent_attr])
+ self.failUnless(not hasattr(module, nonexistent_attr))
+
+ def test_fromlist_existing(self):
+ # A value in fromlist that already exists should not lead to a relative
+ # import.
+ pkg_name = '<pkg name>'
+ pkg_module = mock_importlib.MockModule(pkg_name, pkg_list=['some path'])
+ attr = 'some_attr'
+ setattr(pkg_module, attr, None)
+ sys.modules[pkg_name] = pkg_module
+ failing_import = mock_importlib.ErrorImporter()
+ sys.meta_path.append(failing_import)
+ module = self.importer(pkg_name, fromlist=[attr])
+ self.failUnless(hasattr(module, attr))
+ self.failUnless(not hasattr(failing_import, 'find_request'))
+
+ def test_fromlist_nonpackage(self):
+ # No implicit imports of values in fromlist should be done if a module
+ # is what is being imported specifically.
+ module_name = '<module>'
+ module = mock_importlib.MockModule(module_name)
+ sys.modules[module_name] = module
+ failing_import = mock_importlib.ErrorImporter()
+ sys.meta_path.append(failing_import)
+ imported_module = self.importer(module_name, fromlist=['sadfsdd'])
+ self.failUnless(not hasattr(failing_import, 'find_request'))
+
+ def test_fromlist_relative_import_all(self):
+ # When '*' is passed in for fromlist, __all__ should be used for the
+ # possibility of a relative import.
+ module_name = '<module name>'
+ pkg_name = '<pkg name>'
+ full_module_name = pkg_name + '.' + module_name
+ pkg_module = mock_importlib.MockModule(pkg_name, pkg_list=['some path'],
+ __all__=[module_name])
+ sys.modules[pkg_name] = pkg_module
+ succeed = mock_importlib.SucceedImporter()
+ sys.meta_path.append(succeed)
+ # Also tests that fromlist can be a tuple and still work.
+ module = self.importer(pkg_name, fromlist=('*',))
+ self.failUnless(hasattr(module, module_name))
+ relative_module = getattr(module, module_name)
+ self.failUnlessEqual(relative_module.__name__, full_module_name)
+
+ def test_empty_fromlist(self):
+ # An empty fromlist means that the root module is returned.
+ sys.modules[self.parent_name] = self.parent_module
+ sys.modules[self.full_child_name] = self.child_module
+ module = self.importer._return_module(self.full_child_name, '', [])
+ self.failUnless(module is self.parent_module)
+
+ def test_nonempty_fromlist(self):
+ # A fromlist with values should return the specified module.
+ sys.modules[self.parent_name] = self.parent_module
+ sys.modules[self.full_child_name] = self.child_module
+ test_attr_name = 'test_attr'
+ setattr(self.child_module, test_attr_name, None)
+ module = self.importer._return_module(self.full_child_name, '',
+ [test_attr_name])
+ self.failUnless(module is self.child_module)
+
+ def test_relative_import_empty_fromlist(self):
+ # If a relative import (with no dot) is performed with an empty fromlist
+ # then the module specified in the relative name is returned.
+ sys.modules[self.full_child_name] = self.child_module
+ sys.modules[self.parent_name] = self.parent_module
+ module = self.importer._return_module(self.full_child_name,
+ self.child_name, [])
+ self.failUnless(module is self.child_module)
+
+ def test_deep_relative_import_empty_fromlist(self):
+ # If the relative name of a module has a dot, return the module that
+ # resolves to the absolute name of the module up to the first dot.
+ sys.modules[self.full_child_name] = self.child_module
+ sys.modules[self.parent_name] = self.parent_module
+ extra_depth = '.baz'
+ absolute_name = self.full_child_name + extra_depth
+ relative_name = self.child_name + extra_depth
+ module = self.importer._return_module(absolute_name, relative_name, [])
+ self.failUnless(module is self.child_module)
+
+
+class ImportMiscTests(ImportHelper2):
+
+ """Test miscellaneous parts of the Import class.
+
+ Tests of the default values for __init__ are handled in the integration
+ tests.
+
+ """
+
+ def test_sys_module_return(self):
+ # A module found in sys.modules should be returned immediately.
+ sys.modules[self.parent_name] = self.parent_module
+ module = self.importer._import_module(self.parent_name)
+ self.failUnless(module is self.parent_module)
+
+ def test_sys_module_None(self):
+ # If sys.modules contains None for a module name, then raise ImportError.
+ module_name = '<module>'
+ sys.modules[module_name] = None
+ self.failUnlessRaises(ImportError, self.importer._import_module,
+ module_name)
+
+ def test_parent_missing(self):
+ # An import should fail if a parent module cannot be found.
+ sys.modules[self.full_child_name] = self.child_module
+ self.failUnlessRaises(ImportError, self.importer, self.full_child_name)
+
+ def test_parent_imported(self):
+ # If a parent module is already imported, importing a child module
+ # should work (along with setting the attribute on the parent for the
+ # child module).
+ delattr(self.parent_module, self.child_name)
+ succeed_importer = mock_importlib.SucceedImporter()
+ sys.meta_path.append(succeed_importer)
+ sys.modules[self.parent_name] = self.parent_module
+ self.importer(self.full_child_name)
+ self.failUnless(hasattr(self.parent_module, self.child_name))
+ self.failUnlessEqual(getattr(self.parent_module, self.child_name),
+ sys.modules[self.full_child_name])
+
+ def test_parent_not_imported(self):
+ # If a parent module is not imported yet, it should be imported.
+ # The attribute on the parent module for the child module should also
+ # be set.
+ delattr(self.parent_module, self.child_name)
+ succeed_importer = mock_importlib.SucceedImporter()
+ sys.meta_path.append(succeed_importer)
+ self.importer(self.full_child_name)
+ self.failUnless(self.parent_name in sys.modules)
+ self.failUnless(self.full_child_name in sys.modules)
+ self.failUnless(hasattr(sys.modules[self.parent_name], self.child_name))
+ self.failUnlessEqual(getattr(sys.modules[self.parent_name],
+ self.child_name),
+ sys.modules[self.full_child_name])
+
+ def test_None_not_set_for_import_failure(self):
+ # If an import that is tried both relative and absolute fails there
+ # should not be an entry of None for the resolved relative name.
+ module_name = '<should fail>'
+ pkg_name = '<non-existent package>'
+ resolved_name = module_name + '.' + pkg_name
+ importing_globals = {'__name__':pkg_name, '__path__':['path']}
+ self.failUnlessRaises(ImportError, self.importer, module_name,
+ importing_globals, {}, [], -1)
+ self.failUnless(resolved_name not in sys.modules)
+
+ def test_empty_string(self):
+ # An empty string should raise ValueError if level is not > 0.
+ for level in (-1, 0):
+ self.failUnlessRaises(ValueError, self.importer, '', {}, {}, level)
+
+ def test_module_from_cache(self):
+ # If a value is in sys.modules it should be returned (no matter the
+ # object type), else return False.
+ value = "a 'module'"
+ mod_name = "module name"
+ sys.modules[mod_name] = value
+ returned = self.importer.module_from_cache(mod_name)
+ self.failUnless(returned is value)
+ returned = self.importer.module_from_cache(mod_name + 'asdfeddf')
+ self.failUnless(returned is False)
+
+ def test_post_import(self):
+ # Post-import processing should do nothing but return the module
+ # unscathed.
+ module = "mod"
+ self.failUnless(self.importer.post_import(module) is module)
+
+
+class ImportMetaPathTests(ImportHelper):
+
+ """Test meta_path usage."""
+
+ def test_search_meta_path(self):
+ # Test search method of sys.meta_path.
+ # Should raise ImportError on error.
+ self.clear_sys_modules('sys')
+ self.failUnlessRaises(ImportError, self.importer._search_meta_path,
+ 'sys')
+ # Verify call order.
+ meta_path = (mock_importlib.PassImporter(),
+ mock_importlib.SucceedImporter())
+ sys.meta_path = meta_path
+ loader = self.importer._search_meta_path('sys')
+ for entry in meta_path:
+ self.failUnlessEqual(entry.find_request, ('sys', None))
+ self.failUnless(loader is meta_path[-1])
+
+ def test_extended_meta_path(self):
+ # Default meta_path entries set during initialization should be
+ # queried after sys.meta_path.
+ self.clear_sys_modules('sys')
+ pass_importer = mock_importlib.PassImporter()
+ sys.meta_path = [pass_importer]
+ succeed_importer = mock_importlib.SucceedImporter()
+ importer_ = importlib.Import(extended_meta_path=(succeed_importer,))
+ module = importer_._import_module('sys')
+ for meta_importer in (pass_importer, succeed_importer):
+ self.failUnlessEqual(meta_importer.find_request, ('sys', None))
+ self.failUnless(module in succeed_importer.loaded_modules)
+
+ def test_parent_path(self):
+ # If a parent module has __path__ defined it should be passed as an
+ # argument during importing.
+ pkg_name = '<pkg>'
+ module_name = '<module>'
+ test_path = ['<test path>']
+ pkg_module = mock_importlib.MockModule(pkg_name)
+ pkg_module.__path__ = test_path
+ sys.modules[pkg_name] = pkg_module
+ succeed_importer = mock_importlib.SucceedImporter()
+ sys.meta_path.append(succeed_importer)
+ full_module_name = '.'.join([pkg_name, module_name])
+ lookup_args = (full_module_name, test_path)
+ self.importer(full_module_name)
+ self.failUnless(full_module_name in sys.modules)
+ self.failUnlessEqual(succeed_importer.find_request, lookup_args)
+ self.failUnlessEqual(succeed_importer.load_request, lookup_args[0])
+
+
+class ImportStdPathTests(ImportHelper):
+
+ """Test sys.path usage."""
+
+ def test_default_importer_factory(self):
+ # Make sure that the object passed in during initialization is used
+ # when sys.path_importer_cache has a value of None.
+ module_name = '<dummy>'
+ self.clear_sys_modules(module_name)
+ succeed_importer = mock_importlib.SucceedImporter()
+ importer_ = importlib.Import(succeed_importer, tuple())
+ sys.meta_path = []
+ sys.path = ['<succeed>']
+ sys.path_importer_cache['<succeed>'] = None
+ module = importer_._import_module(module_name)
+ self.failUnlessEqual(succeed_importer.find_request,
+ (module_name, None))
+ self.failUnless(module in succeed_importer.loaded_modules)
+
+ def test_search_std_path(self):
+ # Test sys.path searching for a loader.
+ module_name = '<dummy>'
+ self.clear_sys_modules(module_name)
+ importer_ = importlib.Import(extended_meta_path=())
+ sys.path = []
+ pass_importer = mock_importlib.PassImporter.set_on_sys_path()
+ succeed_importer = mock_importlib.SucceedImporter.set_on_sys_path()
+ sys_path = (pass_importer, succeed_importer)
+ module = importer_._import_module(module_name)
+ for entry in sys_path:
+ self.failUnlessEqual(entry.find_request, (module_name, None))
+ self.failUnless(module in succeed_importer.loaded_modules)
+
+ def test_importer_cache_preexisting(self):
+ # A pre-existing importer should be returned if it exists in
+ # sys.path_importer_cache.
+ module_name = '<dummy>'
+ self.clear_sys_modules(module_name)
+ sys.path = []
+ succeed_importer = mock_importlib.SucceedImporter.set_on_sys_path()
+ loader = self.importer._search_std_path(module_name)
+ self.failUnless(loader is succeed_importer)
+
+ def test_importer_cache_from_path_hooks(self):
+ # If an entry does not exist for a sys.path entry in the importer cache
+ # then sys.path_hooks should be searched and if one is found then cache
+ # it.
+ module_name = '<dummy>'
+ self.clear_sys_modules(module_name)
+ path_entry = '<succeed>'
+ succeed_importer = mock_importlib.SucceedImporter()
+ sys.path = [path_entry]
+ sys.path_importer_cache.clear()
+ sys.path_hooks = [succeed_importer]
+ loader = self.importer._search_std_path(module_name)
+ self.failUnless(loader is succeed_importer)
+ self.failUnless(sys.path_importer_cache[path_entry] is
+ succeed_importer)
+
+ def test_importer_cache_no_path_hooks(self):
+ # If an entry does not exist for a sys.path entry in the importer cache
+ # and sys.path_hooks has nothing for the entry, None should be set.
+ module_name = '<dummy>'
+ self.clear_sys_modules(module_name)
+ path_entry = '<test>'
+ sys.path = [path_entry]
+ sys.path_hooks = []
+ sys.path_importer_cache.clear()
+ self.failUnlessRaises(ImportError, self.importer._search_std_path,
+ module_name)
+ self.failUnless(sys.path_importer_cache[path_entry] is None)
+
+ def test_searching_package_path(self):
+ # If importing in a package then search path is the package's __path__
+ # value; otherwise it is sys.path.
+ succeed_importer = mock_importlib.SucceedImporter()
+ sys.path_hooks.append(succeed_importer)
+ search_paths = ['test path']
+ module_name = '<pkg>.<dummy>'
+ loader = self.importer._search_std_path(module_name, search_paths)
+ self.failUnless(loader is succeed_importer)
+ self.failUnless(search_paths[0] in succeed_importer.path_entries)
+ self.failUnlessEqual(succeed_importer.find_request,
+ (module_name, None))
+ self.failUnless(search_paths[0] in sys.path_importer_cache)
+ self.clear_sys_modules(module_name)
+ del sys.path_importer_cache[search_paths[0]]
+ succeed_importer.path_entries = []
+ self.importer._import_module(module_name, search_paths)
+ self.failUnless(search_paths[0] in succeed_importer.path_entries)
+
+
+def test_main():
+ run_unittest(ImportNameResolutionTests, ImportFromListTests,
+ ImportMiscTests, ImportMetaPathTests, ImportStdPathTests)
+
+
+
+if __name__ == '__main__':
+ test_main()
Added: sandbox/trunk/import_in_py/Py3K/tests/test_regression.py
==============================================================================
--- (empty file)
+++ sandbox/trunk/import_in_py/Py3K/tests/test_regression.py Mon Sep 17 00:20:58 2007
@@ -0,0 +1,251 @@
+import importlib
+
+from tests import mock_importlib
+from tests.py_help import TestPyPycPackages
+from importlib import _r_long
+
+import imp
+import marshal
+import os
+import io
+import sys
+from test.test_support import run_unittest
+
+
+class IntegrationTests(TestPyPycPackages):
+
+ """Tests that verify the default semantics are what is expected.
+
+ Tests should verify that:
+ * The proper module was returned.
+ * All expected modules were added to sys.modules.
+ * All modules imported by the call have the proper attributes.
+
+ There are currently no tests for the import lock.
+
+ Need to test (both relative and absolute imports as needed):
+ * import module
+ * import package
+ * import package.module
+ * from module import attribute
+ * from package import module
+ * from package import attribute
+ * from package.module import attribute
+ * from package import *
+ + With __all__
+
+ """
+
+ def setUp(self):
+ TestPyPycPackages.setUp(self, False)
+ self.import_ = importlib.Import()
+ self.cached_modules = []
+ if self.module_name in sys.modules:
+ del sys.modules[self.module_name]
+ if self.pkg_name in sys.modules:
+ del sys.modules[self.pkg_name]
+ if self.pkg_module_name in sys.modules:
+ del sys.modules[self.pkg_module_name]
+
+ def tearDown(self):
+ TestPyPycPackages.tearDown(self)
+ for module_name, module in self.cached_modules:
+ sys.modules[module_name] = module
+
+ def clear_sys_modules(self, *modules):
+ """Remove specified modules from sys.modules for a test and put back
+ during teardown."""
+ for module_name in modules:
+ try:
+ self.cached_modules.append((module_name,
+ sys.modules[module_name]))
+ del sys.modules[module_name]
+ except KeyError:
+ pass
+
+ def test_builtin(self):
+ # Test importing a built-in module.
+ # ``import sys``
+ self.clear_sys_modules('sys')
+ module = self.import_('sys')
+ self.failUnlessEqual(module.__name__, 'sys')
+ # Only test for attributes that are not set during interpreter
+ # creation!
+ self.failUnless(hasattr(module, 'exc_info'))
+ self.failUnless(hasattr(module, 'version'))
+
+ def test_frozen(self):
+ # Importing a frozen module should work.
+ # ``import __hello__``
+ self.clear_sys_modules('__hello__')
+ faked_stdout = io.StringIO()
+ sys.stdout = faked_stdout
+ try:
+ module = self.import_('__hello__')
+ self.failUnlessEqual(module.__name__, '__hello__')
+ self.failUnless('frozen' in module.__file__)
+ finally:
+ sys.stdout = sys.__stdout__
+
+ def test_extension(self):
+ # Should be able to import extension modules.
+ # ``import time``
+ module = self.import_('time')
+ self.failUnlessEqual(module.__name__, 'time')
+ self.failUnless(hasattr(module, 'time'))
+
+ def test_pyc_w_py(self):
+ # Should be able to import a .pyc file when a .py is also present.
+ # ``import pyc`` with a corresponding .py .
+ module = self.import_(self.module_name)
+ self.verify_module(module, self.pyc_path)
+
+ def test_pyc_wo_py(self):
+ # Importing just a .pyc file (w/ no .py) should be okay.
+ # ``import pyc`` from a .pyc .
+ os.remove(self.py_path)
+ assert os.path.exists(self.pyc_path)
+ module = self.import_(self.module_name)
+ self.verify_module(module, self.pyc_path)
+
+ def test_sys_modules(self):
+ # Should be able to pull from sys.modules even if a file does not exist
+ # for the module.
+ # ``import module`` from sys.modules.
+ test_module_name = '<' + self.module_name + '>'
+ test_module = mock_importlib.MockModule(test_module_name)
+ sys.modules[test_module_name] = test_module
+ try:
+ module = self.import_(test_module_name)
+ self.failUnless(module is test_module)
+ finally:
+ del sys.modules[test_module_name]
+
+ def test_py_creating_pyc(self):
+ # Importing a .py file should work and generate a .pyc file.
+ # ``import py`` creating a .pyc .
+ os.remove(self.pyc_path)
+ module = self.import_(self.module_name)
+ self.verify_module(module)
+ self.failUnless(os.path.exists(self.pyc_path))
+ with open(self.pyc_path, 'rb') as pyc_file:
+ data = pyc_file.read()
+ self.failUnlessEqual(data[:4], imp.get_magic())
+ py_mod = int(os.stat(self.py_path).st_mtime)
+ # XXX Using importer's _r_long.
+ pyc_mod = _r_long(data[4:8])
+ self.failUnlessEqual(py_mod, pyc_mod)
+ code = marshal.loads(data[8:])
+ module = mock_importlib.MockModule(self.module_name)
+ exec(code, module.__dict__)
+ self.verify_module(module)
+
+ def test_top_level_package(self):
+ # Should be able to import a top-level package.
+ # ``import package``
+ module = self.import_(self.pkg_name)
+ self.verify_package(module)
+
+ def test_package_module(self):
+ # A module within a top-level package should work with the package not
+ # already imported.
+ # ``import package.module``
+ assert '.' in self.pkg_module_name
+ module = self.import_(self.pkg_module_name)
+ self.verify_package(module, self.pkg_module_name)
+
+ def test_sub_package(self):
+ # A package within a package should work.
+ # ``import package.subpackage``
+ module = self.import_(self.sub_pkg_name)
+ self.verify_package(module, self.sub_pkg_name)
+
+ def test_sub_package_module(self):
+ # A module contained within a sub-package should work.
+ # ``import package.subpackage.module``
+ module = self.import_(self.sub_pkg_module_name)
+ self.verify_package(module, self.sub_pkg_module_name)
+
+ def test_classic_relative_import_in_package_init(self):
+ # Importing within a package's __init__ file using a relative name
+ # should work properly.
+ # ``import module`` for 'package' where 'module' is 'package.module'.
+ package_globals = {'__name__':self.pkg_name, '__path__':['some_path']}
+ module = self.import_(self.module_name, package_globals, level=-1)
+ self.verify_package(module, self.pkg_module_name)
+
+ def test_classic_relative_import_in_module(self):
+ # Importing using a relative name in a module in a package should work.
+ # ``import module`` for 'package.some_module' where 'module' is
+ # 'package.module'.
+ module_globals = {'__name__':self.pkg_name + '.' + 'another_module'}
+ module = self.import_(self.module_name, module_globals, level=-1)
+ self.verify_package(module, self.pkg_module_name)
+
+ def test_absolute_name_in_classic_relative_context(self):
+ # Importing a module that happens to be ambiguous in terms of being
+ # relative or absolute, but only exists in an absolute name context,
+ # should work. It should also lead to a value for None in sys.modules
+ # for the resolved relative name.
+ # ``import module`` for 'package' where 'module' is 'module'.
+ package_module_globals = {'__name__':self.pkg_module_name}
+ module = self.import_(self.top_level_module_name,
+ package_module_globals, level=-1)
+ resolved_name = self.pkg_name + '.' + self.top_level_module_name
+ self.verify_package(module)
+ self.failUnlessEqual(module.__name__, self.top_level_module_name)
+ self.failUnless(sys.modules[resolved_name] is None)
+
+ def test_relative_import_in_package_init(self):
+ # Importing a module with a relative name in a package's __init__ file
+ # should work.
+ # ``from . import module`` for 'package'.
+ caller_globals = {'__name__':self.pkg_name, '__path__':[self.pkg_path]}
+ module = self.import_('', caller_globals, fromlist=[self.module_name],
+ level=1)
+ self.verify_package(module, self.pkg_name)
+
+ def test_relative_import_in_package(self):
+ # Importing a module with a relative name in another module should
+ # work.
+ # ``from . import module`` for 'package.module'.
+ caller_globals = {'__name__':self.pkg_module_name}
+ module = self.import_('', caller_globals, fromlist=[self.module_name],
+ level=1)
+ self.verify_package(module, self.pkg_name)
+
+ def test_relative_import_in_subpackages(self):
+ # ``from .. import module`` in 'package.subpackage'.
+ caller_globals = {'__name__':self.sub_pkg_name,
+ '__path__':[self.sub_pkg_path]}
+ module = self.import_('', caller_globals, fromlist=[self.module_name],
+ level=2)
+ self.verify_package(module, self.pkg_name)
+
+ def test_relative_import_of_package(self):
+ # ``from ..subpackage import module`` in 'package.subpackage'.
+ # XXX
+ caller_globals = {'__name__':self.sub_pkg_name,
+ '__path__':[self.sub_pkg_path]}
+ module = self.import_(self.sub_pkg_tail_name, caller_globals,
+ fromlist=[self.module_name], level=2)
+ self.verify_package(module, self.sub_pkg_name)
+
+ def test_relative_import_return(self):
+ # When importing from a relative name, the module up to the first dot
+ # of that relative name (made absolute) should be returned.
+ # ``import subpackage.module`` for 'package.module'.
+ module_globals = {'__name__':self.pkg_module_name}
+ relative_name = self.sub_pkg_tail_name + '.' + self.module_name
+ module = self.import_(relative_name, module_globals)
+ self.verify_package(module, self.sub_pkg_name)
+
+
+def test_main():
+ run_unittest(IntegrationTests)
+
+
+
+if __name__ == '__main__':
+ test_main()
+
Deleted: /sandbox/trunk/import_in_py/Py3K_TODO
==============================================================================
--- /sandbox/trunk/import_in_py/Py3K_TODO Mon Sep 17 00:20:58 2007
+++ (empty file)
@@ -1,14 +0,0 @@
-This doc lists what is left to do before importlib can become the
-"One True Import". The items in the list must be done in order!
-
-- 2.6-specific
- * Rewrite (as a whitebox re-engineering job) zipimport.
- + Pass 2.6 unit tests for the module.
- * Put into 2.6 stdlib.
-- Py3K-specific
- * Port to Py3K.
- + Consider it a fork of code as semantics can change compared to 2.6 code.
- * Bootstrap in as default import implementation.
- * Fix C API to call importlib.
- * Remove C implementation.
- * Tune performance.
More information about the Python-checkins
mailing list