[ python-Bugs-1695718 ] PEP 302 broken

[SourceForge.net]

Bugs item #1695718, was opened at 2007-04-06 09:32
Message generated for change (Comment added) made by bcannon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: phil (philipdumont)
Assigned to: Georg Brandl (gbrandl)
Summary: PEP 302 broken

Initial Comment:
The product I'm working on uses a PEP 302 importer hook.  It's a home-grown ltihooks clone.  (We don't use ltihooks because it's GPLed, and we don't want to be.)

Our importer worked on 2.4.3, is broken on 2.5.0.

What's broken is that if the hook's find_module() method returns None for a given module name, say 'spam', then that is supposed to cause the import machinery to fall back on the regular unhooked import behavior -- that is, find 'spam.py' (or 'spam.pyc', 'spam.pyo') in the directory in question.  This used to be what happened, but no longer.

Tracing through the code, the problem seems to be occurring due to the 'continue' at line 1289 (in the 2.5 tarball) of Python/import.c.

Slogging through SVN (aside: this would have been easier if your ViewSVN supported annotate/blame -- any chance you could add that?), it appears that the offending continue statement was added in revision 46372, whose checkin comment claims that it was done for performance reasons.  I'm all for performance improvements, but not at the expense of breakage.

Attached is a tarball with some files that reproduce the problem.  (The LibtoolImporter.py file is a stripped-down toy version of what we are really using.)  Unwind the tarball, cd to the directory, and run script.py.  Here's what I get:



shell prompt> pwd
/home/phil/pep302_bug
shell prompt> ls -CF
eggs.la  LibtoolImporter.py  script.py*  spam.py
shell prompt> python2.4.3 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
module spam loaded
shell prompt> python2.5 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
Traceback (most recent call last):
  File "script.py", line 4, in <module>
    import spam
ImportError: No module named spam
shell prompt>

----------------------------------------------------------------------

>Comment By: Brett Cannon (bcannon)
Date: 2007-04-07 21:55

Message:
Logged In: YES 
user_id=357491
Originator: NO

I don't agree with this interpretation of PEP 302 for this instance.  If
you read the PEP there is no explicit mention that if an importer for an
entry on sys.path fails that it falls back on the default import behaviour.
 The only mention of using the default behaviour is if a value of None is
stored in sys.path_importer_cache (which also occurs when no entry on
sys.path_hooks returns a usable importer).

In my interpretation of PEP 302 (and how I implemented it for my pure
Python import implementation), if an importer exists for an entry on
sys.path then it essentially "owns" that entry.  The default import
semantics only kick in for unclaimed sys.path entries in my view.

Now I could be wrong and if I am I hope Phil can point out where in PEP
302 I am wrong.  Otherwise we have either undocumented behaviour that
changed (and that is always messy) or implied semantics that got fixed.

In other words, this probably needs to go to python-dev if Phil can't
point out exactly where PEP 302 says the semantics he is expecting is
supposed to be that way.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-06 12:21

Message:
Logged In: YES 
user_id=80475
Originator: NO

Georg, I believe this was your checkin.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470