[Python-Dev] python3k : imp.find_module raises SyntaxError

[Ron Adam]

On 11/30/2010 01:41 PM, Brett Cannon wrote:
> On Mon, Nov 29, 2010 at 12:21, Ron Adam<rrr at ronadam.com>  wrote:
>>
>>
>> On 11/29/2010 01:22 PM, Brett Cannon wrote:
>>>
>>> On Mon, Nov 29, 2010 at 03:53, Sylvain Thénault
>>> <sylvain.thenault at logilab.fr>    wrote:
>>>>
>>>> On 25 novembre 11:22, Ron Adam wrote:
>>>>>
>>>>> On 11/25/2010 08:30 AM, Emile Anclin wrote:
>>>>>>
>>>>>> hello,
>>>>>>
>>>>>> working on Pylint, we have a lot of voluntary corrupted files to test
>>>>>> Pylint behavior; for instance
>>>>>>
>>>>>> $ cat /home/emile/var/pylint/test/input/func_unknown_encoding.py
>>>>>> # -*- coding: IBO-8859-1 -*-
>>>>>> """ check correct unknown encoding declaration
>>>>>> """
>>>>>>
>>>>>> __revision__ = 'éééé'
>>>>>>
>>>>>>
>>>>>> and we try to find that module :
>>>>>> find_module('func_unknown_encoding', None). But python3 raises
>>>>>> SyntaxError
>>>>>> in that case ; it didn't raise SyntaxError on python2 nor does so on
>>>>>> our
>>>>>> func_nonascii_noencoding and func_wrong_encoding modules (with obvious
>>>>>> names)
>>>>>>
>>>>>> Python 3.2a2 (r32a2:84522, Sep 14 2010, 15:22:36)
>>>>>> [GCC 4.3.4] on linux2
>>>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>>>>
>>>>>>>> >from imp import find_module
>>>>>>>>>
>>>>>>>>> find_module('func_unknown_encoding', None)
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>>    File "<stdin>", line 1, in<module>
>>>>>> SyntaxError: encoding problem: with BOM
>>>>>
>>>>> I don't think there is a clear reason by design.  Also try importing
>>>>> the same modules directly and noting the differences in the errors
>>>>> you get.
>>>>
>>>> IMO the point is that we can consider as a bug the fact that find_module
>>>> tries to somewhat read the content of the file, no? Though it seems to
>>>> only
>>>> doing this for encoding detection or like since find_module doesn't choke
>>>> on
>>>> a module containing another kind of syntax error.
>>>>
>>>> So the question is, should we deal with this in pylint/astng, or can we
>>>> expect
>>>> this to be fixed at some point?
>>>
>>> Considering these semantics changed between Python 2 and 3 w/o a
>>> discernable benefit (I would consider it a negative as finding a
>>> module should not be impacted by syntactic correctness; the full act
>>> of importing should be the only thing that cares about that), I would
>>> consider it a bug that should be filed.
>>
>> The output of imp.find_module() returns an open file io object, and it's
>> output feeds directly into to imp.load_module().
>>
>>>>> imp.find_module('pydoc')
>> (<_io.TextIOWrapper name=4 encoding='utf-8'>,
>> '/usr/local/lib/python3.2/pydoc.py', ('.py', 'U', 1))
>>
>> So I think the imp.find_module() is suppose to be used when you *do* want to
>> do the full act of importing and not for just finding out if or where module
>> xyz exists.
>
> Going with your line of argument, why can't imp.load_module be the
> call that figures out there is a syntax error? If you look at this
> from the perspective of PEP 302, finding a module has absolutely
> nothing to do with the validity of the found source, just that
> something was found somewhere which (hopefully) contains code that
> represents the module.

The part that I'm looking at, is what would find_module return if the 
encoding is bad or not found for the encoding?

    <_io.TextIOWrapper name=4 encoding='bad_encoding'>


Maybe we could have some library introspection function in the inspect for 
just looking in the library rather than loading modules.  But I think those 
would have the same issues, as packages need to be loaded in order to find 
sub modules.*

* It almost seems like the concept of a sub-module (in a package) is 
flawed.  I'm not sure I can explain what causes me to feel that way at the 
moment though.

Ron