[Distutils] setuptools >= 20.2 may break applications using pyparsing

[Donald Stufft]

> On May 9, 2016, at 8:37 AM, Sebastian Krause <sebastian at realpath.org> wrote:
> 
> Hi,
> 
> I'm developing an application that uses pyparsing and after upgrading setuptools to the newest version I noticed some tests failing. In my main parser module I define an alias for the ParseBaseException which I then use in other parts of the application to catch exceptions:
> 
> # definition of the ParseException
> ParseException = pyparsing.ParseBaseException
> 
> # importing this alias in another module
> from ...filterreader.parser import ParseException
> 
> Now my tests were failing because the ParseException was never actually caught. Some investigation by comparing the id() of the objects showed that the ParseException alias was no longer the same object as pyparsing.ParseBaseException. This was because the module "pyparsing" at the time of the alias definition was not the same "pyparsing" module which is later used for parsing. Looking at sys.module I can see that I have two pyparsing modules:
> 
> pyparsing: <module 'pyparsing' from 'D:\\Project\\python34.zip\\pyparsing.pyc'>
> pkg_resources.extern.pyparsing: <module 'pyparsing' from 'D:\\Project\\python34.zip\\pyparsing.pyc'>
> 
> At the time of the alias definition id(pyparsing) is equal to the id() of pkg_resources.extern.pyparsing. When I later import pyparsing I get the other module. This whole problem only happens when I use the application packaged by cx_Freeze, so maybe some kind of race condition happens when importing from a ZIP file. I'm using 64 bit Python 3.4.4 on Windows.
> 
> The first version of setuptools where I can see this problem is 20.2, until 20.1 everything is fine. Looking at the source I can see that starting with 20.2 setuptools also includes its own pyparsing copy, so most likely that change is related to my problem.
> 
> Is there a simple way in which I can guarantee that there will only ever be a single "pyparsing" module in my application? Of course I could just stop using the alias and use the pyparsing exceptions directly, but I feel a bit uneasy when a module just changes its identity at some point between imports.
> 
> Sebastian
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


This sounds like something you should open as a bug with setuptools, the problem lies with pkg_resources.extern:VendorImporter. Probably it should stop trying to be as tricky and do something more like what pip does here.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160509/2dd01a17/attachment.sig>