setuptools >= 20.2 may break applications using pyparsing
Hi, I'm developing an application that uses pyparsing and after upgrading setuptools to the newest version I noticed some tests failing. In my main parser module I define an alias for the ParseBaseException which I then use in other parts of the application to catch exceptions: # definition of the ParseException ParseException = pyparsing.ParseBaseException # importing this alias in another module from ...filterreader.parser import ParseException Now my tests were failing because the ParseException was never actually caught. Some investigation by comparing the id() of the objects showed that the ParseException alias was no longer the same object as pyparsing.ParseBaseException. This was because the module "pyparsing" at the time of the alias definition was not the same "pyparsing" module which is later used for parsing. Looking at sys.module I can see that I have two pyparsing modules: pyparsing: <module 'pyparsing' from 'D:\\Project\\python34.zip\\pyparsing.pyc'> pkg_resources.extern.pyparsing: <module 'pyparsing' from 'D:\\Project\\python34.zip\\pyparsing.pyc'> At the time of the alias definition id(pyparsing) is equal to the id() of pkg_resources.extern.pyparsing. When I later import pyparsing I get the other module. This whole problem only happens when I use the application packaged by cx_Freeze, so maybe some kind of race condition happens when importing from a ZIP file. I'm using 64 bit Python 3.4.4 on Windows. The first version of setuptools where I can see this problem is 20.2, until 20.1 everything is fine. Looking at the source I can see that starting with 20.2 setuptools also includes its own pyparsing copy, so most likely that change is related to my problem. Is there a simple way in which I can guarantee that there will only ever be a single "pyparsing" module in my application? Of course I could just stop using the alias and use the pyparsing exceptions directly, but I feel a bit uneasy when a module just changes its identity at some point between imports. Sebastian
On May 9, 2016, at 8:37 AM, Sebastian Krause <sebastian@realpath.org> wrote:
Hi,
I'm developing an application that uses pyparsing and after upgrading setuptools to the newest version I noticed some tests failing. In my main parser module I define an alias for the ParseBaseException which I then use in other parts of the application to catch exceptions:
# definition of the ParseException ParseException = pyparsing.ParseBaseException
# importing this alias in another module from ...filterreader.parser import ParseException
Now my tests were failing because the ParseException was never actually caught. Some investigation by comparing the id() of the objects showed that the ParseException alias was no longer the same object as pyparsing.ParseBaseException. This was because the module "pyparsing" at the time of the alias definition was not the same "pyparsing" module which is later used for parsing. Looking at sys.module I can see that I have two pyparsing modules:
pyparsing: <module 'pyparsing' from 'D:\\Project\\python34.zip\\pyparsing.pyc'> pkg_resources.extern.pyparsing: <module 'pyparsing' from 'D:\\Project\\python34.zip\\pyparsing.pyc'>
At the time of the alias definition id(pyparsing) is equal to the id() of pkg_resources.extern.pyparsing. When I later import pyparsing I get the other module. This whole problem only happens when I use the application packaged by cx_Freeze, so maybe some kind of race condition happens when importing from a ZIP file. I'm using 64 bit Python 3.4.4 on Windows.
The first version of setuptools where I can see this problem is 20.2, until 20.1 everything is fine. Looking at the source I can see that starting with 20.2 setuptools also includes its own pyparsing copy, so most likely that change is related to my problem.
Is there a simple way in which I can guarantee that there will only ever be a single "pyparsing" module in my application? Of course I could just stop using the alias and use the pyparsing exceptions directly, but I feel a bit uneasy when a module just changes its identity at some point between imports.
Sebastian _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
This sounds like something you should open as a bug with setuptools, the problem lies with pkg_resources.extern:VendorImporter. Probably it should stop trying to be as tricky and do something more like what pip does here. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 09.05.2016, at 15:17, Donald Stufft <donald@stufft.io> wrote:
This sounds like something you should open as a bug with setuptools, the problem lies with pkg_resources.extern:VendorImporter. Probably it should stop trying to be as tricky and do something more like what pip does here.
Done: https://github.com/pypa/setuptools/issues/580 Sebastian
participants (2)
-
Donald Stufft -
Sebastian Krause