Re: [Python-Dev] If you shadow a module in the standard library that IDLE depends on, bad things happen
Laura, I think what you want should actually be more-or-less doable in IDLE. The main routine that starts IDLE should be able to detect if it starts correctly (something unlikely to happen if a significant stdlib module is shadowed), watch for an attribute error of that form and try to determine if shadowing is the cause, and if so, reissue a saner error message. The subprocess/firewall error is occurring because the ‘string’ problem in your example isn’t being hit right away so a few startup things already are happening. The point where we’re showing that error (as a result of a timeout) should be able to check as per the above that IDLE was able to start alright, and if not, change or ignore the timeout error. There’ll probably be some cases (depending on exactly what gets shadowed) that may be difficult to get to work, but it should be able to handle most things. Mark
In a message of Thu, 29 Oct 2015 13:26:08 -0700, Mark Roseman writes:
Laura, I think what you want should actually be more-or-less doable in IDLE.
The main routine that starts IDLE should be able to detect if it starts correctly (something unlikely to happen if a significant stdlib module is shadowed), watch for an attribute error of that form and try to determine if shadowing is the cause, and if so, reissue a saner error message.
The subprocess/firewall error is occurring because the ‘string’ problem in your example isn’t being hit right away so a few startup things already are happening. The point where we’re showing that error (as a result of a timeout) should be able to check as per the above that IDLE was able to start alright, and if not, change or ignore the timeout error.
There’ll probably be some cases (depending on exactly what gets shadowed) that may be difficult to get to work, but it should be able to handle most things.
Mark
Mark, how splendid. Need I submit a bug report/feature request to get this happening? Very, very pleased to have mentioned it ... Laura
Need I submit a bug report/feature request to get this happening? Very, very pleased to have mentioned it …
I took care of the bug report. Glad you mentioned it. My personal opinion is that many of the error messages in IDLE (to say nothing of large parts of the overall user experience) are a bit of a horror show, but it’s slowly fixable! :-) Don’t hesitate to share other snags, as the situation you’re in is what I’d like to see much better supported. Mark
On 10/29/2015 4:53 PM, Mark Roseman wrote:
Need I submit a bug report/feature request to get this happening? Very, very pleased to have mentioned it …
I took care of the bug report.
The idle issue is https://bugs.python.org/issue25514 As I said there, I think that removing '' from sys.path, so that IDLE can run, is better than a nicer warning that it cannot run. This, of course, requires that sys not be shadowed, so that sys.path can be accessed. -- Terry Jan Reedy
Why not just check the path of the imported modules and compare it with the Python library directory? On October 29, 2015 3:26:08 PM CDT, Mark Roseman <mark@markroseman.com> wrote:
Laura, I think what you want should actually be more-or-less doable in IDLE.
The main routine that starts IDLE should be able to detect if it starts correctly (something unlikely to happen if a significant stdlib module is shadowed), watch for an attribute error of that form and try to determine if shadowing is the cause, and if so, reissue a saner error message.
The subprocess/firewall error is occurring because the ‘string’ problem in your example isn’t being hit right away so a few startup things already are happening. The point where we’re showing that error (as a result of a timeout) should be able to check as per the above that IDLE was able to start alright, and if not, change or ignore the timeout error.
There’ll probably be some cases (depending on exactly what gets shadowed) that may be difficult to get to work, but it should be able to handle most things.
Mark
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
-- Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
In a message of Thu, 29 Oct 2015 15:50:30 -0500, Ryan Gonzalez writes:
Why not just check the path of the imported modules and compare it with the Python library directory?
My friend Åsa who is 12 years old suggested exactly this at the club. If this works then I will be certain to mention this to her. I said that I would ask 'how hard is this?' Laura
Well, tell your friend that that means middle and high schoolers must think alike! :D On Thu, Oct 29, 2015 at 4:18 PM, Laura Creighton <lac@openend.se> wrote:
Why not just check the path of the imported modules and compare it with
In a message of Thu, 29 Oct 2015 15:50:30 -0500, Ryan Gonzalez writes: the Python library directory?
My friend Åsa who is 12 years old suggested exactly this at the club. If this works then I will be certain to mention this to her. I said that I would ask 'how hard is this?'
Laura
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/
On 10/29/2015 5:18 PM, Laura Creighton wrote:
In a message of Thu, 29 Oct 2015 15:50:30 -0500, Ryan Gonzalez writes:
Why not just check the path of the imported modules and compare it with the Python library directory?
My friend Åsa who is 12 years old suggested exactly this at the club.
This was my first idea, until I realized that it would be even better to avoid shadowing in the first place.
If this works then I will be certain to mention this to her.
As far as I can tell, comparison in not foolproof, even if done carefully. This is a proper stdlib import.
import string string.__file__ 'C:\\Programs\\Python35\\lib\\string.py'
If we look at suffixes, the only part guaranteed, after changing Windows' '\\' to '/', is '/lib/string.py'. Now suppose someone runs python in another 'lib' directory containing string.py.
import string string.__file__ 'C:\\Users\\Terry\\lib\\string.py'
Same suffix. Let's try prefixes.
import os.path import sys os.path.dirname(string.__file__) in sys.path False
This is True for the stdlib import. Hooray. But this requires more imports, which also might be shadowed. Having '' at the front of sys.path is a real nuisance when one wants to guaranteed authentic stdlib imports. -- Terry Jan Reedy
On Thu, Oct 29, 2015 at 1:50 PM, Ryan Gonzalez <rymg19@gmail.com> wrote:
Why not just check the path of the imported modules and compare it with the Python library directory?
It works, but it requires that everyone who could run into this problem carefully add some extra guard code to every stdlib import statement, and in practice nobody will (or at least, not until after they've already gotten bitten by this at least once... at which point they no longer need it). Given that AFAICT there's no reason this couldn't be part of the default import system's functionality and "just work" for everyone, if I were going to spend time on trying to fix this I'd probably target that :-). (I guess the trickiest bit would be to find an efficient and maintainable way to check whether a given package name is present in the stdlib.) -n -- Nathaniel J. Smith -- http://vorpus.org
On Thu, 29 Oct 2015 16:56:38 -0700, Nathaniel Smith <njs@pobox.com> wrote:
On Thu, Oct 29, 2015 at 1:50 PM, Ryan Gonzalez <rymg19@gmail.com> wrote:
Why not just check the path of the imported modules and compare it with the Python library directory?
It works, but it requires that everyone who could run into this problem carefully add some extra guard code to every stdlib import statement, and in practice nobody will (or at least, not until after they've already gotten bitten by this at least once... at which point they no longer need it).
Given that AFAICT there's no reason this couldn't be part of the default import system's functionality and "just work" for everyone, if I were going to spend time on trying to fix this I'd probably target that :-).
(I guess the trickiest bit would be to find an efficient and maintainable way to check whether a given package name is present in the stdlib.)
For Idle, though, it sounds like a very viable strategy, and that's what Laura is concerned about. --David
Well, this works on Python 2 only (I'm on a phone with only access to 2 right now), but this is a start (apologies for the screwy syntax highlighting): import sys, imp, logging, os modules = ''' imp string ... '''.split() class StdlibTester(object): base = os.path.dirname(os.__file__) # hack; is there a better way to do this? def find_module(self, fullname, path=None): if fullname in modules: self.path = path return self return None def load_module(self, name): if name in sys.modules: return sys.modules[name] module_info = imp.find_module(name, self.path) module = imp.load_module(name, *module_info) sys.modules[name] = module if hasattr(module, '__file__') and not os.path.dirname(module.__file__).startswith(self.base): logging.warning('stdlib module %s was overriden', name) return module sys.meta_path.append(StdlibTester()) import string On October 29, 2015 7:06:51 PM CDT, "R. David Murray" <rdmurray@bitdance.com> wrote:
On Thu, 29 Oct 2015 16:56:38 -0700, Nathaniel Smith <njs@pobox.com> wrote:
On Thu, Oct 29, 2015 at 1:50 PM, Ryan Gonzalez <rymg19@gmail.com> wrote:
Why not just check the path of the imported modules and compare it with the Python library directory?
It works, but it requires that everyone who could run into this problem carefully add some extra guard code to every stdlib import statement, and in practice nobody will (or at least, not until after they've already gotten bitten by this at least once... at which point they no longer need it).
Given that AFAICT there's no reason this couldn't be part of the default import system's functionality and "just work" for everyone, if I were going to spend time on trying to fix this I'd probably target that :-).
(I guess the trickiest bit would be to find an efficient and maintainable way to check whether a given package name is present in the stdlib.)
For Idle, though, it sounds like a very viable strategy, and that's what Laura is concerned about.
--David _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
-- Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
participants (6)
-
Laura Creighton -
Mark Roseman -
Nathaniel Smith -
R. David Murray -
Ryan Gonzalez -
Terry Reedy