Providing fix for modulefinder.py regression on Windows
modulefinder.py does not open source files in "rb" which prevents compile() from applying the encoding rules. This first showed up for me on Windows with Python 3.8. I'd like to provide patches for 3.8 (if any more releases) and 3.9. What is the process I should follow? I have signed the Contributor Agreement. Here is my test case and the results on Windows with 3.7. import modulefinder with open('import_functools.py', 'w') as f: f.write('import functools\n') mf = modulefinder.ModuleFinder() mf.run_script('import_functools.py') print('Test passed') py -3.8-32 modulefinder_test.py Traceback (most recent call last): File "modulefinder_test.py", line 7, in <module> mf.run_script('import_functools.py') File "C:\Python38.win32\lib\modulefinder.py", line 165, in run_script self.load_module('__main__', fp, pathname, stuff) File "C:\Python38.win32\lib\modulefinder.py", line 360, in load_module self.scan_code(co, m) File "C:\Python38.win32\lib\modulefinder.py", line 433, in scan_code self._safe_import_hook(name, m, fromlist, level=0) File "C:\Python38.win32\lib\modulefinder.py", line 378, in _safe_import_hook self.import_hook(name, caller, level=level) File "C:\Python38.win32\lib\modulefinder.py", line 177, in import_hook q, tail = self.find_head_package(parent, name) File "C:\Python38.win32\lib\modulefinder.py", line 233, in find_head_package q = self.import_module(head, qname, parent) File "C:\Python38.win32\lib\modulefinder.py", line 326, in import_module m = self.load_module(fqname, fp, pathname, stuff) File "C:\Python38.win32\lib\modulefinder.py", line 343, in load_module co = compile(fp.read()+'\n', pathname, 'exec') File "C:\Python38.win32\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 308: character maps to <undefined> Barry
This seems like a noble pursuit. File a bug on bugs.python.org with the results you posted here, and submit a PR via GitHub ( github.com/python/cpython/). On Sat, Apr 11, 2020 at 9:09 AM Barry Scott <barry@barrys-emacs.org> wrote:
modulefinder.py does not open source files in "rb" which prevents compile() from applying the encoding rules.
This first showed up for me on Windows with Python 3.8.
I'd like to provide patches for 3.8 (if any more releases) and 3.9.
What is the process I should follow? I have signed the Contributor Agreement.
Here is my test case and the results on Windows with 3.7.
import modulefinder
with open('import_functools.py', 'w') as f: f.write('import functools\n')
mf = modulefinder.ModuleFinder() mf.run_script('import_functools.py') print('Test passed')
py -3.8-32 modulefinder_test.py Traceback (most recent call last): File "modulefinder_test.py", line 7, in <module> mf.run_script('import_functools.py') File "C:\Python38.win32\lib\modulefinder.py", line 165, in run_script self.load_module('__main__', fp, pathname, stuff) File "C:\Python38.win32\lib\modulefinder.py", line 360, in load_module self.scan_code(co, m) File "C:\Python38.win32\lib\modulefinder.py", line 433, in scan_code self._safe_import_hook(name, m, fromlist, level=0) File "C:\Python38.win32\lib\modulefinder.py", line 378, in _safe_import_hook self.import_hook(name, caller, level=level) File "C:\Python38.win32\lib\modulefinder.py", line 177, in import_hook q, tail = self.find_head_package(parent, name) File "C:\Python38.win32\lib\modulefinder.py", line 233, in find_head_package q = self.import_module(head, qname, parent) File "C:\Python38.win32\lib\modulefinder.py", line 326, in import_module m = self.load_module(fqname, fp, pathname, stuff) File "C:\Python38.win32\lib\modulefinder.py", line 343, in load_module co = compile(fp.read()+'\n', pathname, 'exec') File "C:\Python38.win32\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 308: character maps to <undefined>
Barry _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6IWG2SXT... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On 11 Apr 2020, at 16:28, Barry Scott <barry@barrys-emacs.org> wrote:
modulefinder.py does not open source files in "rb" which prevents compile() from applying the encoding rules.
snip I have created the bpo, the pr and have the checks passing except that I need a NEWS.d entry to make bedevere/news happy. I see the files have a special filename is that from an automatic process of a tool I need to run? Barry
Barry _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6IWG2SXT... Code of Conduct: http://python.org/psf/codeofconduct/
On 4/12/2020 4:08 PM, Barry Scott wrote:
On 11 Apr 2020, at 16:28, Barry Scott <barry@barrys-emacs.org <mailto:barry@barrys-emacs.org>> wrote:
modulefinder.py does not open source files in "rb" which prevents compile() from applying the encoding rules.
snip
*I have created the bpo, the pr and have the checks passing except that I need a NEWS.d entry* *to make bedevere/news happy. I see the files have a special filename is that from an automatic* *process of a tool I need to run?* * * *Barry*
The tool is blurb (or the online blurb-it): https://devguide.python.org/committing/#what-s-new-and-news-entries Eric
On 12 Apr 2020, at 21:49, Eric V. Smith <eric@trueblade.com> wrote:
On 4/12/2020 4:08 PM, Barry Scott wrote:
On 11 Apr 2020, at 16:28, Barry Scott <barry@barrys-emacs.org <mailto:barry@barrys-emacs.org>> wrote:
modulefinder.py does not open source files in "rb" which prevents compile() from applying the encoding rules.
snip
I have created the bpo, the pr and have the checks passing except that I need a NEWS.d entry to make bedevere/news happy. I see the files have a special filename is that from an automatic process of a tool I need to run?
Barry
The tool is blurb (or the online blurb-it): https://devguide.python.org/committing/#what-s-new-and-news-entries <https://devguide.python.org/committing/#what-s-new-and-news-entries>
I was reading this https://devguide.python.org/#contributing <https://devguide.python.org/#contributing> that does not mention blurb. I would not have though to look in the section on Accepting Pull Requests, as that not what I'm doing. Barry
Eric
Barry Scott wrote:
On 12 Apr 2020, at 21:49, Eric V. Smith eric@trueblade.com wrote: On 4/12/2020 4:08 PM, Barry Scott wrote: On 11 Apr 2020, at 16:28, Barry Scott <barry@barrys-emacs.org mailto:barry@barrys-emacs.org> wrote: modulefinder.py does not open source files in "rb" which prevents compile() from applying the encoding rules. snip I have created the bpo, the pr and have the checks passing except that I need a NEWS.d entry to make bedevere/news happy. I see the files have a special filename is that from an automatic process of a tool I need to run? Barry The tool is blurb (or the online blurb-it): https://devguide.python.org/committing/#what-s-new-and-news-entries https://devguide.python.org/committing/#what-s-new-and-news-entries I was reading this https://devguide.python.org/#contributing https://devguide.python.org/#contributing that does not mention blurb. I would not have though to look in the section on Accepting Pull Requests, as that not what I'm doing.
It's also available through the "Details" link of the failing status check on the PR. -Brett
Barry
Eric
Barry Scott created https://bugs.python.org/issue40260 and https://github.com/python/cpython/pull/19488. Victor Le dim. 12 avr. 2020 à 22:11, Barry Scott <barry@barrys-emacs.org> a écrit :
On 11 Apr 2020, at 16:28, Barry Scott <barry@barrys-emacs.org> wrote:
modulefinder.py does not open source files in "rb" which prevents compile() from applying the encoding rules.
snip
I have created the bpo, the pr and have the checks passing except that I need a NEWS.d entry to make bedevere/news happy. I see the files have a special filename is that from an automatic process of a tool I need to run?
Barry
Barry _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6IWG2SXT... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MATZ26CE... Code of Conduct: http://python.org/psf/codeofconduct/
-- Night gathers, and now my watch begins. It shall not end until my death.
participants (5)
-
Barry Scott -
Brett Cannon -
Eric V. Smith -
Guido van Rossum -
Victor Stinner