Python-checkins
Threads by month
- ----- 2024 -----
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2007 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2006 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2005 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2004 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2003 -----
- December
- November
- October
- September
- August
April 2021
- 1 participants
- 580 discussions
https://github.com/python/cpython/commit/b38b2fa0218911ccc20d576ff504f39c9c…
commit: b38b2fa0218911ccc20d576ff504f39c9c9d47ec
branch: master
author: Paul Moore <p.f.moore(a)gmail.com>
committer: jaraco <jaraco(a)jaraco.com>
date: 2021-04-28T19:27:37-04:00
summary:
Document importlib.metadata.PackagePath.locate method (GH-25669)
files:
M Doc/library/importlib.metadata.rst
diff --git a/Doc/library/importlib.metadata.rst b/Doc/library/importlib.metadata.rst
index 0f0a8dd91eea1..40e48d1beec0c 100644
--- a/Doc/library/importlib.metadata.rst
+++ b/Doc/library/importlib.metadata.rst
@@ -191,7 +191,7 @@ Distribution files
You can also get the full set of files contained within a distribution. The
``files()`` function takes a distribution package name and returns all of the
files installed by this distribution. Each file object returned is a
-``PackagePath``, a :class:`pathlib.Path` derived object with additional ``dist``,
+``PackagePath``, a :class:`pathlib.PurePath` derived object with additional ``dist``,
``size``, and ``hash`` properties as indicated by the metadata. For example::
>>> util = [p for p in files('wheel') if 'util.py' in str(p)][0] # doctest: +SKIP
@@ -215,6 +215,12 @@ Once you have the file, you can also read its contents::
return s.encode('utf-8')
return s
+You can also use the ``locate`` method to get a the absolute path to the
+file::
+
+ >>> util.locate() # doctest: +SKIP
+ PosixPath('/home/gustav/example/lib/site-packages/wheel/util.py')
+
In the case where the metadata file listing files
(RECORD or SOURCES.txt) is missing, ``files()`` will
return ``None``. The caller may wish to wrap calls to
1
0
bpo-43970: Optimize Path.cwd() in pathlib by not instantiating a class unnecessarily (GH-25699)
by ericvsmith 28 Apr '21
by ericvsmith 28 Apr '21
28 Apr '21
https://github.com/python/cpython/commit/4a85718212fd032c922ca7d630b2602dd4…
commit: 4a85718212fd032c922ca7d630b2602dd4b29a35
branch: master
author: kfollstad <kfollstad(a)gmail.com>
committer: ericvsmith <ericvsmith(a)users.noreply.github.com>
date: 2021-04-28T19:01:51-04:00
summary:
bpo-43970: Optimize Path.cwd() in pathlib by not instantiating a class unnecessarily (GH-25699)
files:
M Lib/pathlib.py
diff --git a/Lib/pathlib.py b/Lib/pathlib.py
index 073fce82ad570..cf40370c049a3 100644
--- a/Lib/pathlib.py
+++ b/Lib/pathlib.py
@@ -978,7 +978,7 @@ def cwd(cls):
"""Return a new path pointing to the current working directory
(as returned by os.getcwd()).
"""
- return cls(cls()._accessor.getcwd())
+ return cls(cls._accessor.getcwd())
@classmethod
def home(cls):
1
0
https://github.com/python/cpython/commit/15d386185659683fc044ccaa300aa8cd7d…
commit: 15d386185659683fc044ccaa300aa8cd7d49cc1a
branch: master
author: Tal Einat <532281+taleinat(a)users.noreply.github.com>
committer: terryjreedy <tjreedy(a)udel.edu>
date: 2021-04-28T18:27:55-04:00
summary:
bpo-37903: IDLE: Shell sidebar with prompts (GH-22682)
The first followup will change shell indents to spaces.
More are expected.
Co-authored-by: Terry Jan Reedy <tjreedy(a)udel.edu>
files:
A Lib/idlelib/idle_test/tkinter_testing_utils.py
A Misc/NEWS.d/next/IDLE/2019-08-24-23-49-36.bpo-37903.4xjast.rst
M Lib/idlelib/colorizer.py
M Lib/idlelib/editor.py
M Lib/idlelib/history.py
M Lib/idlelib/idle_test/test_editor.py
M Lib/idlelib/idle_test/test_pyshell.py
M Lib/idlelib/idle_test/test_sidebar.py
M Lib/idlelib/idle_test/test_squeezer.py
M Lib/idlelib/percolator.py
M Lib/idlelib/pyshell.py
M Lib/idlelib/replace.py
M Lib/idlelib/sidebar.py
M Lib/idlelib/squeezer.py
diff --git a/Lib/idlelib/colorizer.py b/Lib/idlelib/colorizer.py
index 0aae1778a580c0..3c527409731afa 100644
--- a/Lib/idlelib/colorizer.py
+++ b/Lib/idlelib/colorizer.py
@@ -133,7 +133,6 @@ def LoadTagDefs(self):
# non-modal alternative.
"hit": idleConf.GetHighlight(theme, "hit"),
}
-
if DEBUG: print('tagdefs', self.tagdefs)
def insert(self, index, chars, tags=None):
diff --git a/Lib/idlelib/editor.py b/Lib/idlelib/editor.py
index b9cb50264ff06f..8b544407da2e0d 100644
--- a/Lib/idlelib/editor.py
+++ b/Lib/idlelib/editor.py
@@ -60,7 +60,6 @@ class EditorWindow:
from idlelib.sidebar import LineNumbers
from idlelib.format import FormatParagraph, FormatRegion, Indents, Rstrip
from idlelib.parenmatch import ParenMatch
- from idlelib.squeezer import Squeezer
from idlelib.zoomheight import ZoomHeight
filesystemencoding = sys.getfilesystemencoding() # for file names
@@ -68,6 +67,7 @@ class EditorWindow:
allow_code_context = True
allow_line_numbers = True
+ user_input_insert_tags = None
def __init__(self, flist=None, filename=None, key=None, root=None):
# Delay import: runscript imports pyshell imports EditorWindow.
@@ -784,9 +784,7 @@ def _addcolorizer(self):
self.color = self.ColorDelegator()
# can add more colorizers here...
if self.color:
- self.per.removefilter(self.undo)
- self.per.insertfilter(self.color)
- self.per.insertfilter(self.undo)
+ self.per.insertfilterafter(filter=self.color, after=self.undo)
def _rmcolorizer(self):
if not self.color:
@@ -1303,8 +1301,6 @@ def smart_backspace_event(self, event):
# Debug prompt is multilined....
ncharsdeleted = 0
while 1:
- if chars == self.prompt_last_line: # '' unless PyShell
- break
chars = chars[:-1]
ncharsdeleted = ncharsdeleted + 1
have = len(chars.expandtabs(tabwidth))
@@ -1313,7 +1309,8 @@ def smart_backspace_event(self, event):
text.undo_block_start()
text.delete("insert-%dc" % ncharsdeleted, "insert")
if have < want:
- text.insert("insert", ' ' * (want - have))
+ text.insert("insert", ' ' * (want - have),
+ self.user_input_insert_tags)
text.undo_block_stop()
return "break"
@@ -1346,7 +1343,7 @@ def smart_indent_event(self, event):
effective = len(prefix.expandtabs(self.tabwidth))
n = self.indentwidth
pad = ' ' * (n - effective % n)
- text.insert("insert", pad)
+ text.insert("insert", pad, self.user_input_insert_tags)
text.see("insert")
return "break"
finally:
@@ -1377,13 +1374,14 @@ def newline_and_indent_event(self, event):
if i == n:
# The cursor is in or at leading indentation in a continuation
# line; just inject an empty line at the start.
- text.insert("insert linestart", '\n')
+ text.insert("insert linestart", '\n',
+ self.user_input_insert_tags)
return "break"
indent = line[:i]
# Strip whitespace before insert point unless it's in the prompt.
i = 0
- while line and line[-1] in " \t" and line != self.prompt_last_line:
+ while line and line[-1] in " \t":
line = line[:-1]
i += 1
if i:
@@ -1394,7 +1392,7 @@ def newline_and_indent_event(self, event):
text.delete("insert")
# Insert new line.
- text.insert("insert", '\n')
+ text.insert("insert", '\n', self.user_input_insert_tags)
# Adjust indentation for continuations and block open/close.
# First need to find the last statement.
@@ -1430,7 +1428,7 @@ def newline_and_indent_event(self, event):
elif c == pyparse.C_STRING_NEXT_LINES:
# Inside a string which started before this line;
# just mimic the current indent.
- text.insert("insert", indent)
+ text.insert("insert", indent, self.user_input_insert_tags)
elif c == pyparse.C_BRACKET:
# Line up with the first (if any) element of the
# last open bracket structure; else indent one
@@ -1444,7 +1442,8 @@ def newline_and_indent_event(self, event):
# beyond leftmost =; else to beyond first chunk of
# non-whitespace on initial line.
if y.get_num_lines_in_stmt() > 1:
- text.insert("insert", indent)
+ text.insert("insert", indent,
+ self.user_input_insert_tags)
else:
self.reindent_to(y.compute_backslash_indent())
else:
@@ -1455,7 +1454,7 @@ def newline_and_indent_event(self, event):
# indentation of initial line of closest preceding
# interesting statement.
indent = y.get_base_indent_string()
- text.insert("insert", indent)
+ text.insert("insert", indent, self.user_input_insert_tags)
if y.is_block_opener():
self.smart_indent_event(event)
elif indent and y.is_block_closer():
@@ -1502,7 +1501,8 @@ def reindent_to(self, column):
if text.compare("insert linestart", "!=", "insert"):
text.delete("insert linestart", "insert")
if column:
- text.insert("insert", self._make_blanks(column))
+ text.insert("insert", self._make_blanks(column),
+ self.user_input_insert_tags)
text.undo_block_stop()
# Guess indentwidth from text content.
diff --git a/Lib/idlelib/history.py b/Lib/idlelib/history.py
index ad44a96a9de2c0..7ce09253eff5c9 100644
--- a/Lib/idlelib/history.py
+++ b/Lib/idlelib/history.py
@@ -74,13 +74,13 @@ def fetch(self, reverse):
else:
if self.text.get("iomark", "end-1c") != prefix:
self.text.delete("iomark", "end-1c")
- self.text.insert("iomark", prefix)
+ self.text.insert("iomark", prefix, "stdin")
pointer = prefix = None
break
item = self.history[pointer]
if item[:nprefix] == prefix and len(item) > nprefix:
self.text.delete("iomark", "end-1c")
- self.text.insert("iomark", item)
+ self.text.insert("iomark", item, "stdin")
break
self.text.see("insert")
self.text.tag_remove("sel", "1.0", "end")
diff --git a/Lib/idlelib/idle_test/test_editor.py b/Lib/idlelib/idle_test/test_editor.py
index 443dcf021679fc..8665d680c0118f 100644
--- a/Lib/idlelib/idle_test/test_editor.py
+++ b/Lib/idlelib/idle_test/test_editor.py
@@ -167,7 +167,6 @@ def test_indent_and_newline_event(self):
'2.end'),
)
- w.prompt_last_line = ''
for test in tests:
with self.subTest(label=test.label):
insert(text, test.text)
@@ -182,13 +181,6 @@ def test_indent_and_newline_event(self):
# Deletes selected text before adding new line.
eq(get('1.0', 'end'), ' def f1(self, a,\n \n return a + b\n')
- # Preserves the whitespace in shell prompt.
- w.prompt_last_line = '>>> '
- insert(text, '>>> \t\ta =')
- text.mark_set('insert', '1.5')
- nl(None)
- eq(get('1.0', 'end'), '>>> \na =\n')
-
class RMenuTest(unittest.TestCase):
diff --git a/Lib/idlelib/idle_test/test_pyshell.py b/Lib/idlelib/idle_test/test_pyshell.py
index 4a096676f25796..706703965bffd6 100644
--- a/Lib/idlelib/idle_test/test_pyshell.py
+++ b/Lib/idlelib/idle_test/test_pyshell.py
@@ -60,5 +60,89 @@ def test_init(self):
## self.assertIsInstance(ps, pyshell.PyShell)
+class PyShellRemoveLastNewlineAndSurroundingWhitespaceTest(unittest.TestCase):
+ regexp = pyshell.PyShell._last_newline_re
+
+ def all_removed(self, text):
+ self.assertEqual('', self.regexp.sub('', text))
+
+ def none_removed(self, text):
+ self.assertEqual(text, self.regexp.sub('', text))
+
+ def check_result(self, text, expected):
+ self.assertEqual(expected, self.regexp.sub('', text))
+
+ def test_empty(self):
+ self.all_removed('')
+
+ def test_newline(self):
+ self.all_removed('\n')
+
+ def test_whitespace_no_newline(self):
+ self.all_removed(' ')
+ self.all_removed(' ')
+ self.all_removed(' ')
+ self.all_removed(' ' * 20)
+ self.all_removed('\t')
+ self.all_removed('\t\t')
+ self.all_removed('\t\t\t')
+ self.all_removed('\t' * 20)
+ self.all_removed('\t ')
+ self.all_removed(' \t')
+ self.all_removed(' \t \t ')
+ self.all_removed('\t \t \t')
+
+ def test_newline_with_whitespace(self):
+ self.all_removed(' \n')
+ self.all_removed('\t\n')
+ self.all_removed(' \t\n')
+ self.all_removed('\t \n')
+ self.all_removed('\n ')
+ self.all_removed('\n\t')
+ self.all_removed('\n \t')
+ self.all_removed('\n\t ')
+ self.all_removed(' \n ')
+ self.all_removed('\t\n ')
+ self.all_removed(' \n\t')
+ self.all_removed('\t\n\t')
+ self.all_removed('\t \t \t\n')
+ self.all_removed(' \t \t \n')
+ self.all_removed('\n\t \t \t')
+ self.all_removed('\n \t \t ')
+
+ def test_multiple_newlines(self):
+ self.check_result('\n\n', '\n')
+ self.check_result('\n' * 5, '\n' * 4)
+ self.check_result('\n' * 5 + '\t', '\n' * 4)
+ self.check_result('\n' * 20, '\n' * 19)
+ self.check_result('\n' * 20 + ' ', '\n' * 19)
+ self.check_result(' \n \n ', ' \n')
+ self.check_result(' \n\n ', ' \n')
+ self.check_result(' \n\n', ' \n')
+ self.check_result('\t\n\n', '\t\n')
+ self.check_result('\n\n ', '\n')
+ self.check_result('\n\n\t', '\n')
+ self.check_result(' \n \n ', ' \n')
+ self.check_result('\t\n\t\n\t', '\t\n')
+
+ def test_non_whitespace(self):
+ self.none_removed('a')
+ self.check_result('a\n', 'a')
+ self.check_result('a\n ', 'a')
+ self.check_result('a \n ', 'a')
+ self.check_result('a \n\t', 'a')
+ self.none_removed('-')
+ self.check_result('-\n', '-')
+ self.none_removed('.')
+ self.check_result('.\n', '.')
+
+ def test_unsupported_whitespace(self):
+ self.none_removed('\v')
+ self.none_removed('\n\v')
+ self.check_result('\v\n', '\v')
+ self.none_removed(' \n\v')
+ self.check_result('\v\n ', '\v')
+
+
if __name__ == '__main__':
unittest.main(verbosity=2)
diff --git a/Lib/idlelib/idle_test/test_sidebar.py b/Lib/idlelib/idle_test/test_sidebar.py
index 2974a9a7b09874..7228d0ee731fa5 100644
--- a/Lib/idlelib/idle_test/test_sidebar.py
+++ b/Lib/idlelib/idle_test/test_sidebar.py
@@ -1,13 +1,23 @@
-"""Test sidebar, coverage 93%"""
-import idlelib.sidebar
+"""Test sidebar, coverage 85%"""
+from textwrap import dedent
+import sys
+
from itertools import chain
import unittest
import unittest.mock
-from test.support import requires
+from test.support import requires, swap_attr
import tkinter as tk
+from .tkinter_testing_utils import run_in_tk_mainloop
from idlelib.delegator import Delegator
+from idlelib.editor import fixwordbreaks
+from idlelib import macosx
from idlelib.percolator import Percolator
+import idlelib.pyshell
+from idlelib.pyshell import fix_x11_paste, PyShell, PyShellFileList
+from idlelib.run import fix_scaling
+import idlelib.sidebar
+from idlelib.sidebar import get_end_linenumber, get_lineno
class Dummy_editwin:
@@ -31,6 +41,7 @@ class LineNumbersTest(unittest.TestCase):
def setUpClass(cls):
requires('gui')
cls.root = tk.Tk()
+ cls.root.withdraw()
cls.text_frame = tk.Frame(cls.root)
cls.text_frame.pack(side=tk.LEFT, fill=tk.BOTH, expand=True)
@@ -154,7 +165,7 @@ def test_delete(self):
self.assert_sidebar_n_lines(3)
self.assert_state_disabled()
- # Note: deleting up to "2.end" doesn't delete the final newline.
+ # Deleting up to "2.end" doesn't delete the final newline.
self.text.delete('2.0', '2.end')
self.assert_text_equals('fbarfoo\n\n\n')
self.assert_sidebar_n_lines(3)
@@ -165,7 +176,7 @@ def test_delete(self):
self.assert_sidebar_n_lines(1)
self.assert_state_disabled()
- # Note: Text widgets always keep a single '\n' character at the end.
+ # Text widgets always keep a single '\n' character at the end.
self.text.delete('1.0', 'end')
self.assert_text_equals('\n')
self.assert_sidebar_n_lines(1)
@@ -234,11 +245,19 @@ def get_width():
self.assert_sidebar_n_lines(4)
self.assertEqual(get_width(), 1)
- # Note: Text widgets always keep a single '\n' character at the end.
+ # Text widgets always keep a single '\n' character at the end.
self.text.delete('1.0', 'end -1c')
self.assert_sidebar_n_lines(1)
self.assertEqual(get_width(), 1)
+ # The following tests are temporarily disabled due to relying on
+ # simulated user input and inspecting which text is selected, which
+ # are fragile and can fail when several GUI tests are run in parallel
+ # or when the windows created by the test lose focus.
+ #
+ # TODO: Re-work these tests or remove them from the test suite.
+
+ @unittest.skip('test disabled')
def test_click_selection(self):
self.linenumber.show_sidebar()
self.text.insert('1.0', 'one\ntwo\nthree\nfour\n')
@@ -252,6 +271,7 @@ def test_click_selection(self):
self.assertEqual(self.get_selection(), ('2.0', '3.0'))
+ @unittest.skip('test disabled')
def simulate_drag(self, start_line, end_line):
start_x, start_y = self.get_line_screen_position(start_line)
end_x, end_y = self.get_line_screen_position(end_line)
@@ -277,6 +297,7 @@ def lerp(a, b, steps):
x=end_x, y=end_y)
self.root.update()
+ @unittest.skip('test disabled')
def test_drag_selection_down(self):
self.linenumber.show_sidebar()
self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n')
@@ -286,6 +307,7 @@ def test_drag_selection_down(self):
self.simulate_drag(2, 4)
self.assertEqual(self.get_selection(), ('2.0', '5.0'))
+ @unittest.skip('test disabled')
def test_drag_selection_up(self):
self.linenumber.show_sidebar()
self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n')
@@ -353,7 +375,7 @@ def assert_colors_are_equal(colors):
ln.hide_sidebar()
self.highlight_cfg = test_colors
- # Nothing breaks with inactive code context.
+ # Nothing breaks with inactive line numbers.
ln.update_colors()
# Show line numbers, previous colors change is immediately effective.
@@ -370,5 +392,319 @@ def assert_colors_are_equal(colors):
assert_colors_are_equal(orig_colors)
+class ShellSidebarTest(unittest.TestCase):
+ root: tk.Tk = None
+ shell: PyShell = None
+
+ @classmethod
+ def setUpClass(cls):
+ requires('gui')
+
+ cls.root = root = tk.Tk()
+ root.withdraw()
+
+ fix_scaling(root)
+ fixwordbreaks(root)
+ fix_x11_paste(root)
+
+ cls.flist = flist = PyShellFileList(root)
+ macosx.setupApp(root, flist)
+ root.update_idletasks()
+
+ cls.init_shell()
+
+ @classmethod
+ def tearDownClass(cls):
+ if cls.shell is not None:
+ cls.shell.executing = False
+ cls.shell.close()
+ cls.shell = None
+ cls.flist = None
+ cls.root.update_idletasks()
+ cls.root.destroy()
+ cls.root = None
+
+ @classmethod
+ def init_shell(cls):
+ cls.shell = cls.flist.open_shell()
+ cls.shell.pollinterval = 10
+ cls.root.update()
+ cls.n_preface_lines = get_lineno(cls.shell.text, 'end-1c') - 1
+
+ @classmethod
+ def reset_shell(cls):
+ cls.shell.per.bottom.delete(f'{cls.n_preface_lines+1}.0', 'end-1c')
+ cls.shell.shell_sidebar.update_sidebar()
+ cls.root.update()
+
+ def setUp(self):
+ # In some test environments, e.g. Azure Pipelines (as of
+ # Apr. 2021), sys.stdout is changed between tests. However,
+ # PyShell relies on overriding sys.stdout when run without a
+ # sub-process (as done here; see setUpClass).
+ self._saved_stdout = None
+ if sys.stdout != self.shell.stdout:
+ self._saved_stdout = sys.stdout
+ sys.stdout = self.shell.stdout
+
+ self.reset_shell()
+
+ def tearDown(self):
+ if self._saved_stdout is not None:
+ sys.stdout = self._saved_stdout
+
+ def get_sidebar_lines(self):
+ canvas = self.shell.shell_sidebar.canvas
+ texts = list(canvas.find(tk.ALL))
+ texts_by_y_coords = {
+ canvas.bbox(text)[1]: canvas.itemcget(text, 'text')
+ for text in texts
+ }
+ line_y_coords = self.get_shell_line_y_coords()
+ return [texts_by_y_coords.get(y, None) for y in line_y_coords]
+
+ def assert_sidebar_lines_end_with(self, expected_lines):
+ self.shell.shell_sidebar.update_sidebar()
+ self.assertEqual(
+ self.get_sidebar_lines()[-len(expected_lines):],
+ expected_lines,
+ )
+
+ def get_shell_line_y_coords(self):
+ text = self.shell.text
+ y_coords = []
+ index = text.index("@0,0")
+ if index.split('.', 1)[1] != '0':
+ index = text.index(f"{index} +1line linestart")
+ while True:
+ lineinfo = text.dlineinfo(index)
+ if lineinfo is None:
+ break
+ y_coords.append(lineinfo[1])
+ index = text.index(f"{index} +1line")
+ return y_coords
+
+ def get_sidebar_line_y_coords(self):
+ canvas = self.shell.shell_sidebar.canvas
+ texts = list(canvas.find(tk.ALL))
+ texts.sort(key=lambda text: canvas.bbox(text)[1])
+ return [canvas.bbox(text)[1] for text in texts]
+
+ def assert_sidebar_lines_synced(self):
+ self.assertLessEqual(
+ set(self.get_sidebar_line_y_coords()),
+ set(self.get_shell_line_y_coords()),
+ )
+
+ def do_input(self, input):
+ shell = self.shell
+ text = shell.text
+ for line_index, line in enumerate(input.split('\n')):
+ if line_index > 0:
+ text.event_generate('<<newline-and-indent>>')
+ text.insert('insert', line, 'stdin')
+
+ def test_initial_state(self):
+ sidebar_lines = self.get_sidebar_lines()
+ self.assertEqual(
+ sidebar_lines,
+ [None] * (len(sidebar_lines) - 1) + ['>>>'],
+ )
+ self.assert_sidebar_lines_synced()
+
+ @run_in_tk_mainloop
+ def test_single_empty_input(self):
+ self.do_input('\n')
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', '>>>'])
+
+ @run_in_tk_mainloop
+ def test_single_line_statement(self):
+ self.do_input('1\n')
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', None, '>>>'])
+
+ @run_in_tk_mainloop
+ def test_multi_line_statement(self):
+ # Block statements are not indented because IDLE auto-indents.
+ self.do_input(dedent('''\
+ if True:
+ print(1)
+
+ '''))
+ yield
+ self.assert_sidebar_lines_end_with([
+ '>>>',
+ '...',
+ '...',
+ '...',
+ None,
+ '>>>',
+ ])
+
+ @run_in_tk_mainloop
+ def test_single_long_line_wraps(self):
+ self.do_input('1' * 200 + '\n')
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', None, '>>>'])
+ self.assert_sidebar_lines_synced()
+
+ @run_in_tk_mainloop
+ def test_squeeze_multi_line_output(self):
+ shell = self.shell
+ text = shell.text
+
+ self.do_input('print("a\\nb\\nc")\n')
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', None, None, None, '>>>'])
+
+ text.mark_set('insert', f'insert -1line linestart')
+ text.event_generate('<<squeeze-current-text>>')
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', None, '>>>'])
+ self.assert_sidebar_lines_synced()
+
+ shell.squeezer.expandingbuttons[0].expand()
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', None, None, None, '>>>'])
+ self.assert_sidebar_lines_synced()
+
+ @run_in_tk_mainloop
+ def test_interrupt_recall_undo_redo(self):
+ text = self.shell.text
+ # Block statements are not indented because IDLE auto-indents.
+ initial_sidebar_lines = self.get_sidebar_lines()
+
+ self.do_input(dedent('''\
+ if True:
+ print(1)
+ '''))
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', '...', '...'])
+ with_block_sidebar_lines = self.get_sidebar_lines()
+ self.assertNotEqual(with_block_sidebar_lines, initial_sidebar_lines)
+
+ # Control-C
+ text.event_generate('<<interrupt-execution>>')
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', '...', '...', None, '>>>'])
+
+ # Recall previous via history
+ text.event_generate('<<history-previous>>')
+ text.event_generate('<<interrupt-execution>>')
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', '...', None, '>>>'])
+
+ # Recall previous via recall
+ text.mark_set('insert', text.index('insert -2l'))
+ text.event_generate('<<newline-and-indent>>')
+ yield
+
+ text.event_generate('<<undo>>')
+ yield
+ self.assert_sidebar_lines_end_with(['>>>'])
+
+ text.event_generate('<<redo>>')
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', '...'])
+
+ text.event_generate('<<newline-and-indent>>')
+ text.event_generate('<<newline-and-indent>>')
+ yield
+ self.assert_sidebar_lines_end_with(
+ ['>>>', '...', '...', '...', None, '>>>']
+ )
+
+ @run_in_tk_mainloop
+ def test_very_long_wrapped_line(self):
+ with swap_attr(self.shell, 'squeezer', None):
+ self.do_input('x = ' + '1'*10_000 + '\n')
+ yield
+ self.assertEqual(self.get_sidebar_lines(), ['>>>'])
+
+ def test_font(self):
+ sidebar = self.shell.shell_sidebar
+
+ test_font = 'TkTextFont'
+
+ def mock_idleconf_GetFont(root, configType, section):
+ return test_font
+ GetFont_patcher = unittest.mock.patch.object(
+ idlelib.sidebar.idleConf, 'GetFont', mock_idleconf_GetFont)
+ GetFont_patcher.start()
+ def cleanup():
+ GetFont_patcher.stop()
+ sidebar.update_font()
+ self.addCleanup(cleanup)
+
+ def get_sidebar_font():
+ canvas = sidebar.canvas
+ texts = list(canvas.find(tk.ALL))
+ fonts = {canvas.itemcget(text, 'font') for text in texts}
+ self.assertEqual(len(fonts), 1)
+ return next(iter(fonts))
+
+ self.assertNotEqual(get_sidebar_font(), test_font)
+ sidebar.update_font()
+ self.assertEqual(get_sidebar_font(), test_font)
+
+ def test_highlight_colors(self):
+ sidebar = self.shell.shell_sidebar
+
+ test_colors = {"background": '#abcdef', "foreground": '#123456'}
+
+ orig_idleConf_GetHighlight = idlelib.sidebar.idleConf.GetHighlight
+ def mock_idleconf_GetHighlight(theme, element):
+ if element in ['linenumber', 'console']:
+ return test_colors
+ return orig_idleConf_GetHighlight(theme, element)
+ GetHighlight_patcher = unittest.mock.patch.object(
+ idlelib.sidebar.idleConf, 'GetHighlight',
+ mock_idleconf_GetHighlight)
+ GetHighlight_patcher.start()
+ def cleanup():
+ GetHighlight_patcher.stop()
+ sidebar.update_colors()
+ self.addCleanup(cleanup)
+
+ def get_sidebar_colors():
+ canvas = sidebar.canvas
+ texts = list(canvas.find(tk.ALL))
+ fgs = {canvas.itemcget(text, 'fill') for text in texts}
+ self.assertEqual(len(fgs), 1)
+ fg = next(iter(fgs))
+ bg = canvas.cget('background')
+ return {"background": bg, "foreground": fg}
+
+ self.assertNotEqual(get_sidebar_colors(), test_colors)
+ sidebar.update_colors()
+ self.assertEqual(get_sidebar_colors(), test_colors)
+
+ @run_in_tk_mainloop
+ def test_mousewheel(self):
+ sidebar = self.shell.shell_sidebar
+ text = self.shell.text
+
+ # Enter a 100-line string to scroll the shell screen down.
+ self.do_input('x = """' + '\n'*100 + '"""\n')
+ yield
+ self.assertGreater(get_lineno(text, '@0,0'), 1)
+
+ last_lineno = get_end_linenumber(text)
+ self.assertIsNotNone(text.dlineinfo(text.index(f'{last_lineno}.0')))
+
+ # Scroll up using the <MouseWheel> event.
+ # The meaning delta is platform-dependant.
+ delta = -1 if sys.platform == 'darwin' else 120
+ sidebar.canvas.event_generate('<MouseWheel>', x=0, y=0, delta=delta)
+ yield
+ self.assertIsNone(text.dlineinfo(text.index(f'{last_lineno}.0')))
+
+ # Scroll back down using the <Button-5> event.
+ sidebar.canvas.event_generate('<Button-5>', x=0, y=0)
+ yield
+ self.assertIsNotNone(text.dlineinfo(text.index(f'{last_lineno}.0')))
+
+
if __name__ == '__main__':
unittest.main(verbosity=2)
diff --git a/Lib/idlelib/idle_test/test_squeezer.py b/Lib/idlelib/idle_test/test_squeezer.py
index ee1bbd76b50562..eaf81a5fc1a053 100644
--- a/Lib/idlelib/idle_test/test_squeezer.py
+++ b/Lib/idlelib/idle_test/test_squeezer.py
@@ -7,13 +7,12 @@
from test.support import requires
from idlelib.config import idleConf
+from idlelib.percolator import Percolator
from idlelib.squeezer import count_lines_with_wrapping, ExpandingButton, \
Squeezer
from idlelib import macosx
from idlelib.textview import view_text
from idlelib.tooltip import Hovertip
-from idlelib.pyshell import PyShell
-
SENTINEL_VALUE = sentinel.SENTINEL_VALUE
@@ -205,8 +204,8 @@ def test_auto_squeeze(self):
self.assertEqual(text_widget.get('1.0', 'end'), '\n')
self.assertEqual(len(squeezer.expandingbuttons), 1)
- def test_squeeze_current_text_event(self):
- """Test the squeeze_current_text event."""
+ def test_squeeze_current_text(self):
+ """Test the squeeze_current_text method."""
# Squeezing text should work for both stdout and stderr.
for tag_name in ["stdout", "stderr"]:
editwin = self.make_mock_editor_window(with_text_widget=True)
@@ -222,7 +221,7 @@ def test_squeeze_current_text_event(self):
self.assertEqual(len(squeezer.expandingbuttons), 0)
# Test squeezing the current text.
- retval = squeezer.squeeze_current_text_event(event=Mock())
+ retval = squeezer.squeeze_current_text()
self.assertEqual(retval, "break")
self.assertEqual(text_widget.get('1.0', 'end'), '\n\n')
self.assertEqual(len(squeezer.expandingbuttons), 1)
@@ -230,11 +229,11 @@ def test_squeeze_current_text_event(self):
# Test that expanding the squeezed text works and afterwards
# the Text widget contains the original text.
- squeezer.expandingbuttons[0].expand(event=Mock())
+ squeezer.expandingbuttons[0].expand()
self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n')
self.assertEqual(len(squeezer.expandingbuttons), 0)
- def test_squeeze_current_text_event_no_allowed_tags(self):
+ def test_squeeze_current_text_no_allowed_tags(self):
"""Test that the event doesn't squeeze text without a relevant tag."""
editwin = self.make_mock_editor_window(with_text_widget=True)
text_widget = editwin.text
@@ -249,7 +248,7 @@ def test_squeeze_current_text_event_no_allowed_tags(self):
self.assertEqual(len(squeezer.expandingbuttons), 0)
# Test squeezing the current text.
- retval = squeezer.squeeze_current_text_event(event=Mock())
+ retval = squeezer.squeeze_current_text()
self.assertEqual(retval, "break")
self.assertEqual(text_widget.get('1.0', 'end'), 'SOME\nTEXT\n\n')
self.assertEqual(len(squeezer.expandingbuttons), 0)
@@ -264,13 +263,13 @@ def test_squeeze_text_before_existing_squeezed_text(self):
# Prepare some text in the Text widget and squeeze it.
text_widget.insert("1.0", "SOME\nTEXT\n", "stdout")
text_widget.mark_set("insert", "1.0")
- squeezer.squeeze_current_text_event(event=Mock())
+ squeezer.squeeze_current_text()
self.assertEqual(len(squeezer.expandingbuttons), 1)
# Test squeezing the current text.
text_widget.insert("1.0", "MORE\nSTUFF\n", "stdout")
text_widget.mark_set("insert", "1.0")
- retval = squeezer.squeeze_current_text_event(event=Mock())
+ retval = squeezer.squeeze_current_text()
self.assertEqual(retval, "break")
self.assertEqual(text_widget.get('1.0', 'end'), '\n\n\n')
self.assertEqual(len(squeezer.expandingbuttons), 2)
@@ -311,6 +310,7 @@ def make_mock_squeezer(self):
root = get_test_tk_root(self)
squeezer = Mock()
squeezer.editwin.text = Text(root)
+ squeezer.editwin.per = Percolator(squeezer.editwin.text)
# Set default values for the configuration settings.
squeezer.auto_squeeze_min_lines = 50
@@ -352,14 +352,9 @@ def test_expand(self):
# Insert the button into the text widget
# (this is normally done by the Squeezer class).
- text_widget = expandingbutton.text
+ text_widget = squeezer.editwin.text
text_widget.window_create("1.0", window=expandingbutton)
- # Set base_text to the text widget, so that changes are actually
- # made to it (by ExpandingButton) and we can inspect these
- # changes afterwards.
- expandingbutton.base_text = expandingbutton.text
-
# trigger the expand event
retval = expandingbutton.expand(event=Mock())
self.assertEqual(retval, None)
@@ -390,11 +385,6 @@ def test_expand_dangerous_oupput(self):
text_widget = expandingbutton.text
text_widget.window_create("1.0", window=expandingbutton)
- # Set base_text to the text widget, so that changes are actually
- # made to it (by ExpandingButton) and we can inspect these
- # changes afterwards.
- expandingbutton.base_text = expandingbutton.text
-
# Patch the message box module to always return False.
with patch('idlelib.squeezer.messagebox') as mock_msgbox:
mock_msgbox.askokcancel.return_value = False
diff --git a/Lib/idlelib/idle_test/tkinter_testing_utils.py b/Lib/idlelib/idle_test/tkinter_testing_utils.py
new file mode 100644
index 00000000000000..a9f8386e2cd9f6
--- /dev/null
+++ b/Lib/idlelib/idle_test/tkinter_testing_utils.py
@@ -0,0 +1,56 @@
+"""Utilities for testing with Tkinter"""
+import functools
+
+
+def run_in_tk_mainloop(test_method):
+ """Decorator for running a test method with a real Tk mainloop.
+
+ This starts a Tk mainloop before running the test, and stops it
+ at the end. This is faster and more robust than the common
+ alternative method of calling .update() and/or .update_idletasks().
+
+ Test methods using this must be written as generator functions,
+ using "yield" to allow the mainloop to process events and "after"
+ callbacks, and then continue the test from that point.
+
+ This also assumes that the test class has a .root attribute,
+ which is a tkinter.Tk object.
+
+ For example (from test_sidebar.py):
+
+ @run_test_with_tk_mainloop
+ def test_single_empty_input(self):
+ self.do_input('\n')
+ yield
+ self.assert_sidebar_lines_end_with(['>>>', '>>>'])
+ """
+ @functools.wraps(test_method)
+ def new_test_method(self):
+ test_generator = test_method(self)
+ root = self.root
+ # Exceptions raised by self.assert...() need to be raised
+ # outside of the after() callback in order for the test
+ # harness to capture them.
+ exception = None
+ def after_callback():
+ nonlocal exception
+ try:
+ next(test_generator)
+ except StopIteration:
+ root.quit()
+ except Exception as exc:
+ exception = exc
+ root.quit()
+ else:
+ # Schedule the Tk mainloop to call this function again,
+ # using a robust method of ensuring that it gets a
+ # chance to process queued events before doing so.
+ # See: https://stackoverflow.com/q/18499082#comment65004099_38817470
+ root.after(1, root.after_idle, after_callback)
+ root.after(0, root.after_idle, after_callback)
+ root.mainloop()
+
+ if exception:
+ raise exception
+
+ return new_test_method
diff --git a/Lib/idlelib/percolator.py b/Lib/idlelib/percolator.py
index db70304f589159..1fe34d29f54eb2 100644
--- a/Lib/idlelib/percolator.py
+++ b/Lib/idlelib/percolator.py
@@ -38,6 +38,21 @@ def insertfilter(self, filter):
filter.setdelegate(self.top)
self.top = filter
+ def insertfilterafter(self, filter, after):
+ assert isinstance(filter, Delegator)
+ assert isinstance(after, Delegator)
+ assert filter.delegate is None
+
+ f = self.top
+ f.resetcache()
+ while f is not after:
+ assert f is not self.bottom
+ f = f.delegate
+ f.resetcache()
+
+ filter.setdelegate(f.delegate)
+ f.setdelegate(filter)
+
def removefilter(self, filter):
# XXX Perhaps should only support popfilter()?
assert isinstance(filter, Delegator)
diff --git a/Lib/idlelib/pyshell.py b/Lib/idlelib/pyshell.py
index 0ee2254807fe8f..5830b7aa31a72c 100755
--- a/Lib/idlelib/pyshell.py
+++ b/Lib/idlelib/pyshell.py
@@ -48,15 +48,20 @@
from idlelib.colorizer import ColorDelegator
from idlelib.config import idleConf
+from idlelib.delegator import Delegator
from idlelib import debugger
from idlelib import debugger_r
from idlelib.editor import EditorWindow, fixwordbreaks
from idlelib.filelist import FileList
from idlelib.outwin import OutputWindow
+from idlelib import replace
from idlelib import rpc
from idlelib.run import idle_formatwarning, StdInputFile, StdOutputFile
from idlelib.undo import UndoDelegator
+# Default for testing; defaults to True in main() for running.
+use_subprocess = False
+
HOST = '127.0.0.1' # python execution server on localhost loopback
PORT = 0 # someday pass in host, port for remote debug capability
@@ -335,34 +340,19 @@ def open_shell(self, event=None):
class ModifiedColorDelegator(ColorDelegator):
"Extend base class: colorizer for the shell window itself"
-
- def __init__(self):
- ColorDelegator.__init__(self)
- self.LoadTagDefs()
-
def recolorize_main(self):
self.tag_remove("TODO", "1.0", "iomark")
self.tag_add("SYNC", "1.0", "iomark")
ColorDelegator.recolorize_main(self)
- def LoadTagDefs(self):
- ColorDelegator.LoadTagDefs(self)
- theme = idleConf.CurrentTheme()
- self.tagdefs.update({
- "stdin": {'background':None,'foreground':None},
- "stdout": idleConf.GetHighlight(theme, "stdout"),
- "stderr": idleConf.GetHighlight(theme, "stderr"),
- "console": idleConf.GetHighlight(theme, "console"),
- })
-
def removecolors(self):
# Don't remove shell color tags before "iomark"
for tag in self.tagdefs:
self.tag_remove(tag, "iomark", "end")
+
class ModifiedUndoDelegator(UndoDelegator):
"Extend base class: forbid insert/delete before the I/O mark"
-
def insert(self, index, chars, tags=None):
try:
if self.delegate.compare(index, "<", "iomark"):
@@ -381,6 +371,27 @@ def delete(self, index1, index2=None):
pass
UndoDelegator.delete(self, index1, index2)
+ def undo_event(self, event):
+ # Temporarily monkey-patch the delegate's .insert() method to
+ # always use the "stdin" tag. This is needed for undo-ing
+ # deletions to preserve the "stdin" tag, because UndoDelegator
+ # doesn't preserve tags for deleted text.
+ orig_insert = self.delegate.insert
+ self.delegate.insert = \
+ lambda index, chars: orig_insert(index, chars, "stdin")
+ try:
+ super().undo_event(event)
+ finally:
+ self.delegate.insert = orig_insert
+
+
+class UserInputTaggingDelegator(Delegator):
+ """Delegator used to tag user input with "stdin"."""
+ def insert(self, index, chars, tags=None):
+ if tags is None:
+ tags = "stdin"
+ self.delegate.insert(index, chars, tags)
+
class MyRPCClient(rpc.RPCClient):
@@ -832,6 +843,7 @@ def display_executing_dialog(self):
class PyShell(OutputWindow):
+ from idlelib.squeezer import Squeezer
shell_title = "IDLE Shell " + python_version()
@@ -855,9 +867,11 @@ class PyShell(OutputWindow):
]
allow_line_numbers = False
+ user_input_insert_tags = "stdin"
# New classes
from idlelib.history import History
+ from idlelib.sidebar import ShellSidebar
def __init__(self, flist=None):
if use_subprocess:
@@ -871,6 +885,8 @@ def __init__(self, flist=None):
root.withdraw()
flist = PyShellFileList(root)
+ self.shell_sidebar = None # initialized below
+
OutputWindow.__init__(self, flist, None, None)
self.usetabs = True
@@ -893,9 +909,9 @@ def __init__(self, flist=None):
if use_subprocess:
text.bind("<<view-restart>>", self.view_restart_mark)
text.bind("<<restart-shell>>", self.restart_shell)
- squeezer = self.Squeezer(self)
+ self.squeezer = self.Squeezer(self)
text.bind("<<squeeze-current-text>>",
- squeezer.squeeze_current_text_event)
+ self.squeeze_current_text_event)
self.save_stdout = sys.stdout
self.save_stderr = sys.stderr
@@ -926,6 +942,40 @@ def __init__(self, flist=None):
#
self.pollinterval = 50 # millisec
+ self.shell_sidebar = self.ShellSidebar(self)
+
+ # Insert UserInputTaggingDelegator at the top of the percolator,
+ # but make calls to text.insert() skip it. This causes only insert
+ # events generated in Tcl/Tk to go through this delegator.
+ self.text.insert = self.per.top.insert
+ self.per.insertfilter(UserInputTaggingDelegator())
+
+ def ResetFont(self):
+ super().ResetFont()
+
+ if self.shell_sidebar is not None:
+ self.shell_sidebar.update_font()
+
+ def ResetColorizer(self):
+ super().ResetColorizer()
+
+ theme = idleConf.CurrentTheme()
+ tag_colors = {
+ "stdin": {'background': None, 'foreground': None},
+ "stdout": idleConf.GetHighlight(theme, "stdout"),
+ "stderr": idleConf.GetHighlight(theme, "stderr"),
+ "console": idleConf.GetHighlight(theme, "normal"),
+ }
+ for tag, tag_colors_config in tag_colors.items():
+ self.text.tag_configure(tag, **tag_colors_config)
+
+ if self.shell_sidebar is not None:
+ self.shell_sidebar.update_colors()
+
+ def replace_event(self, event):
+ replace.replace(self.text, insert_tags="stdin")
+ return "break"
+
def get_standard_extension_names(self):
return idleConf.GetExtensions(shell_only=True)
@@ -1166,13 +1216,30 @@ def enter_callback(self, event):
# the current line, less a leading prompt, less leading or
# trailing whitespace
if self.text.compare("insert", "<", "iomark linestart"):
- # Check if there's a relevant stdin range -- if so, use it
+ # Check if there's a relevant stdin range -- if so, use it.
+ # Note: "stdin" blocks may include several successive statements,
+ # so look for "console" tags on the newline before each statement
+ # (and possibly on prompts).
prev = self.text.tag_prevrange("stdin", "insert")
- if prev and self.text.compare("insert", "<", prev[1]):
+ if (
+ prev and
+ self.text.compare("insert", "<", prev[1]) and
+ # The following is needed to handle empty statements.
+ "console" not in self.text.tag_names("insert")
+ ):
+ prev_cons = self.text.tag_prevrange("console", "insert")
+ if prev_cons and self.text.compare(prev_cons[1], ">=", prev[0]):
+ prev = (prev_cons[1], prev[1])
+ next_cons = self.text.tag_nextrange("console", "insert")
+ if next_cons and self.text.compare(next_cons[0], "<", prev[1]):
+ prev = (prev[0], self.text.index(next_cons[0] + "+1c"))
self.recall(self.text.get(prev[0], prev[1]), event)
return "break"
next = self.text.tag_nextrange("stdin", "insert")
if next and self.text.compare("insert lineend", ">=", next[0]):
+ next_cons = self.text.tag_nextrange("console", "insert lineend")
+ if next_cons and self.text.compare(next_cons[0], "<", next[1]):
+ next = (next[0], self.text.index(next_cons[0] + "+1c"))
self.recall(self.text.get(next[0], next[1]), event)
return "break"
# No stdin mark -- just get the current line, less any prompt
@@ -1204,7 +1271,6 @@ def enter_callback(self, event):
self.text.see("insert")
else:
self.newline_and_indent_event(event)
- self.text.tag_add("stdin", "iomark", "end-1c")
self.text.update_idletasks()
if self.reading:
self.top.quit() # Break out of recursive mainloop()
@@ -1214,7 +1280,7 @@ def enter_callback(self, event):
def recall(self, s, event):
# remove leading and trailing empty or whitespace lines
- s = re.sub(r'^\s*\n', '' , s)
+ s = re.sub(r'^\s*\n', '', s)
s = re.sub(r'\n\s*$', '', s)
lines = s.split('\n')
self.text.undo_block_start()
@@ -1225,7 +1291,8 @@ def recall(self, s, event):
if prefix.rstrip().endswith(':'):
self.newline_and_indent_event(event)
prefix = self.text.get("insert linestart", "insert")
- self.text.insert("insert", lines[0].strip())
+ self.text.insert("insert", lines[0].strip(),
+ self.user_input_insert_tags)
if len(lines) > 1:
orig_base_indent = re.search(r'^([ \t]*)', lines[0]).group(0)
new_base_indent = re.search(r'^([ \t]*)', prefix).group(0)
@@ -1233,24 +1300,24 @@ def recall(self, s, event):
if line.startswith(orig_base_indent):
# replace orig base indentation with new indentation
line = new_base_indent + line[len(orig_base_indent):]
- self.text.insert('insert', '\n'+line.rstrip())
+ self.text.insert('insert', '\n' + line.rstrip(),
+ self.user_input_insert_tags)
finally:
self.text.see("insert")
self.text.undo_block_stop()
+ _last_newline_re = re.compile(r"[ \t]*(\n[ \t]*)?\Z")
def runit(self):
+ index_before = self.text.index("end-2c")
line = self.text.get("iomark", "end-1c")
# Strip off last newline and surrounding whitespace.
# (To allow you to hit return twice to end a statement.)
- i = len(line)
- while i > 0 and line[i-1] in " \t":
- i = i-1
- if i > 0 and line[i-1] == "\n":
- i = i-1
- while i > 0 and line[i-1] in " \t":
- i = i-1
- line = line[:i]
- self.interp.runsource(line)
+ line = self._last_newline_re.sub("", line)
+ input_is_complete = self.interp.runsource(line)
+ if not input_is_complete:
+ if self.text.get(index_before) == '\n':
+ self.text.tag_remove(self.user_input_insert_tags, index_before)
+ self.shell_sidebar.update_sidebar()
def open_stack_viewer(self, event=None):
if self.interp.rpcclt:
@@ -1276,7 +1343,14 @@ def restart_shell(self, event=None):
def showprompt(self):
self.resetoutput()
- self.console.write(self.prompt)
+
+ prompt = self.prompt
+ if self.sys_ps1 and prompt.endswith(self.sys_ps1):
+ prompt = prompt[:-len(self.sys_ps1)]
+ self.text.tag_add("console", "iomark-1c")
+ self.console.write(prompt)
+
+ self.shell_sidebar.update_sidebar()
self.text.mark_set("insert", "end-1c")
self.set_line_and_column()
self.io.reset_undo()
@@ -1326,6 +1400,13 @@ def rmenu_check_paste(self):
return 'disabled'
return super().rmenu_check_paste()
+ def squeeze_current_text_event(self, event=None):
+ self.squeezer.squeeze_current_text()
+ self.shell_sidebar.update_sidebar()
+
+ def on_squeezed_expand(self, index, text, tags):
+ self.shell_sidebar.update_sidebar()
+
def fix_x11_paste(root):
"Make paste replace selection on x11. See issue #5124."
diff --git a/Lib/idlelib/replace.py b/Lib/idlelib/replace.py
index 6be034af9626b3..2f9ca231a05e49 100644
--- a/Lib/idlelib/replace.py
+++ b/Lib/idlelib/replace.py
@@ -11,7 +11,7 @@
from idlelib import searchengine
-def replace(text):
+def replace(text, insert_tags=None):
"""Create or reuse a singleton ReplaceDialog instance.
The singleton dialog saves user entries and preferences
@@ -25,7 +25,7 @@ def replace(text):
if not hasattr(engine, "_replacedialog"):
engine._replacedialog = ReplaceDialog(root, engine)
dialog = engine._replacedialog
- dialog.open(text)
+ dialog.open(text, insert_tags=insert_tags)
class ReplaceDialog(SearchDialogBase):
@@ -49,8 +49,9 @@ def __init__(self, root, engine):
"""
super().__init__(root, engine)
self.replvar = StringVar(root)
+ self.insert_tags = None
- def open(self, text):
+ def open(self, text, insert_tags=None):
"""Make dialog visible on top of others and ready to use.
Also, highlight the currently selected text and set the
@@ -72,6 +73,7 @@ def open(self, text):
last = last or first
self.show_hit(first, last)
self.ok = True
+ self.insert_tags = insert_tags
def create_entries(self):
"Create base and additional label and text entry widgets."
@@ -177,7 +179,7 @@ def replace_all(self, event=None):
if first != last:
text.delete(first, last)
if new:
- text.insert(first, new)
+ text.insert(first, new, self.insert_tags)
col = i + len(new)
ok = False
text.undo_block_stop()
@@ -231,7 +233,7 @@ def do_replace(self):
if m.group():
text.delete(first, last)
if new:
- text.insert(first, new)
+ text.insert(first, new, self.insert_tags)
text.undo_block_stop()
self.show_hit(first, text.index("insert"))
self.ok = False
@@ -264,6 +266,7 @@ def close(self, event=None):
"Close the dialog and remove hit tags."
SearchDialogBase.close(self, event)
self.text.tag_remove("hit", "1.0", "end")
+ self.insert_tags = None
def _replace_dialog(parent): # htest #
diff --git a/Lib/idlelib/sidebar.py b/Lib/idlelib/sidebar.py
index 41c09684a20251..a947961b858d68 100644
--- a/Lib/idlelib/sidebar.py
+++ b/Lib/idlelib/sidebar.py
@@ -1,19 +1,33 @@
"""Line numbering implementation for IDLE as an extension.
Includes BaseSideBar which can be extended for other sidebar based extensions
"""
+import contextlib
import functools
import itertools
import tkinter as tk
+from tkinter.font import Font
from idlelib.config import idleConf
from idlelib.delegator import Delegator
+def get_lineno(text, index):
+ """Return the line number of an index in a Tk text widget."""
+ return int(float(text.index(index)))
+
+
def get_end_linenumber(text):
- """Utility to get the last line's number in a Tk text widget."""
- return int(float(text.index('end-1c')))
+ """Return the number of the last line in a Tk text widget."""
+ return get_lineno(text, 'end-1c')
+def get_displaylines(text, index):
+ """Display height, in lines, of a logical line in a Tk text widget."""
+ res = text.count(f"{index} linestart",
+ f"{index} lineend",
+ "displaylines")
+ return res[0] if res else 0
+
def get_widget_padding(widget):
"""Get the total padding of a Tk widget, including its border."""
# TODO: use also in codecontext.py
@@ -40,10 +54,17 @@ def get_widget_padding(widget):
return padx, pady
+(a)contextlib.contextmanager
+def temp_enable_text_widget(text):
+ text.configure(state=tk.NORMAL)
+ try:
+ yield
+ finally:
+ text.configure(state=tk.DISABLED)
+
+
class BaseSideBar:
- """
- The base class for extensions which require a sidebar.
- """
+ """A base class for sidebars using Text."""
def __init__(self, editwin):
self.editwin = editwin
self.parent = editwin.text_frame
@@ -119,14 +140,11 @@ def redirect_mousewheel_event(self, event):
class EndLineDelegator(Delegator):
- """Generate callbacks with the current end line number after
- insert or delete operations"""
+ """Generate callbacks with the current end line number.
+
+ The provided callback is called after every insert and delete.
+ """
def __init__(self, changed_callback):
- """
- changed_callback - Callable, will be called after insert
- or delete operations with the current
- end line number.
- """
Delegator.__init__(self)
self.changed_callback = changed_callback
@@ -159,16 +177,8 @@ def __init__(self, editwin):
end_line_delegator = EndLineDelegator(self.update_sidebar_text)
# Insert the delegator after the undo delegator, so that line numbers
# are properly updated after undo and redo actions.
- end_line_delegator.setdelegate(self.editwin.undo.delegate)
- self.editwin.undo.setdelegate(end_line_delegator)
- # Reset the delegator caches of the delegators "above" the
- # end line delegator we just inserted.
- delegator = self.editwin.per.top
- while delegator is not end_line_delegator:
- delegator.resetcache()
- delegator = delegator.delegate
-
- self.is_shown = False
+ self.editwin.per.insertfilterafter(filter=end_line_delegator,
+ after=self.editwin.undo)
def bind_events(self):
# Ensure focus is always redirected to the main editor text widget.
@@ -297,20 +307,209 @@ def update_sidebar_text(self, end):
new_width = cur_width + width_difference
self.sidebar_text['width'] = self._sidebar_width_type(new_width)
- self.sidebar_text.config(state=tk.NORMAL)
- if end > self.prev_end:
- new_text = '\n'.join(itertools.chain(
- [''],
- map(str, range(self.prev_end + 1, end + 1)),
- ))
- self.sidebar_text.insert(f'end -1c', new_text, 'linenumber')
- else:
- self.sidebar_text.delete(f'{end+1}.0 -1c', 'end -1c')
- self.sidebar_text.config(state=tk.DISABLED)
+ with temp_enable_text_widget(self.sidebar_text):
+ if end > self.prev_end:
+ new_text = '\n'.join(itertools.chain(
+ [''],
+ map(str, range(self.prev_end + 1, end + 1)),
+ ))
+ self.sidebar_text.insert(f'end -1c', new_text, 'linenumber')
+ else:
+ self.sidebar_text.delete(f'{end+1}.0 -1c', 'end -1c')
self.prev_end = end
+class WrappedLineHeightChangeDelegator(Delegator):
+ def __init__(self, callback):
+ """
+ callback - Callable, will be called when an insert, delete or replace
+ action on the text widget may require updating the shell
+ sidebar.
+ """
+ Delegator.__init__(self)
+ self.callback = callback
+
+ def insert(self, index, chars, tags=None):
+ is_single_line = '\n' not in chars
+ if is_single_line:
+ before_displaylines = get_displaylines(self, index)
+
+ self.delegate.insert(index, chars, tags)
+
+ if is_single_line:
+ after_displaylines = get_displaylines(self, index)
+ if after_displaylines == before_displaylines:
+ return # no need to update the sidebar
+
+ self.callback()
+
+ def delete(self, index1, index2=None):
+ if index2 is None:
+ index2 = index1 + "+1c"
+ is_single_line = get_lineno(self, index1) == get_lineno(self, index2)
+ if is_single_line:
+ before_displaylines = get_displaylines(self, index1)
+
+ self.delegate.delete(index1, index2)
+
+ if is_single_line:
+ after_displaylines = get_displaylines(self, index1)
+ if after_displaylines == before_displaylines:
+ return # no need to update the sidebar
+
+ self.callback()
+
+
+class ShellSidebar:
+ """Sidebar for the PyShell window, for prompts etc."""
+ def __init__(self, editwin):
+ self.editwin = editwin
+ self.parent = editwin.text_frame
+ self.text = editwin.text
+
+ self.canvas = tk.Canvas(self.parent, width=30,
+ borderwidth=0, highlightthickness=0,
+ takefocus=False)
+
+ self.bind_events()
+
+ change_delegator = \
+ WrappedLineHeightChangeDelegator(self.change_callback)
+
+ # Insert the TextChangeDelegator after the last delegator, so that
+ # the sidebar reflects final changes to the text widget contents.
+ d = self.editwin.per.top
+ if d.delegate is not self.text:
+ while d.delegate is not self.editwin.per.bottom:
+ d = d.delegate
+ self.editwin.per.insertfilterafter(change_delegator, after=d)
+
+ self.text['yscrollcommand'] = self.yscroll_event
+
+ self.is_shown = False
+
+ self.update_font()
+ self.update_colors()
+ self.update_sidebar()
+ self.canvas.grid(row=1, column=0, sticky=tk.NSEW, padx=2, pady=0)
+ self.is_shown = True
+
+ def change_callback(self):
+ if self.is_shown:
+ self.update_sidebar()
+
+ def update_sidebar(self):
+ text = self.text
+ text_tagnames = text.tag_names
+ canvas = self.canvas
+
+ canvas.delete(tk.ALL)
+
+ index = text.index("@0,0")
+ if index.split('.', 1)[1] != '0':
+ index = text.index(f'{index}+1line linestart')
+ while True:
+ lineinfo = text.dlineinfo(index)
+ if lineinfo is None:
+ break
+ y = lineinfo[1]
+ prev_newline_tagnames = text_tagnames(f"{index} linestart -1c")
+ prompt = (
+ '>>>' if "console" in prev_newline_tagnames else
+ '...' if "stdin" in prev_newline_tagnames else
+ None
+ )
+ if prompt:
+ canvas.create_text(2, y, anchor=tk.NW, text=prompt,
+ font=self.font, fill=self.colors[0])
+ index = text.index(f'{index}+1line')
+
+ def yscroll_event(self, *args, **kwargs):
+ """Redirect vertical scrolling to the main editor text widget.
+
+ The scroll bar is also updated.
+ """
+ self.editwin.vbar.set(*args)
+ self.change_callback()
+ return 'break'
+
+ def update_font(self):
+ """Update the sidebar text font, usually after config changes."""
+ font = idleConf.GetFont(self.text, 'main', 'EditorWindow')
+ tk_font = Font(self.text, font=font)
+ char_width = max(tk_font.measure(char) for char in ['>', '.'])
+ self.canvas.configure(width=char_width * 3 + 4)
+ self._update_font(font)
+
+ def _update_font(self, font):
+ self.font = font
+ self.change_callback()
+
+ def update_colors(self):
+ """Update the sidebar text colors, usually after config changes."""
+ linenumbers_colors = idleConf.GetHighlight(idleConf.CurrentTheme(), 'linenumber')
+ prompt_colors = idleConf.GetHighlight(idleConf.CurrentTheme(), 'console')
+ self._update_colors(foreground=prompt_colors['foreground'],
+ background=linenumbers_colors['background'])
+
+ def _update_colors(self, foreground, background):
+ self.colors = (foreground, background)
+ self.canvas.configure(background=self.colors[1])
+ self.change_callback()
+
+ def redirect_focusin_event(self, event):
+ """Redirect focus-in events to the main editor text widget."""
+ self.text.focus_set()
+ return 'break'
+
+ def redirect_mousebutton_event(self, event, event_name):
+ """Redirect mouse button events to the main editor text widget."""
+ self.text.focus_set()
+ self.text.event_generate(event_name, x=0, y=event.y)
+ return 'break'
+
+ def redirect_mousewheel_event(self, event):
+ """Redirect mouse wheel events to the editwin text widget."""
+ self.text.event_generate('<MouseWheel>',
+ x=0, y=event.y, delta=event.delta)
+ return 'break'
+
+ def bind_events(self):
+ # Ensure focus is always redirected to the main editor text widget.
+ self.canvas.bind('<FocusIn>', self.redirect_focusin_event)
+
+ # Redirect mouse scrolling to the main editor text widget.
+ #
+ # Note that without this, scrolling with the mouse only scrolls
+ # the line numbers.
+ self.canvas.bind('<MouseWheel>', self.redirect_mousewheel_event)
+
+ # Redirect mouse button events to the main editor text widget,
+ # except for the left mouse button (1).
+ #
+ # Note: X-11 sends Button-4 and Button-5 events for the scroll wheel.
+ def bind_mouse_event(event_name, target_event_name):
+ handler = functools.partial(self.redirect_mousebutton_event,
+ event_name=target_event_name)
+ self.canvas.bind(event_name, handler)
+
+ for button in [2, 3, 4, 5]:
+ for event_name in (f'<Button-{button}>',
+ f'<ButtonRelease-{button}>',
+ f'<B{button}-Motion>',
+ ):
+ bind_mouse_event(event_name, target_event_name=event_name)
+
+ # Convert double- and triple-click events to normal click events,
+ # since event_generate() doesn't allow generating such events.
+ for event_name in (f'<Double-Button-{button}>',
+ f'<Triple-Button-{button}>',
+ ):
+ bind_mouse_event(event_name,
+ target_event_name=f'<Button-{button}>')
+
+
def _linenumbers_drag_scrolling(parent): # htest #
from idlelib.idle_test.test_sidebar import Dummy_editwin
diff --git a/Lib/idlelib/squeezer.py b/Lib/idlelib/squeezer.py
index 3046d803b74a4e..929c3fd3a507f4 100644
--- a/Lib/idlelib/squeezer.py
+++ b/Lib/idlelib/squeezer.py
@@ -160,8 +160,10 @@ def expand(self, event=None):
if not confirm:
return "break"
- self.base_text.insert(self.text.index(self), self.s, self.tags)
+ index = self.text.index(self)
+ self.base_text.insert(index, self.s, self.tags)
self.base_text.delete(self)
+ self.editwin.on_squeezed_expand(index, self.s, self.tags)
self.squeezer.expandingbuttons.remove(self)
def copy(self, event=None):
@@ -285,12 +287,10 @@ def count_lines(self, s):
"""
return count_lines_with_wrapping(s, self.editwin.width)
- def squeeze_current_text_event(self, event):
- """squeeze-current-text event handler
+ def squeeze_current_text(self):
+ """Squeeze the text block where the insertion cursor is.
- Squeeze the block of text inside which contains the "insert" cursor.
-
- If the insert cursor is not in a squeezable block of text, give the
+ If the cursor is not in a squeezable block of text, give the
user a small warning and do nothing.
"""
# Set tag_name to the first valid tag found on the "insert" cursor.
diff --git a/Misc/NEWS.d/next/IDLE/2019-08-24-23-49-36.bpo-37903.4xjast.rst b/Misc/NEWS.d/next/IDLE/2019-08-24-23-49-36.bpo-37903.4xjast.rst
new file mode 100644
index 00000000000000..56b50e2e91e467
--- /dev/null
+++ b/Misc/NEWS.d/next/IDLE/2019-08-24-23-49-36.bpo-37903.4xjast.rst
@@ -0,0 +1 @@
+IDLE's shell now shows prompts in a separate side-bar.
1
0
28 Apr '21
https://github.com/python/cpython/commit/103d5e420dd90489933ad9da8bb1d60087…
commit: 103d5e420dd90489933ad9da8bb1d6008773384d
branch: master
author: Victor Stinner <vstinner(a)python.org>
committer: vstinner <vstinner(a)python.org>
date: 2021-04-28T19:09:29+02:00
summary:
bpo-28254: _posixsubprocess uses PyGC_Enable/PyGC_Disable (GH-25693)
files:
M Lib/test/test_subprocess.py
M Modules/_posixsubprocess.c
diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
index 7c79365f41191..27ccd3e5cb3a8 100644
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -2147,8 +2147,6 @@ def raise_it():
def test_preexec_gc_module_failure(self):
# This tests the code that disables garbage collection if the child
# process will execute any Python.
- def raise_runtime_error():
- raise RuntimeError("this shouldn't escape")
enabled = gc.isenabled()
orig_gc_disable = gc.disable
orig_gc_isenabled = gc.isenabled
@@ -2165,16 +2163,6 @@ def raise_runtime_error():
subprocess.call([sys.executable, '-c', ''],
preexec_fn=lambda: None)
self.assertTrue(gc.isenabled(), "Popen left gc disabled.")
-
- gc.disable = raise_runtime_error
- self.assertRaises(RuntimeError, subprocess.Popen,
- [sys.executable, '-c', ''],
- preexec_fn=lambda: None)
-
- del gc.isenabled # force an AttributeError
- self.assertRaises(AttributeError, subprocess.Popen,
- [sys.executable, '-c', ''],
- preexec_fn=lambda: None)
finally:
gc.disable = orig_gc_disable
gc.isenabled = orig_gc_isenabled
diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c
index 3b0651620e551..a58159a277bea 100644
--- a/Modules/_posixsubprocess.c
+++ b/Modules/_posixsubprocess.c
@@ -69,47 +69,8 @@
#define POSIX_CALL(call) do { if ((call) == -1) goto error; } while (0)
-typedef struct {
- PyObject* disable;
- PyObject* enable;
- PyObject* isenabled;
-} _posixsubprocessstate;
-
static struct PyModuleDef _posixsubprocessmodule;
-static inline _posixsubprocessstate*
-get_posixsubprocess_state(PyObject *module)
-{
- void *state = PyModule_GetState(module);
- assert(state != NULL);
- return (_posixsubprocessstate *)state;
-}
-
-/* If gc was disabled, call gc.enable(). Ignore errors. */
-static void
-_enable_gc(int need_to_reenable_gc, PyObject *gc_module, _posixsubprocessstate *state)
-{
- PyObject *result;
- PyObject *exctype, *val, *tb;
-
- if (need_to_reenable_gc) {
- PyErr_Fetch(&exctype, &val, &tb);
- result = PyObject_CallMethodNoArgs(
- gc_module, state->enable);
- if (result == NULL) {
- /* We might have created a child process at this point, we
- * we have no good way to handle a failure to reenable GC
- * and return information about the child process. */
- PyErr_Print();
- }
- Py_XDECREF(result);
- if (exctype != NULL) {
- PyErr_Restore(exctype, val, tb);
- }
- }
-}
-
-
/* Convert ASCII to a positive int, no libc call. no overflow. -1 on error. */
static int
_pos_int_from_ascii(const char *name)
@@ -780,7 +741,6 @@ subprocess_fork_exec(PyObject *module, PyObject *args)
Py_ssize_t arg_num, num_groups = 0;
int need_after_fork = 0;
int saved_errno = 0;
- _posixsubprocessstate *state = get_posixsubprocess_state(module);
if (!PyArg_ParseTuple(
args, "OOpO!OOiiiiiiiiiiOOOiO:fork_exec",
@@ -820,30 +780,7 @@ subprocess_fork_exec(PyObject *module, PyObject *args)
/* We need to call gc.disable() when we'll be calling preexec_fn */
if (preexec_fn != Py_None) {
- PyObject *result;
-
- gc_module = PyImport_ImportModule("gc");
- if (gc_module == NULL)
- return NULL;
- result = PyObject_CallMethodNoArgs(
- gc_module, state->isenabled);
- if (result == NULL) {
- Py_DECREF(gc_module);
- return NULL;
- }
- need_to_reenable_gc = PyObject_IsTrue(result);
- Py_DECREF(result);
- if (need_to_reenable_gc == -1) {
- Py_DECREF(gc_module);
- return NULL;
- }
- result = PyObject_CallMethodNoArgs(
- gc_module, state->disable);
- if (result == NULL) {
- Py_DECREF(gc_module);
- return NULL;
- }
- Py_DECREF(result);
+ need_to_reenable_gc = PyGC_Disable();
}
exec_array = _PySequence_BytesToCharpArray(executable_list);
@@ -1068,7 +1005,9 @@ subprocess_fork_exec(PyObject *module, PyObject *args)
if (exec_array)
_Py_FreeCharPArray(exec_array);
- _enable_gc(need_to_reenable_gc, gc_module, state);
+ if (need_to_reenable_gc) {
+ PyGC_Enable();
+ }
Py_XDECREF(gc_module);
return pid == -1 ? NULL : PyLong_FromPid(pid);
@@ -1108,67 +1047,22 @@ Raises: Only on an error in the parent process.\n\
PyDoc_STRVAR(module_doc,
"A POSIX helper for the subprocess module.");
-static int
-_posixsubprocess_exec(PyObject *module)
-{
- _posixsubprocessstate *state = get_posixsubprocess_state(module);
-
- state->disable = PyUnicode_InternFromString("disable");
- if (state->disable == NULL) {
- return -1;
- }
-
- state->enable = PyUnicode_InternFromString("enable");
- if (state->enable == NULL) {
- return -1;
- }
-
- state->isenabled = PyUnicode_InternFromString("isenabled");
- if (state->isenabled == NULL) {
- return -1;
- }
-
- return 0;
-}
-
static PyMethodDef module_methods[] = {
{"fork_exec", subprocess_fork_exec, METH_VARARGS, subprocess_fork_exec_doc},
{NULL, NULL} /* sentinel */
};
static PyModuleDef_Slot _posixsubprocess_slots[] = {
- {Py_mod_exec, _posixsubprocess_exec},
{0, NULL}
};
-static int _posixsubprocess_traverse(PyObject *m, visitproc visit, void *arg) {
- Py_VISIT(get_posixsubprocess_state(m)->disable);
- Py_VISIT(get_posixsubprocess_state(m)->enable);
- Py_VISIT(get_posixsubprocess_state(m)->isenabled);
- return 0;
-}
-
-static int _posixsubprocess_clear(PyObject *m) {
- Py_CLEAR(get_posixsubprocess_state(m)->disable);
- Py_CLEAR(get_posixsubprocess_state(m)->enable);
- Py_CLEAR(get_posixsubprocess_state(m)->isenabled);
- return 0;
-}
-
-static void _posixsubprocess_free(void *m) {
- _posixsubprocess_clear((PyObject *)m);
-}
-
static struct PyModuleDef _posixsubprocessmodule = {
PyModuleDef_HEAD_INIT,
.m_name = "_posixsubprocess",
.m_doc = module_doc,
- .m_size = sizeof(_posixsubprocessstate),
+ .m_size = 0,
.m_methods = module_methods,
.m_slots = _posixsubprocess_slots,
- .m_traverse = _posixsubprocess_traverse,
- .m_clear = _posixsubprocess_clear,
- .m_free = _posixsubprocess_free,
};
PyMODINIT_FUNC
1
0
https://github.com/python/cpython/commit/3b52c8d66b25415f09478ab43f93d59a35…
commit: 3b52c8d66b25415f09478ab43f93d59a3547dc13
branch: master
author: Erlend Egeberg Aasland <erlend.aasland(a)innova.no>
committer: vstinner <vstinner(a)python.org>
date: 2021-04-28T19:02:42+02:00
summary:
bpo-43908: Add Py_TPFLAGS_IMMUTABLETYPE flag (GH-25520)
Introduce Py_TPFLAGS_IMMUTABLETYPE flag for immutable type objects, and
modify PyType_Ready() to set it for static types.
Co-authored-by: Victor Stinner <vstinner(a)python.org>
files:
A Misc/NEWS.d/next/C API/2021-04-22-10-46-40.bpo-43908.Co3YhZ.rst
M Doc/c-api/typeobj.rst
M Include/object.h
M Objects/typeobject.c
diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
index 9efe3aac2e1c9..4c75a12194d18 100644
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -1177,6 +1177,18 @@ and :c:type:`PyType_Type` effectively act as defaults.)
.. versionadded:: 3.10
+ .. data:: Py_TPFLAGS_IMMUTABLETYPE
+
+ This bit is set for type objects that are immutable: type attributes cannot be set nor deleted.
+
+ :c:func:`PyType_Ready` automatically applies this flag to static types.
+
+ **Inheritance:**
+
+ This flag is not inherited.
+
+ .. versionadded:: 3.10
+
.. c:member:: const char* PyTypeObject.tp_doc
diff --git a/Include/object.h b/Include/object.h
index 695f01564282c..d8476f9213760 100644
--- a/Include/object.h
+++ b/Include/object.h
@@ -320,6 +320,9 @@ Code can use PyType_HasFeature(type_ob, flag_value) to test whether the
given type object has a specified feature.
*/
+/* Set if the type object is immutable: type attributes cannot be set nor deleted */
+#define Py_TPFLAGS_IMMUTABLETYPE (1UL << 8)
+
/* Set if the type object is dynamically allocated */
#define Py_TPFLAGS_HEAPTYPE (1UL << 9)
diff --git a/Misc/NEWS.d/next/C API/2021-04-22-10-46-40.bpo-43908.Co3YhZ.rst b/Misc/NEWS.d/next/C API/2021-04-22-10-46-40.bpo-43908.Co3YhZ.rst
new file mode 100644
index 0000000000000..0413c20a1b6b2
--- /dev/null
+++ b/Misc/NEWS.d/next/C API/2021-04-22-10-46-40.bpo-43908.Co3YhZ.rst
@@ -0,0 +1,3 @@
+Introduce :const:`Py_TPFLAGS_IMMUTABLETYPE` flag for immutable type objects, and
+modify :c:func:`PyType_Ready` to set it for static types. Patch by
+Erlend E. Aasland.
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
index 254d12cc97014..e1c8be4b81545 100644
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -3875,7 +3875,7 @@ static int
type_setattro(PyTypeObject *type, PyObject *name, PyObject *value)
{
int res;
- if (!(type->tp_flags & Py_TPFLAGS_HEAPTYPE)) {
+ if (type->tp_flags & Py_TPFLAGS_IMMUTABLETYPE) {
PyErr_Format(
PyExc_TypeError,
"can't set attributes of built-in/extension type '%s'",
@@ -6229,6 +6229,11 @@ PyType_Ready(PyTypeObject *type)
type->tp_flags |= Py_TPFLAGS_READYING;
+ /* Historically, all static types were immutable. See bpo-43908 */
+ if (!(type->tp_flags & Py_TPFLAGS_HEAPTYPE)) {
+ type->tp_flags |= Py_TPFLAGS_IMMUTABLETYPE;
+ }
+
if (type_ready(type) < 0) {
type->tp_flags &= ~Py_TPFLAGS_READYING;
return -1;
1
0
bpo-43472: Ensure PyInterpreterState_New audit events are raised when called through _xxsubinterpreters module (GH-25506) (GH-25508)
by vstinner 28 Apr '21
by vstinner 28 Apr '21
28 Apr '21
https://github.com/python/cpython/commit/0252ce35712f4a12e824fb8b40a867ec34…
commit: 0252ce35712f4a12e824fb8b40a867ec3460443e
branch: 3.9
author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com>
committer: vstinner <vstinner(a)python.org>
date: 2021-04-28T18:20:40+02:00
summary:
bpo-43472: Ensure PyInterpreterState_New audit events are raised when called through _xxsubinterpreters module (GH-25506) (GH-25508)
(cherry picked from commit 7b86e47617d81a4b14d929743425f448971e8c86)
Co-authored-by: Steve Dower <steve.dower(a)python.org>
Co-authored-by: Steve Dower <steve.dower(a)python.org>
files:
A Misc/NEWS.d/next/Security/2021-04-21-22-53-31.bpo-43472.gjLBTb.rst
M Doc/library/sys.rst
M Modules/_xxsubinterpretersmodule.c
diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
index cb2f254b1fbe4c..6b23b6f0ab74aa 100644
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -26,12 +26,12 @@ always available.
.. function:: addaudithook(hook)
Append the callable *hook* to the list of active auditing hooks for the
- current interpreter.
+ current (sub)interpreter.
When an auditing event is raised through the :func:`sys.audit` function, each
hook will be called in the order it was added with the event name and the
tuple of arguments. Native hooks added by :c:func:`PySys_AddAuditHook` are
- called first, followed by hooks added in the current interpreter. Hooks
+ called first, followed by hooks added in the current (sub)interpreter. Hooks
can then log the event, raise an exception to abort the operation,
or terminate the process entirely.
diff --git a/Misc/NEWS.d/next/Security/2021-04-21-22-53-31.bpo-43472.gjLBTb.rst b/Misc/NEWS.d/next/Security/2021-04-21-22-53-31.bpo-43472.gjLBTb.rst
new file mode 100644
index 00000000000000..e38dc169def96e
--- /dev/null
+++ b/Misc/NEWS.d/next/Security/2021-04-21-22-53-31.bpo-43472.gjLBTb.rst
@@ -0,0 +1,3 @@
+Ensures interpreter-level audit hooks receive the
+``cpython.PyInterpreterState_New`` event when called through the
+``_xxsubinterpreters`` module.
diff --git a/Modules/_xxsubinterpretersmodule.c b/Modules/_xxsubinterpretersmodule.c
index de11c090870f94..314059d10828cb 100644
--- a/Modules/_xxsubinterpretersmodule.c
+++ b/Modules/_xxsubinterpretersmodule.c
@@ -2010,7 +2010,7 @@ interp_create(PyObject *self, PyObject *args, PyObject *kwds)
}
// Create and initialize the new interpreter.
- PyThreadState *save_tstate = PyThreadState_Swap(NULL);
+ PyThreadState *save_tstate = PyThreadState_Get();
// XXX Possible GILState issues?
PyThreadState *tstate = _Py_NewInterpreter(isolated);
PyThreadState_Swap(save_tstate);
1
0
https://github.com/python/cpython/commit/3cc481b9de43c234889c8010e7da3af7c0…
commit: 3cc481b9de43c234889c8010e7da3af7c0f42319
branch: master
author: scoder <stefan_ml(a)behnel.de>
committer: vstinner <vstinner(a)python.org>
date: 2021-04-28T18:12:16+02:00
summary:
bpo-28254: Add a C-API for controlling the GC state (GH-25687)
Add new C-API functions to control the state of the garbage collector:
PyGC_Enable(), PyGC_Disable(), PyGC_IsEnabled(),
corresponding to the functions in the gc module.
Co-authored-by: Pablo Galindo <Pablogsal(a)gmail.com>
Co-authored-by: Victor Stinner <vstinner(a)python.org>
files:
A Misc/NEWS.d/next/C API/2021-04-28-12-33-44.bpo-28254.a2561e.rst
M Doc/c-api/gcsupport.rst
M Doc/data/stable_abi.dat
M Doc/whatsnew/3.10.rst
M Include/objimpl.h
M Modules/_testcapimodule.c
M Modules/gcmodule.c
diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst
index eee114c19d5904..55ed9d4f7fad48 100644
--- a/Doc/c-api/gcsupport.rst
+++ b/Doc/c-api/gcsupport.rst
@@ -173,3 +173,46 @@ if the object is immutable.
this method (don't just call :c:func:`Py_DECREF` on a reference). The
collector will call this method if it detects that this object is involved
in a reference cycle.
+
+
+Controlling the Garbage Collector State
+---------------------------------------
+
+The C-API provides the following functions for controlling
+garbage collection runs.
+
+.. c:function:: Py_ssize_t PyGC_Collect(void)
+
+ Perform a full garbage collection, if the garbage collector is enabled.
+ (Note that :func:`gc.collect` runs it unconditionally.)
+
+ Returns the number of collected + unreachable objects which cannot
+ be collected.
+ If the garbage collector is disabled or already collecting,
+ returns ``0`` immediately.
+ Errors during garbage collection are passed to :data:`sys.unraisablehook`.
+ This function does not raise exceptions.
+
+
+.. c:function:: int PyGC_Enable(void)
+
+ Enable the garbage collector: similar to :func:`gc.enable`.
+ Returns the previous state, 0 for disabled and 1 for enabled.
+
+ .. versionadded:: 3.10
+
+
+.. c:function:: int PyGC_Disable(void)
+
+ Disable the garbage collector: similar to :func:`gc.disable`.
+ Returns the previous state, 0 for disabled and 1 for enabled.
+
+ .. versionadded:: 3.10
+
+
+.. c:function:: int PyGC_IsEnabled(void)
+
+ Query the state of the garbage collector: similar to :func:`gc.isenabled`.
+ Returns the current state, 0 for disabled and 1 for enabled.
+
+ .. versionadded:: 3.10
diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat
index cdc7160250243b..491a5fbb96f5a5 100644
--- a/Doc/data/stable_abi.dat
+++ b/Doc/data/stable_abi.dat
@@ -268,6 +268,9 @@ PyFrame_GetLineNumber
PyFrozenSet_New
PyFrozenSet_Type
PyGC_Collect
+PyGC_Disable
+PyGC_Enable
+PyGC_IsEnabled
PyGILState_Ensure
PyGILState_GetThisThreadState
PyGILState_Release
diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst
index 2d8bb285fe981b..37c1b8a0cb2ebc 100644
--- a/Doc/whatsnew/3.10.rst
+++ b/Doc/whatsnew/3.10.rst
@@ -1697,6 +1697,13 @@ New Features
singleton or the ``False`` singleton.
(Contributed by Victor Stinner in :issue:`43753`.)
+* Add new functions to quickly control the garbage collector from C code:
+ :c:func:`PyGC_Enable()`,
+ :c:func:`PyGC_Disable()`,
+ :c:func:`PyGC_IsEnabled()`.
+ These functions allow to activate, deactivate and query the state of the garbage collector from C code without
+ having to import the :mod:`gc` module.
+
Porting to Python 3.10
----------------------
diff --git a/Include/objimpl.h b/Include/objimpl.h
index 1408d051ba7efe..689c42b7475c89 100644
--- a/Include/objimpl.h
+++ b/Include/objimpl.h
@@ -150,8 +150,12 @@ PyAPI_FUNC(PyVarObject *) _PyObject_NewVar(PyTypeObject *, Py_ssize_t);
* ==========================
*/
-/* C equivalent of gc.collect() which ignores the state of gc.enabled. */
+/* C equivalent of gc.collect(). */
PyAPI_FUNC(Py_ssize_t) PyGC_Collect(void);
+/* C API for controlling the state of the garbage collector */
+PyAPI_FUNC(int) PyGC_Enable(void);
+PyAPI_FUNC(int) PyGC_Disable(void);
+PyAPI_FUNC(int) PyGC_IsEnabled(void);
/* Test if a type has a GC head */
#define PyType_IS_GC(t) PyType_HasFeature((t), Py_TPFLAGS_HAVE_GC)
diff --git a/Misc/NEWS.d/next/C API/2021-04-28-12-33-44.bpo-28254.a2561e.rst b/Misc/NEWS.d/next/C API/2021-04-28-12-33-44.bpo-28254.a2561e.rst
new file mode 100644
index 00000000000000..015acc9803d474
--- /dev/null
+++ b/Misc/NEWS.d/next/C API/2021-04-28-12-33-44.bpo-28254.a2561e.rst
@@ -0,0 +1,3 @@
+Add new C-API functions to control the state of the garbage collector:
+:c:func:`PyGC_Enable()`, :c:func:`PyGC_Disable()`, :c:func:`PyGC_IsEnabled()`,
+corresponding to the functions in the :mod:`gc` module.
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
index db62aea421c806..26ebacba642a4f 100644
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -144,6 +144,67 @@ test_sizeof_c_types(PyObject *self, PyObject *Py_UNUSED(ignored))
#endif
}
+static PyObject*
+test_gc_control(PyObject *self, PyObject *Py_UNUSED(ignored))
+{
+ int orig_enabled = PyGC_IsEnabled();
+ const char* msg = "ok";
+ int old_state;
+
+ old_state = PyGC_Enable();
+ msg = "Enable(1)";
+ if (old_state != orig_enabled) {
+ goto failed;
+ }
+ msg = "IsEnabled(1)";
+ if (!PyGC_IsEnabled()) {
+ goto failed;
+ }
+
+ old_state = PyGC_Disable();
+ msg = "disable(2)";
+ if (!old_state) {
+ goto failed;
+ }
+ msg = "IsEnabled(2)";
+ if (PyGC_IsEnabled()) {
+ goto failed;
+ }
+
+ old_state = PyGC_Enable();
+ msg = "enable(3)";
+ if (old_state) {
+ goto failed;
+ }
+ msg = "IsEnabled(3)";
+ if (!PyGC_IsEnabled()) {
+ goto failed;
+ }
+
+ if (!orig_enabled) {
+ old_state = PyGC_Disable();
+ msg = "disable(4)";
+ if (old_state) {
+ goto failed;
+ }
+ msg = "IsEnabled(4)";
+ if (PyGC_IsEnabled()) {
+ goto failed;
+ }
+ }
+
+ Py_RETURN_NONE;
+
+failed:
+ /* Try to clean up if we can. */
+ if (orig_enabled) {
+ PyGC_Enable();
+ } else {
+ PyGC_Disable();
+ }
+ PyErr_Format(TestError, "GC control failed in %s", msg);
+ return NULL;
+}
static PyObject*
test_list_api(PyObject *self, PyObject *Py_UNUSED(ignored))
@@ -5544,6 +5605,7 @@ static PyMethodDef TestMethods[] = {
{"PyDateTime_DATE_GET", test_PyDateTime_DATE_GET, METH_O},
{"PyDateTime_TIME_GET", test_PyDateTime_TIME_GET, METH_O},
{"PyDateTime_DELTA_GET", test_PyDateTime_DELTA_GET, METH_O},
+ {"test_gc_control", test_gc_control, METH_NOARGS},
{"test_list_api", test_list_api, METH_NOARGS},
{"test_dict_iteration", test_dict_iteration, METH_NOARGS},
{"dict_getitem_knownhash", dict_getitem_knownhash, METH_VARARGS},
diff --git a/Modules/gcmodule.c b/Modules/gcmodule.c
index d6b51426c4e2d8..e5e5aa3287b0d6 100644
--- a/Modules/gcmodule.c
+++ b/Modules/gcmodule.c
@@ -1484,8 +1484,7 @@ static PyObject *
gc_enable_impl(PyObject *module)
/*[clinic end generated code: output=45a427e9dce9155c input=81ac4940ca579707]*/
{
- GCState *gcstate = get_gc_state();
- gcstate->enabled = 1;
+ PyGC_Enable();
Py_RETURN_NONE;
}
@@ -1499,8 +1498,7 @@ static PyObject *
gc_disable_impl(PyObject *module)
/*[clinic end generated code: output=97d1030f7aa9d279 input=8c2e5a14e800d83b]*/
{
- GCState *gcstate = get_gc_state();
- gcstate->enabled = 0;
+ PyGC_Disable();
Py_RETURN_NONE;
}
@@ -1514,8 +1512,7 @@ static int
gc_isenabled_impl(PyObject *module)
/*[clinic end generated code: output=1874298331c49130 input=30005e0422373b31]*/
{
- GCState *gcstate = get_gc_state();
- return gcstate->enabled;
+ return PyGC_IsEnabled();
}
/*[clinic input]
@@ -2053,6 +2050,32 @@ PyInit_gc(void)
return PyModuleDef_Init(&gcmodule);
}
+/* C API for controlling the state of the garbage collector */
+int
+PyGC_Enable(void)
+{
+ GCState *gcstate = get_gc_state();
+ int old_state = gcstate->enabled;
+ gcstate->enabled = 1;
+ return old_state;
+}
+
+int
+PyGC_Disable(void)
+{
+ GCState *gcstate = get_gc_state();
+ int old_state = gcstate->enabled;
+ gcstate->enabled = 0;
+ return old_state;
+}
+
+int
+PyGC_IsEnabled(void)
+{
+ GCState *gcstate = get_gc_state();
+ return gcstate->enabled;
+}
+
/* Public API to invoke gc.collect() from C */
Py_ssize_t
PyGC_Collect(void)
1
0
bpo-43757: Make pathlib use os.path.realpath() to resolve symlinks in a path (GH-25264)
by zooba 28 Apr '21
by zooba 28 Apr '21
28 Apr '21
https://github.com/python/cpython/commit/baecfbd849dbf42360d3a84af6cc131608…
commit: baecfbd849dbf42360d3a84af6cc13160838f24d
branch: master
author: Barney Gale <barney.gale(a)gmail.com>
committer: zooba <steve.dower(a)microsoft.com>
date: 2021-04-28T16:50:17+01:00
summary:
bpo-43757: Make pathlib use os.path.realpath() to resolve symlinks in a path (GH-25264)
Also adds a new "strict" argument to realpath() to avoid changing the default behaviour of pathlib while sharing the implementation.
files:
A Misc/NEWS.d/next/Library/2021-04-08-22-11-27.bpo-25264.b33fa0.rst
M Doc/library/os.path.rst
M Lib/ntpath.py
M Lib/pathlib.py
M Lib/posixpath.py
M Lib/test/test_ntpath.py
M Lib/test/test_posixpath.py
diff --git a/Doc/library/os.path.rst b/Doc/library/os.path.rst
index 4cab3113c9e4e..d06d9ce8c9e3d 100644
--- a/Doc/library/os.path.rst
+++ b/Doc/library/os.path.rst
@@ -344,15 +344,24 @@ the :mod:`glob` module.)
Accepts a :term:`path-like object`.
-.. function:: realpath(path)
+.. function:: realpath(path, *, strict=False)
Return the canonical path of the specified filename, eliminating any symbolic
links encountered in the path (if they are supported by the operating
system).
+ If a path doesn't exist or a symlink loop is encountered, and *strict* is
+ ``True``, :exc:`OSError` is raised. If *strict* is ``False``, the path is
+ resolved as far as possible and any remainder is appended without checking
+ whether it exists.
+
.. note::
- When symbolic link cycles occur, the returned path will be one member of
- the cycle, but no guarantee is made about which member that will be.
+ This function emulates the operating system's procedure for making a path
+ canonical, which differs slightly between Windows and UNIX with respect
+ to how links and subsequent path components interact.
+
+ Operating system APIs make paths canonical as needed, so it's not
+ normally necessary to call this function.
.. versionchanged:: 3.6
Accepts a :term:`path-like object`.
@@ -360,6 +369,9 @@ the :mod:`glob` module.)
.. versionchanged:: 3.8
Symbolic links and junctions are now resolved on Windows.
+ .. versionchanged:: 3.10
+ The *strict* parameter was added.
+
.. function:: relpath(path, start=os.curdir)
diff --git a/Lib/ntpath.py b/Lib/ntpath.py
index 5ae8079074cd9..527c7ae1938fb 100644
--- a/Lib/ntpath.py
+++ b/Lib/ntpath.py
@@ -635,7 +635,7 @@ def _getfinalpathname_nonstrict(path):
tail = join(name, tail) if tail else name
return tail
- def realpath(path):
+ def realpath(path, *, strict=False):
path = normpath(path)
if isinstance(path, bytes):
prefix = b'\\\\?\\'
@@ -660,6 +660,8 @@ def realpath(path):
path = _getfinalpathname(path)
initial_winerror = 0
except OSError as ex:
+ if strict:
+ raise
initial_winerror = ex.winerror
path = _getfinalpathname_nonstrict(path)
# The path returned by _getfinalpathname will always start with \\?\ -
diff --git a/Lib/pathlib.py b/Lib/pathlib.py
index 37934c6038e1d..073fce82ad570 100644
--- a/Lib/pathlib.py
+++ b/Lib/pathlib.py
@@ -14,12 +14,6 @@
from urllib.parse import quote_from_bytes as urlquote_from_bytes
-if os.name == 'nt':
- from nt import _getfinalpathname
-else:
- _getfinalpathname = None
-
-
__all__ = [
"PurePath", "PurePosixPath", "PureWindowsPath",
"Path", "PosixPath", "WindowsPath",
@@ -29,14 +23,17 @@
# Internals
#
+_WINERROR_NOT_READY = 21 # drive exists but is not accessible
+_WINERROR_INVALID_NAME = 123 # fix for bpo-35306
+_WINERROR_CANT_RESOLVE_FILENAME = 1921 # broken symlink pointing to itself
+
# EBADF - guard against macOS `stat` throwing EBADF
_IGNORED_ERROS = (ENOENT, ENOTDIR, EBADF, ELOOP)
_IGNORED_WINERRORS = (
- 21, # ERROR_NOT_READY - drive exists but is not accessible
- 123, # ERROR_INVALID_NAME - fix for bpo-35306
- 1921, # ERROR_CANT_RESOLVE_FILENAME - fix for broken symlink pointing to itself
-)
+ _WINERROR_NOT_READY,
+ _WINERROR_INVALID_NAME,
+ _WINERROR_CANT_RESOLVE_FILENAME)
def _ignore_error(exception):
return (getattr(exception, 'errno', None) in _IGNORED_ERROS or
@@ -186,30 +183,6 @@ def casefold_parts(self, parts):
def compile_pattern(self, pattern):
return re.compile(fnmatch.translate(pattern), re.IGNORECASE).fullmatch
- def resolve(self, path, strict=False):
- s = str(path)
- if not s:
- return path._accessor.getcwd()
- previous_s = None
- if _getfinalpathname is not None:
- if strict:
- return self._ext_to_normal(_getfinalpathname(s))
- else:
- tail_parts = [] # End of the path after the first one not found
- while True:
- try:
- s = self._ext_to_normal(_getfinalpathname(s))
- except FileNotFoundError:
- previous_s = s
- s, tail = os.path.split(s)
- tail_parts.append(tail)
- if previous_s == s:
- return path
- else:
- return os.path.join(s, *reversed(tail_parts))
- # Means fallback on absolute
- return None
-
def _split_extended_path(self, s, ext_prefix=ext_namespace_prefix):
prefix = ''
if s.startswith(ext_prefix):
@@ -220,10 +193,6 @@ def _split_extended_path(self, s, ext_prefix=ext_namespace_prefix):
s = '\\' + s[3:]
return prefix, s
- def _ext_to_normal(self, s):
- # Turn back an extended path into a normal DOS-like path
- return self._split_extended_path(s)[1]
-
def is_reserved(self, parts):
# NOTE: the rules for reserved names seem somewhat complicated
# (e.g. r"..\NUL" is reserved but not r"foo\NUL").
@@ -281,54 +250,6 @@ def casefold_parts(self, parts):
def compile_pattern(self, pattern):
return re.compile(fnmatch.translate(pattern)).fullmatch
- def resolve(self, path, strict=False):
- sep = self.sep
- accessor = path._accessor
- seen = {}
- def _resolve(path, rest):
- if rest.startswith(sep):
- path = ''
-
- for name in rest.split(sep):
- if not name or name == '.':
- # current dir
- continue
- if name == '..':
- # parent dir
- path, _, _ = path.rpartition(sep)
- continue
- if path.endswith(sep):
- newpath = path + name
- else:
- newpath = path + sep + name
- if newpath in seen:
- # Already seen this path
- path = seen[newpath]
- if path is not None:
- # use cached value
- continue
- # The symlink is not resolved, so we must have a symlink loop.
- raise RuntimeError("Symlink loop from %r" % newpath)
- # Resolve the symbolic link
- try:
- target = accessor.readlink(newpath)
- except OSError as e:
- if e.errno != EINVAL and strict:
- raise
- # Not a symlink, or non-strict mode. We just leave the path
- # untouched.
- path = newpath
- else:
- seen[newpath] = None # not resolved symlink
- path = _resolve(path, target)
- seen[newpath] = path # resolved symlink
-
- return path
- # NOTE: according to POSIX, getcwd() cannot contain path components
- # which are symlinks.
- base = '' if path.is_absolute() else accessor.getcwd()
- return _resolve(base, str(path)) or sep
-
def is_reserved(self, parts):
return False
@@ -424,6 +345,8 @@ def group(self, path):
expanduser = staticmethod(os.path.expanduser)
+ realpath = staticmethod(os.path.realpath)
+
_normal_accessor = _NormalAccessor()
@@ -1132,15 +1055,27 @@ def resolve(self, strict=False):
normalizing it (for example turning slashes into backslashes under
Windows).
"""
- s = self._flavour.resolve(self, strict=strict)
- if s is None:
- # No symlink resolution => for consistency, raise an error if
- # the path doesn't exist or is forbidden
- self.stat()
- s = str(self.absolute())
- # Now we have no symlinks in the path, it's safe to normalize it.
- normed = self._flavour.pathmod.normpath(s)
- return self._from_parts((normed,))
+
+ def check_eloop(e):
+ winerror = getattr(e, 'winerror', 0)
+ if e.errno == ELOOP or winerror == _WINERROR_CANT_RESOLVE_FILENAME:
+ raise RuntimeError("Symlink loop from %r" % e.filename)
+
+ try:
+ s = self._accessor.realpath(self, strict=strict)
+ except OSError as e:
+ check_eloop(e)
+ raise
+ p = self._from_parts((s,))
+
+ # In non-strict mode, realpath() doesn't raise on symlink loops.
+ # Ensure we get an exception by calling stat()
+ if not strict:
+ try:
+ p.stat()
+ except OSError as e:
+ check_eloop(e)
+ return p
def stat(self, *, follow_symlinks=True):
"""
diff --git a/Lib/posixpath.py b/Lib/posixpath.py
index 62afbd0ccf0f0..259baa64b193b 100644
--- a/Lib/posixpath.py
+++ b/Lib/posixpath.py
@@ -387,16 +387,16 @@ def abspath(path):
# Return a canonical path (i.e. the absolute location of a file on the
# filesystem).
-def realpath(filename):
+def realpath(filename, *, strict=False):
"""Return the canonical path of the specified filename, eliminating any
symbolic links encountered in the path."""
filename = os.fspath(filename)
- path, ok = _joinrealpath(filename[:0], filename, {})
+ path, ok = _joinrealpath(filename[:0], filename, strict, {})
return abspath(path)
# Join two paths, normalizing and eliminating any symbolic links
# encountered in the second path.
-def _joinrealpath(path, rest, seen):
+def _joinrealpath(path, rest, strict, seen):
if isinstance(path, bytes):
sep = b'/'
curdir = b'.'
@@ -425,7 +425,15 @@ def _joinrealpath(path, rest, seen):
path = pardir
continue
newpath = join(path, name)
- if not islink(newpath):
+ try:
+ st = os.lstat(newpath)
+ except OSError:
+ if strict:
+ raise
+ is_link = False
+ else:
+ is_link = stat.S_ISLNK(st.st_mode)
+ if not is_link:
path = newpath
continue
# Resolve the symbolic link
@@ -436,10 +444,14 @@ def _joinrealpath(path, rest, seen):
# use cached value
continue
# The symlink is not resolved, so we must have a symlink loop.
- # Return already resolved part + rest of the path unchanged.
- return join(newpath, rest), False
+ if strict:
+ # Raise OSError(errno.ELOOP)
+ os.stat(newpath)
+ else:
+ # Return already resolved part + rest of the path unchanged.
+ return join(newpath, rest), False
seen[newpath] = None # not resolved symlink
- path, ok = _joinrealpath(path, os.readlink(newpath), seen)
+ path, ok = _joinrealpath(path, os.readlink(newpath), strict, seen)
if not ok:
return join(path, rest), False
seen[newpath] = path # resolved symlink
diff --git a/Lib/test/test_ntpath.py b/Lib/test/test_ntpath.py
index f97aca5f94f57..661c59d617163 100644
--- a/Lib/test/test_ntpath.py
+++ b/Lib/test/test_ntpath.py
@@ -269,6 +269,17 @@ def test_realpath_basic(self):
self.assertPathEqual(ntpath.realpath(os.fsencode(ABSTFN + "1")),
os.fsencode(ABSTFN))
+ @os_helper.skip_unless_symlink
+ @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
+ def test_realpath_strict(self):
+ # Bug #43757: raise FileNotFoundError in strict mode if we encounter
+ # a path that does not exist.
+ ABSTFN = ntpath.abspath(os_helper.TESTFN)
+ os.symlink(ABSTFN + "1", ABSTFN)
+ self.addCleanup(os_helper.unlink, ABSTFN)
+ self.assertRaises(FileNotFoundError, ntpath.realpath, ABSTFN, strict=True)
+ self.assertRaises(FileNotFoundError, ntpath.realpath, ABSTFN + "2", strict=True)
+
@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
def test_realpath_relative(self):
@@ -340,8 +351,9 @@ def test_realpath_broken_symlinks(self):
@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
def test_realpath_symlink_loops(self):
- # Symlink loops are non-deterministic as to which path is returned, but
- # it will always be the fully resolved path of one member of the cycle
+ # Symlink loops in non-strict mode are non-deterministic as to which
+ # path is returned, but it will always be the fully resolved path of
+ # one member of the cycle
ABSTFN = ntpath.abspath(os_helper.TESTFN)
self.addCleanup(os_helper.unlink, ABSTFN)
self.addCleanup(os_helper.unlink, ABSTFN + "1")
@@ -383,6 +395,50 @@ def test_realpath_symlink_loops(self):
# Test using relative path as well.
self.assertPathEqual(ntpath.realpath(ntpath.basename(ABSTFN)), ABSTFN)
+ @os_helper.skip_unless_symlink
+ @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
+ def test_realpath_symlink_loops_strict(self):
+ # Symlink loops raise OSError in strict mode
+ ABSTFN = ntpath.abspath(os_helper.TESTFN)
+ self.addCleanup(os_helper.unlink, ABSTFN)
+ self.addCleanup(os_helper.unlink, ABSTFN + "1")
+ self.addCleanup(os_helper.unlink, ABSTFN + "2")
+ self.addCleanup(os_helper.unlink, ABSTFN + "y")
+ self.addCleanup(os_helper.unlink, ABSTFN + "c")
+ self.addCleanup(os_helper.unlink, ABSTFN + "a")
+
+ os.symlink(ABSTFN, ABSTFN)
+ self.assertRaises(OSError, ntpath.realpath, ABSTFN, strict=True)
+
+ os.symlink(ABSTFN + "1", ABSTFN + "2")
+ os.symlink(ABSTFN + "2", ABSTFN + "1")
+ self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1", strict=True)
+ self.assertRaises(OSError, ntpath.realpath, ABSTFN + "2", strict=True)
+ self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1\\x", strict=True)
+ # Windows eliminates '..' components before resolving links, so the
+ # following call is not expected to raise.
+ self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\..", strict=True),
+ ntpath.dirname(ABSTFN))
+ self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1\\..\\x", strict=True)
+ os.symlink(ABSTFN + "x", ABSTFN + "y")
+ self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1\\..\\"
+ + ntpath.basename(ABSTFN) + "y",
+ strict=True)
+ self.assertRaises(OSError, ntpath.realpath,
+ ABSTFN + "1\\..\\" + ntpath.basename(ABSTFN) + "1",
+ strict=True)
+
+ os.symlink(ntpath.basename(ABSTFN) + "a\\b", ABSTFN + "a")
+ self.assertRaises(OSError, ntpath.realpath, ABSTFN + "a", strict=True)
+
+ os.symlink("..\\" + ntpath.basename(ntpath.dirname(ABSTFN))
+ + "\\" + ntpath.basename(ABSTFN) + "c", ABSTFN + "c")
+ self.assertRaises(OSError, ntpath.realpath, ABSTFN + "c", strict=True)
+
+ # Test using relative path as well.
+ self.assertRaises(OSError, ntpath.realpath, ntpath.basename(ABSTFN),
+ strict=True)
+
@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
def test_realpath_symlink_prefix(self):
diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py
index e18d01f4635a3..8d398ec010354 100644
--- a/Lib/test/test_posixpath.py
+++ b/Lib/test/test_posixpath.py
@@ -355,6 +355,19 @@ def test_realpath_basic(self):
finally:
os_helper.unlink(ABSTFN)
+ @unittest.skipUnless(hasattr(os, "symlink"),
+ "Missing symlink implementation")
+ @skip_if_ABSTFN_contains_backslash
+ def test_realpath_strict(self):
+ # Bug #43757: raise FileNotFoundError in strict mode if we encounter
+ # a path that does not exist.
+ try:
+ os.symlink(ABSTFN+"1", ABSTFN)
+ self.assertRaises(FileNotFoundError, realpath, ABSTFN, strict=True)
+ self.assertRaises(FileNotFoundError, realpath, ABSTFN + "2", strict=True)
+ finally:
+ os_helper.unlink(ABSTFN)
+
@unittest.skipUnless(hasattr(os, "symlink"),
"Missing symlink implementation")
@skip_if_ABSTFN_contains_backslash
@@ -370,7 +383,7 @@ def test_realpath_relative(self):
@skip_if_ABSTFN_contains_backslash
def test_realpath_symlink_loops(self):
# Bug #930024, return the path unchanged if we get into an infinite
- # symlink loop.
+ # symlink loop in non-strict mode (default).
try:
os.symlink(ABSTFN, ABSTFN)
self.assertEqual(realpath(ABSTFN), ABSTFN)
@@ -407,6 +420,48 @@ def test_realpath_symlink_loops(self):
os_helper.unlink(ABSTFN+"c")
os_helper.unlink(ABSTFN+"a")
+ @unittest.skipUnless(hasattr(os, "symlink"),
+ "Missing symlink implementation")
+ @skip_if_ABSTFN_contains_backslash
+ def test_realpath_symlink_loops_strict(self):
+ # Bug #43757, raise OSError if we get into an infinite symlink loop in
+ # strict mode.
+ try:
+ os.symlink(ABSTFN, ABSTFN)
+ self.assertRaises(OSError, realpath, ABSTFN, strict=True)
+
+ os.symlink(ABSTFN+"1", ABSTFN+"2")
+ os.symlink(ABSTFN+"2", ABSTFN+"1")
+ self.assertRaises(OSError, realpath, ABSTFN+"1", strict=True)
+ self.assertRaises(OSError, realpath, ABSTFN+"2", strict=True)
+
+ self.assertRaises(OSError, realpath, ABSTFN+"1/x", strict=True)
+ self.assertRaises(OSError, realpath, ABSTFN+"1/..", strict=True)
+ self.assertRaises(OSError, realpath, ABSTFN+"1/../x", strict=True)
+ os.symlink(ABSTFN+"x", ABSTFN+"y")
+ self.assertRaises(OSError, realpath,
+ ABSTFN+"1/../" + basename(ABSTFN) + "y", strict=True)
+ self.assertRaises(OSError, realpath,
+ ABSTFN+"1/../" + basename(ABSTFN) + "1", strict=True)
+
+ os.symlink(basename(ABSTFN) + "a/b", ABSTFN+"a")
+ self.assertRaises(OSError, realpath, ABSTFN+"a", strict=True)
+
+ os.symlink("../" + basename(dirname(ABSTFN)) + "/" +
+ basename(ABSTFN) + "c", ABSTFN+"c")
+ self.assertRaises(OSError, realpath, ABSTFN+"c", strict=True)
+
+ # Test using relative path as well.
+ with os_helper.change_cwd(dirname(ABSTFN)):
+ self.assertRaises(OSError, realpath, basename(ABSTFN), strict=True)
+ finally:
+ os_helper.unlink(ABSTFN)
+ os_helper.unlink(ABSTFN+"1")
+ os_helper.unlink(ABSTFN+"2")
+ os_helper.unlink(ABSTFN+"y")
+ os_helper.unlink(ABSTFN+"c")
+ os_helper.unlink(ABSTFN+"a")
+
@unittest.skipUnless(hasattr(os, "symlink"),
"Missing symlink implementation")
@skip_if_ABSTFN_contains_backslash
diff --git a/Misc/NEWS.d/next/Library/2021-04-08-22-11-27.bpo-25264.b33fa0.rst b/Misc/NEWS.d/next/Library/2021-04-08-22-11-27.bpo-25264.b33fa0.rst
new file mode 100644
index 0000000000000..593846ec15c5b
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2021-04-08-22-11-27.bpo-25264.b33fa0.rst
@@ -0,0 +1,3 @@
+:func:`os.path.realpath` now accepts a *strict* keyword-only argument.
+When set to ``True``, :exc:`OSError` is raised if a path doesn't exist
+or a symlink loop is encountered.
1
0
bpo-43960: test_pdb resets breakpoints to make tests deterministic (GH-25691) (GH-25692)
by vstinner 28 Apr '21
by vstinner 28 Apr '21
28 Apr '21
https://github.com/python/cpython/commit/b52cc7c5f1a6c5b48d51cd718719a766c3…
commit: b52cc7c5f1a6c5b48d51cd718719a766c37d6e38
branch: 3.8
author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com>
committer: vstinner <vstinner(a)python.org>
date: 2021-04-28T17:43:01+02:00
summary:
bpo-43960: test_pdb resets breakpoints to make tests deterministic (GH-25691) (GH-25692)
(cherry picked from commit 2dc6b1789ec86dc80ea290fe33edd61140e47f6f)
Co-authored-by: Irit Katriel <iritkatriel(a)yahoo.com>
Co-authored-by: Irit Katriel <iritkatriel(a)yahoo.com>
files:
M Lib/test/test_pdb.py
diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py
index ac731fefdd24b2..cdee716d6ac0ec 100644
--- a/Lib/test/test_pdb.py
+++ b/Lib/test/test_pdb.py
@@ -17,6 +17,13 @@
from test.test_doctest import _FakeInput
from unittest.mock import patch
+from bdb import Breakpoint
+
+def reset_Breakpoint():
+ Breakpoint.next = 1
+ Breakpoint.bplist = {}
+ Breakpoint.bpbynumber = [None]
+
class PdbTestInput(object):
"""Context manager that makes testing Pdb in doctests easier."""
@@ -227,10 +234,7 @@ def test_pdb_breakpoint_commands():
First, need to clear bdb state that might be left over from previous tests.
Otherwise, the new breakpoints might get assigned different numbers.
- >>> from bdb import Breakpoint
- >>> Breakpoint.next = 1
- >>> Breakpoint.bplist = {}
- >>> Breakpoint.bpbynumber = [None]
+ >>> reset_Breakpoint()
Now test the breakpoint commands. NORMALIZE_WHITESPACE is needed because
the breakpoint list outputs a tab for the "stop only" and "ignore next"
@@ -699,8 +703,7 @@ def test_next_until_return_at_return_event():
... test_function_2()
... end = 1
- >>> from bdb import Breakpoint
- >>> Breakpoint.next = 1
+ >>> reset_Breakpoint()
>>> with PdbTestInput(['break test_function_2',
... 'continue',
... 'return',
@@ -1127,6 +1130,7 @@ def test_pdb_next_command_in_generator_for_loop():
... print('value', i)
... x = 123
+ >>> reset_Breakpoint()
>>> with PdbTestInput(['break test_gen',
... 'continue',
... 'next',
@@ -1137,7 +1141,7 @@ def test_pdb_next_command_in_generator_for_loop():
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
-> for i in test_gen():
(Pdb) break test_gen
- Breakpoint 6 at <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>:1
+ Breakpoint 1 at <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>:1
(Pdb) continue
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(2)test_gen()
-> yield 0
1
0
28 Apr '21
https://github.com/python/cpython/commit/859577c24981d6b36960d309f99f7fc810…
commit: 859577c24981d6b36960d309f99f7fc810fe75c2
branch: master
author: Ken Jin <28750310+Fidget-Spinner(a)users.noreply.github.com>
committer: gvanrossum <gvanrossum(a)gmail.com>
date: 2021-04-28T08:38:14-07:00
summary:
bpo-41559: Change PEP 612 implementation to pure Python (#25449)
files:
A Misc/NEWS.d/next/Library/2021-04-17-10-49-57.bpo-41559.caIwt9.rst
M Lib/_collections_abc.py
M Lib/test/test_genericalias.py
M Objects/genericaliasobject.c
diff --git a/Lib/_collections_abc.py b/Lib/_collections_abc.py
index 87302ac76d801..dddf8a23ff985 100644
--- a/Lib/_collections_abc.py
+++ b/Lib/_collections_abc.py
@@ -443,6 +443,18 @@ def __create_ga(cls, origin, args):
ga_args = args
return super().__new__(cls, origin, ga_args)
+ @property
+ def __parameters__(self):
+ params = []
+ for arg in self.__args__:
+ # Looks like a genericalias
+ if hasattr(arg, "__parameters__") and isinstance(arg.__parameters__, tuple):
+ params.extend(arg.__parameters__)
+ else:
+ if _is_typevarlike(arg):
+ params.append(arg)
+ return tuple(dict.fromkeys(params))
+
def __repr__(self):
if _has_special_args(self.__args__):
return super().__repr__()
@@ -458,16 +470,50 @@ def __reduce__(self):
def __getitem__(self, item):
# Called during TypeVar substitution, returns the custom subclass
- # rather than the default types.GenericAlias object.
- ga = super().__getitem__(item)
- args = ga.__args__
- # args[0] occurs due to things like Z[[int, str, bool]] from PEP 612
- if not isinstance(ga.__args__[0], tuple):
- t_result = ga.__args__[-1]
- t_args = ga.__args__[:-1]
- args = (t_args, t_result)
- return _CallableGenericAlias(Callable, args)
+ # rather than the default types.GenericAlias object. Most of the
+ # code is copied from typing's _GenericAlias and the builtin
+ # types.GenericAlias.
+
+ # A special case in PEP 612 where if X = Callable[P, int],
+ # then X[int, str] == X[[int, str]].
+ param_len = len(self.__parameters__)
+ if param_len == 0:
+ raise TypeError(f'There are no type or parameter specification'
+ f'variables left in {self}')
+ if (param_len == 1
+ and isinstance(item, (tuple, list))
+ and len(item) > 1) or not isinstance(item, tuple):
+ item = (item,)
+ item_len = len(item)
+ if item_len != param_len:
+ raise TypeError(f'Too {"many" if item_len > param_len else "few"}'
+ f' arguments for {self};'
+ f' actual {item_len}, expected {param_len}')
+ subst = dict(zip(self.__parameters__, item))
+ new_args = []
+ for arg in self.__args__:
+ if _is_typevarlike(arg):
+ arg = subst[arg]
+ # Looks like a GenericAlias
+ elif hasattr(arg, '__parameters__') and isinstance(arg.__parameters__, tuple):
+ subparams = arg.__parameters__
+ if subparams:
+ subargs = tuple(subst[x] for x in subparams)
+ arg = arg[subargs]
+ new_args.append(arg)
+ # args[0] occurs due to things like Z[[int, str, bool]] from PEP 612
+ if not isinstance(new_args[0], (tuple, list)):
+ t_result = new_args[-1]
+ t_args = new_args[:-1]
+ new_args = (t_args, t_result)
+ return _CallableGenericAlias(Callable, tuple(new_args))
+
+def _is_typevarlike(arg):
+ obj = type(arg)
+ # looks like a TypeVar/ParamSpec
+ return (obj.__module__ == 'typing'
+ and obj.__name__ in {'ParamSpec', 'TypeVar'})
def _has_special_args(args):
"""Checks if args[0] matches either ``...``, ``ParamSpec`` or
diff --git a/Lib/test/test_genericalias.py b/Lib/test/test_genericalias.py
index fd024dcec8208..9f927392fc874 100644
--- a/Lib/test/test_genericalias.py
+++ b/Lib/test/test_genericalias.py
@@ -353,6 +353,12 @@ def test_abc_callable(self):
self.assertEqual(repr(C4[dict]).split(".")[-1], "Callable[[int, dict], str]")
self.assertEqual(C4[dict], Callable[[int, dict], str])
+ # substitute a nested GenericAlias (both typing and the builtin
+ # version)
+ C5 = Callable[[typing.List[T], tuple[K, T], V], int]
+ self.assertEqual(C5[int, str, float],
+ Callable[[typing.List[int], tuple[str, int], float], int])
+
with self.subTest("Testing type erasure"):
class C1(Callable):
def __call__(self):
@@ -391,5 +397,16 @@ def __call__(self):
self.assertEqual(repr(C1), "collections.abc.Callable"
"[typing.Concatenate[int, ~P], int]")
+ with self.subTest("Testing TypeErrors"):
+ with self.assertRaisesRegex(TypeError, "variables left in"):
+ alias[int]
+ P = typing.ParamSpec('P')
+ C1 = Callable[P, T]
+ with self.assertRaisesRegex(TypeError, "many arguments for"):
+ C1[int, str, str]
+ with self.assertRaisesRegex(TypeError, "few arguments for"):
+ C1[int]
+
+
if __name__ == "__main__":
unittest.main()
diff --git a/Misc/NEWS.d/next/Library/2021-04-17-10-49-57.bpo-41559.caIwt9.rst b/Misc/NEWS.d/next/Library/2021-04-17-10-49-57.bpo-41559.caIwt9.rst
new file mode 100644
index 0000000000000..11db42350eb7e
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2021-04-17-10-49-57.bpo-41559.caIwt9.rst
@@ -0,0 +1,6 @@
+:pep:`612` is now implemented purely in Python; builtin ``types.GenericAlias``
+objects no longer include ``typing.ParamSpec`` in ``__parameters__``
+(with the exception of ``collections.abc.Callable``\ 's ``GenericAlias``).
+This means previously invalid uses of ``ParamSpec`` (such as
+``list[P]``) which worked in earlier versions of Python 3.10 alpha,
+will now raise ``TypeError`` during substitution.
diff --git a/Objects/genericaliasobject.c b/Objects/genericaliasobject.c
index 8fae83b27297d..756a7ce474aee 100644
--- a/Objects/genericaliasobject.c
+++ b/Objects/genericaliasobject.c
@@ -156,25 +156,13 @@ ga_repr(PyObject *self)
return NULL;
}
-/* Checks if a variable number of names are from typing.py.
-* If any one of the names are found, return 1, else 0.
-**/
-static inline int
-is_typing_name(PyObject *obj, int num, ...)
+// isinstance(obj, TypeVar) without importing typing.py.
+// Returns -1 for errors.
+static int
+is_typevar(PyObject *obj)
{
- va_list names;
- va_start(names, num);
-
PyTypeObject *type = Py_TYPE(obj);
- int hit = 0;
- for (int i = 0; i < num; ++i) {
- if (!strcmp(type->tp_name, va_arg(names, const char *))) {
- hit = 1;
- break;
- }
- }
- va_end(names);
- if (!hit) {
+ if (strcmp(type->tp_name, "TypeVar") != 0) {
return 0;
}
PyObject *module = PyObject_GetAttrString((PyObject *)type, "__module__");
@@ -184,24 +172,9 @@ is_typing_name(PyObject *obj, int num, ...)
int res = PyUnicode_Check(module)
&& _PyUnicode_EqualToASCIIString(module, "typing");
Py_DECREF(module);
-
return res;
}
-// isinstance(obj, (TypeVar, ParamSpec)) without importing typing.py.
-// Returns -1 for errors.
-static inline int
-is_typevarlike(PyObject *obj)
-{
- return is_typing_name(obj, 2, "TypeVar", "ParamSpec");
-}
-
-static inline int
-is_paramspec(PyObject *obj)
-{
- return is_typing_name(obj, 1, "ParamSpec");
-}
-
// Index of item in self[:len], or -1 if not found (self is a tuple)
static Py_ssize_t
tuple_index(PyObject *self, Py_ssize_t len, PyObject *item)
@@ -236,7 +209,7 @@ make_parameters(PyObject *args)
Py_ssize_t iparam = 0;
for (Py_ssize_t iarg = 0; iarg < nargs; iarg++) {
PyObject *t = PyTuple_GET_ITEM(args, iarg);
- int typevar = is_typevarlike(t);
+ int typevar = is_typevar(t);
if (typevar < 0) {
Py_DECREF(parameters);
return NULL;
@@ -306,14 +279,7 @@ subs_tvars(PyObject *obj, PyObject *params, PyObject **argitems)
if (iparam >= 0) {
arg = argitems[iparam];
}
- // convert all the lists inside args to tuples to help
- // with caching in other libaries
- if (PyList_CheckExact(arg)) {
- arg = PyList_AsTuple(arg);
- }
- else {
- Py_INCREF(arg);
- }
+ Py_INCREF(arg);
PyTuple_SET_ITEM(subargs, i, arg);
}
@@ -348,19 +314,11 @@ ga_getitem(PyObject *self, PyObject *item)
int is_tuple = PyTuple_Check(item);
Py_ssize_t nitems = is_tuple ? PyTuple_GET_SIZE(item) : 1;
PyObject **argitems = is_tuple ? &PyTuple_GET_ITEM(item, 0) : &item;
- // A special case in PEP 612 where if X = Callable[P, int],
- // then X[int, str] == X[[int, str]].
- if (nparams == 1 && nitems > 1 && is_tuple &&
- is_paramspec(PyTuple_GET_ITEM(alias->parameters, 0))) {
- argitems = &item;
- }
- else {
- if (nitems != nparams) {
- return PyErr_Format(PyExc_TypeError,
- "Too %s arguments for %R",
- nitems > nparams ? "many" : "few",
- self);
- }
+ if (nitems != nparams) {
+ return PyErr_Format(PyExc_TypeError,
+ "Too %s arguments for %R",
+ nitems > nparams ? "many" : "few",
+ self);
}
/* Replace all type variables (specified by alias->parameters)
with corresponding values specified by argitems.
@@ -375,7 +333,7 @@ ga_getitem(PyObject *self, PyObject *item)
}
for (Py_ssize_t iarg = 0; iarg < nargs; iarg++) {
PyObject *arg = PyTuple_GET_ITEM(alias->args, iarg);
- int typevar = is_typevarlike(arg);
+ int typevar = is_typevar(arg);
if (typevar < 0) {
Py_DECREF(newargs);
return NULL;
@@ -384,13 +342,7 @@ ga_getitem(PyObject *self, PyObject *item)
Py_ssize_t iparam = tuple_index(alias->parameters, nparams, arg);
assert(iparam >= 0);
arg = argitems[iparam];
- // convert lists to tuples to help with caching in other libaries.
- if (PyList_CheckExact(arg)) {
- arg = PyList_AsTuple(arg);
- }
- else {
- Py_INCREF(arg);
- }
+ Py_INCREF(arg);
}
else {
arg = subs_tvars(arg, alias->parameters, argitems);
1
0