[Python-checkins] bpo-43592: Raise RLIMIT_NOFILE in test.libregrtest (GH-29127)

[miss-islington]

https://github.com/python/cpython/commit/216c040bb1fdf73e78d67ab82a43563d7593f874
commit: 216c040bb1fdf73e78d67ab82a43563d7593f874
branch: 3.9
author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com>
committer: miss-islington <31488909+miss-islington at users.noreply.github.com>
date: 2021-10-21T21:48:44-07:00
summary:

bpo-43592: Raise RLIMIT_NOFILE in test.libregrtest (GH-29127)


Raise RLIMIT_NOFILE in test.libregrtest.

On macOS the default is often too low for our testsuite to succeed.

Co-authored by reviewer: Victor Stinner
(cherry picked from commit 843b890334ca30cf6af27dffe29cecd06b49f7d9)

Co-authored-by: Gregory P. Smith <greg at krypto.org>

files:
A Misc/NEWS.d/next/Tests/2021-10-21-17-22-26.bpo-43592.kHRsra.rst
M Lib/test/libregrtest/setup.py

diff --git a/Lib/test/libregrtest/setup.py b/Lib/test/libregrtest/setup.py
index 1f264c1be49fe..cedf905c35c55 100644
--- a/Lib/test/libregrtest/setup.py
+++ b/Lib/test/libregrtest/setup.py
@@ -35,6 +35,7 @@ def setup_tests(ns):
         for signum in signals:
             faulthandler.register(signum, chain=True, file=stderr_fd)
 
+    _adjust_resource_limits()
     replace_stdout()
     support.record_original_stdout(sys.stdout)
 
@@ -117,3 +118,26 @@ def restore_stdout():
         sys.stdout.close()
         sys.stdout = stdout
     atexit.register(restore_stdout)
+
+
+def _adjust_resource_limits():
+    """Adjust the system resource limits (ulimit) if needed."""
+    try:
+        import resource
+        from resource import RLIMIT_NOFILE, RLIM_INFINITY
+    except ImportError:
+        return
+    fd_limit, max_fds = resource.getrlimit(RLIMIT_NOFILE)
+    # On macOS the default fd limit is sometimes too low (256) for our
+    # test suite to succeed.  Raise it to something more reasonable.
+    # 1024 is a common Linux default.
+    desired_fds = 1024
+    if fd_limit < desired_fds and fd_limit < max_fds:
+        new_fd_limit = min(desired_fds, max_fds)
+        try:
+            resource.setrlimit(RLIMIT_NOFILE, (new_fd_limit, max_fds))
+            print(f"Raised RLIMIT_NOFILE: {fd_limit} -> {new_fd_limit}")
+        except (ValueError, OSError) as err:
+            print(f"Unable to raise RLIMIT_NOFILE from {fd_limit} to "
+                  f"{new_fd_limit}: {err}.")
+
diff --git a/Misc/NEWS.d/next/Tests/2021-10-21-17-22-26.bpo-43592.kHRsra.rst b/Misc/NEWS.d/next/Tests/2021-10-21-17-22-26.bpo-43592.kHRsra.rst
new file mode 100644
index 0000000000000..2528857caf9c0
--- /dev/null
+++ b/Misc/NEWS.d/next/Tests/2021-10-21-17-22-26.bpo-43592.kHRsra.rst
@@ -0,0 +1,3 @@
+:mod:`test.libregrtest` now raises the soft resource limit for the maximum
+number of file descriptors when the default is too low for our test suite as
+was often the case on macOS.