[Python-checkins] bpo-43592: Raise RLIMIT_NOFILE in test.libregrtest (GH-29127)

[gpshead]

https://github.com/python/cpython/commit/843b890334ca30cf6af27dffe29cecd06b49f7d9
commit: 843b890334ca30cf6af27dffe29cecd06b49f7d9
branch: main
author: Gregory P. Smith <greg at krypto.org>
committer: gpshead <greg at krypto.org>
date: 2021-10-21T18:22:25-07:00
summary:

bpo-43592: Raise RLIMIT_NOFILE in test.libregrtest (GH-29127)

Raise RLIMIT_NOFILE in test.libregrtest.

On macOS the default is often too low for our testsuite to succeed.

Co-authored by reviewer: Victor Stinner

files:
A Misc/NEWS.d/next/Tests/2021-10-21-17-22-26.bpo-43592.kHRsra.rst
M Lib/test/libregrtest/setup.py

diff --git a/Lib/test/libregrtest/setup.py b/Lib/test/libregrtest/setup.py
index f9460ae7ed18f..4ffd15478f643 100644
--- a/Lib/test/libregrtest/setup.py
+++ b/Lib/test/libregrtest/setup.py
@@ -39,6 +39,7 @@ def setup_tests(ns):
         for signum in signals:
             faulthandler.register(signum, chain=True, file=stderr_fd)
 
+    _adjust_resource_limits()
     replace_stdout()
     support.record_original_stdout(sys.stdout)
 
@@ -133,3 +134,26 @@ def restore_stdout():
         sys.stdout.close()
         sys.stdout = stdout
     atexit.register(restore_stdout)
+
+
+def _adjust_resource_limits():
+    """Adjust the system resource limits (ulimit) if needed."""
+    try:
+        import resource
+        from resource import RLIMIT_NOFILE, RLIM_INFINITY
+    except ImportError:
+        return
+    fd_limit, max_fds = resource.getrlimit(RLIMIT_NOFILE)
+    # On macOS the default fd limit is sometimes too low (256) for our
+    # test suite to succeed.  Raise it to something more reasonable.
+    # 1024 is a common Linux default.
+    desired_fds = 1024
+    if fd_limit < desired_fds and fd_limit < max_fds:
+        new_fd_limit = min(desired_fds, max_fds)
+        try:
+            resource.setrlimit(RLIMIT_NOFILE, (new_fd_limit, max_fds))
+            print(f"Raised RLIMIT_NOFILE: {fd_limit} -> {new_fd_limit}")
+        except (ValueError, OSError) as err:
+            print(f"Unable to raise RLIMIT_NOFILE from {fd_limit} to "
+                  f"{new_fd_limit}: {err}.")
+
diff --git a/Misc/NEWS.d/next/Tests/2021-10-21-17-22-26.bpo-43592.kHRsra.rst b/Misc/NEWS.d/next/Tests/2021-10-21-17-22-26.bpo-43592.kHRsra.rst
new file mode 100644
index 0000000000000..2528857caf9c0
--- /dev/null
+++ b/Misc/NEWS.d/next/Tests/2021-10-21-17-22-26.bpo-43592.kHRsra.rst
@@ -0,0 +1,3 @@
+:mod:`test.libregrtest` now raises the soft resource limit for the maximum
+number of file descriptors when the default is too low for our test suite as
+was often the case on macOS.