Preventing out of memory conditions
Sometimes, I have the flexibility to reduce the memory used by my program (e.g., by destroying large cached objects, etc.). It would be great if I could ask Python interpreter to notify me when memory is running out, so I can take such actions. Of course, it's nearly impossible for Python to know in advance if the OS would run out of memory with the next malloc call. Furthermore, Python shouldn't guess which memory (physical, virtual, etc.) is relevant in the particular situation (for instance, in my case, I only care about physical memory, since swapping to disk makes my application as good as frozen). So the problem as stated above is unsolvable. But let's say I am willing to do some work to estimate the maximum amount of memory my application can be allowed to use. If I provide that number to Python interpreter, it may be possible for it to notify me when the next memory allocation would exceed this limit by calling a function I provide it (hopefully passing as arguments the amount of memory being requested, as well as the amount currently in use). My callback function could then destroy some objects, and return True to indicate that some objects were destroyed. At that point, the intepreter could run its standard garbage collection routines to release the memory that corresponded to those objects - before proceeding with whatever it was trying to do originally. (If I returned False, or if I didn't provide a callback function at all, the interpreter would simply behave as it does today.) Any memory allocations that happen while the callback function itself is executing, would not trigger further calls to it. The whole mechanism would be disabled for the rest of the session if the memory freed by the callback function was insufficient to prevent going over the memory limit. Would this be worth considering for a future language extension? How hard would it be to implement? Max
Interesting idea, though I don't think it's something that should be a Python language extension. For instance, iOS (typically more resource-constrained) sends the application signals when system memory is getting low so it can free stuff -- this is done at the OS level. And I think that's the right place, because this will almost certainly be setup- and system-dependent. For instance, it would depend hugely on whether there's a virtual memory manager, and how it's configured. I'd say your best bet is to write a little library that does the appropriate thing for your needs (your system or setup). Say starts a thread that checks the system's free memory every so often and sends your application a signal/callback saying "we're getting low, free some of your caches". It could even send a "level flag" to your callback saying "fairly low", "very low", or "critically low" -- I think iOS does this. -Ben On Tue, Jan 1, 2013 at 11:16 AM, Max Moroz <maxmoroz@gmail.com> wrote:
Sometimes, I have the flexibility to reduce the memory used by my program (e.g., by destroying large cached objects, etc.). It would be great if I could ask Python interpreter to notify me when memory is running out, so I can take such actions.
Of course, it's nearly impossible for Python to know in advance if the OS would run out of memory with the next malloc call. Furthermore, Python shouldn't guess which memory (physical, virtual, etc.) is relevant in the particular situation (for instance, in my case, I only care about physical memory, since swapping to disk makes my application as good as frozen). So the problem as stated above is unsolvable.
But let's say I am willing to do some work to estimate the maximum amount of memory my application can be allowed to use. If I provide that number to Python interpreter, it may be possible for it to notify me when the next memory allocation would exceed this limit by calling a function I provide it (hopefully passing as arguments the amount of memory being requested, as well as the amount currently in use). My callback function could then destroy some objects, and return True to indicate that some objects were destroyed. At that point, the intepreter could run its standard garbage collection routines to release the memory that corresponded to those objects - before proceeding with whatever it was trying to do originally. (If I returned False, or if I didn't provide a callback function at all, the interpreter would simply behave as it does today.) Any memory allocations that happen while the callback function itself is executing, would not trigger further calls to it. The whole mechanism would be disabled for the rest of the session if the memory freed by the callback function was insufficient to prevent going over the memory limit.
Would this be worth considering for a future language extension? How hard would it be to implement?
Max _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
Within CPython the way the C API is today it is too late by the time the code to raise a MemoryError has been called so capturing all places that could occur is not easy. Implementing this at the C level malloc later makes more sense. Have it dip into a reserved low memory pool to satisfy the current request and send the process a signal indicating it is running low. This approach would also work with C extension modules or an embedded Python. I'd expect this already exists but I haven't looked for one. Having a thread polling memory use it not generally wise as that is polling rather than event driven and could easily miss low memory situations before it is too late and a failure has already happened (allocation demand can come in large spikes depending on the application). OSes running processes in constrained environments or ones where the resources available can be reduced by the OS later may already send their own warning signals prior to outright killing the process but that should not preclude an application being able to monitor and constrain itself on its own without needing the OS to do it. -gps On Mon, Dec 31, 2012 at 4:11 PM, Ben Hoyt <benhoyt@gmail.com> wrote:
Interesting idea, though I don't think it's something that should be a Python language extension. For instance, iOS (typically more resource-constrained) sends the application signals when system memory is getting low so it can free stuff -- this is done at the OS level. And I think that's the right place, because this will almost certainly be setup- and system-dependent. For instance, it would depend hugely on whether there's a virtual memory manager, and how it's configured.
I'd say your best bet is to write a little library that does the appropriate thing for your needs (your system or setup). Say starts a thread that checks the system's free memory every so often and sends your application a signal/callback saying "we're getting low, free some of your caches". It could even send a "level flag" to your callback saying "fairly low", "very low", or "critically low" -- I think iOS does this.
-Ben
On Tue, Jan 1, 2013 at 11:16 AM, Max Moroz <maxmoroz@gmail.com> wrote:
Sometimes, I have the flexibility to reduce the memory used by my program (e.g., by destroying large cached objects, etc.). It would be great if I could ask Python interpreter to notify me when memory is running out, so I can take such actions.
Of course, it's nearly impossible for Python to know in advance if the OS would run out of memory with the next malloc call. Furthermore, Python shouldn't guess which memory (physical, virtual, etc.) is relevant in the particular situation (for instance, in my case, I only care about physical memory, since swapping to disk makes my application as good as frozen). So the problem as stated above is unsolvable.
But let's say I am willing to do some work to estimate the maximum amount of memory my application can be allowed to use. If I provide that number to Python interpreter, it may be possible for it to notify me when the next memory allocation would exceed this limit by calling a function I provide it (hopefully passing as arguments the amount of memory being requested, as well as the amount currently in use). My callback function could then destroy some objects, and return True to indicate that some objects were destroyed. At that point, the intepreter could run its standard garbage collection routines to release the memory that corresponded to those objects - before proceeding with whatever it was trying to do originally. (If I returned False, or if I didn't provide a callback function at all, the interpreter would simply behave as it does today.) Any memory allocations that happen while the callback function itself is executing, would not trigger further calls to it. The whole mechanism would be disabled for the rest of the session if the memory freed by the callback function was insufficient to prevent going over the memory limit.
Would this be worth considering for a future language extension? How hard would it be to implement?
Max _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On Mon, Dec 31, 2012 at 7:22 PM, Gregory P. Smith <greg@krypto.org> wrote:
Within CPython the way the C API is today it is too late by the time the code to raise a MemoryError has been called so capturing all places that could occur is not easy. Implementing this at the C level malloc later makes more sense. Have it dip into a reserved low memory pool to satisfy the current request and send the process a signal indicating it is running low. This approach would also work with C extension modules or an embedded Python.
Regarding the C malloc solution, wouldn't a callback be preferable to a signal? If I understood you correctly, signal implies that a different thread will handle it. At any reasonable size of the emergency memory pool is, there will be situations when the next memory allocation is greater than that size, leading to the very same problem you described later in your message when you talked about the disadvantage of polling. In addition, if the signal processing happens a bit slow (perhaps simply due to the thread scheduler being slow to switch), by the time enough memory is released, it may be too late - the next memory allocation may have already come in. Unless I'm missing something, the (synchronous) callback seems to be a strictly better than the (asynchronous) signal. As to your main point that this functionality should be inside C malloc rather than pymalloc, I agree, but only if the objective is to provide an all-purpose, highly general "low memory condition" handling. (I'm not sure if malloc knows enough about the OS to define "low memory condition" well; but it's certain that pymalloc doesn't). But I was going for a more modest goal. Rather than be warned of the pending for MemoryError exception, a developer could simply be notified via callback when the maximum absolute memory used by his app exceeds a certain limit. pymalloc could very easily call back a designated function when when the next memory allocation exceeds this threshold. In many real-life situations, it's not that hard to estimate how much RAM the application should be allowed to consume. Sure, the developer would need to learn a little about the platforms his app is running on, and use OS-specific rules to set the memory limit, but that effort is modest, and the payoff is huge. Not to mention, a developer with a particularly technically savvy end users could even skip this work entirely by letting his end users set the memory limit per-session. There is a huge advantage of the pymalloc solution (with the set memory limit) vs. the C malloc solution (with the generic low memory condition). On my system, I don't want the application to use (almost) all the available memory before it starts to manage its cache. In fact, by the time the physical memory use approaches my total physical RAM, the system slows down considerably as many other applications get swapped to disk by the OS. With a set memory limit, I can provide a much more granular control over the memory used by the application. Of course, the set memory limit could also be implemented inside C malloc rather than inside pymalloc. But this requires that developers rewrite C runtime's memory manager on every platform, and then recompile their Python with it. The changes to pymalloc, on the other hand, would be relatively small.
I'd expect this already exists but I haven't looked for one.
All I found is this comment in XEmacs documentation about vm-limit.c: http://www.xemacs.org/Documentation/21.5/html/internals_17.html, but I'm not sure if it's XEmacs feature or if malloc itself supports it.
Having a thread polling memory use it not generally wise as that is polling rather than event driven and could easily miss low memory situations before it is too late and a failure has already happened (allocation demand can come in large spikes depending on the application).
Precisely. That's the problem with the best existing solutions (e.g., http://stackoverflow.com/a/7332782/336527).
OSes running processes in constrained environments or ones where the resources available can be reduced by the OS later may already send their own warning signals prior to outright killing the process but that should not preclude an application being able to monitor and constrain itself on its own without needing the OS to do it.
I was thinking about regular desktop OS, which certainly doesn't warn the process sufficiently in advance. The MemoryError exception basically tells the process that it's going to die soon, and there's nothing it can do about it. Max
Max Moroz writes:
All I found is this comment in XEmacs documentation about vm-limit.c: http://www.xemacs.org/Documentation/21.5/html/internals_17.html, but I'm not sure if it's XEmacs feature or if malloc itself supports it.
It's an XEmacs feature. Works for me (but then it would, wouldn't it ;-). The implementation is just generic C, except for the macros that are used to access the LISP arena's bounds. It uses standard functions like getrlimit where available, otherwise it just uses the end of the address space to determine the available amount of memory. I can't vouch for accuracy or efficiency in determining usage (which is why I didn't bring it up myself), but there hasn't been a complaint about its functionality since I've been consistently reading the lists (1997). https://bitbucket.org/xemacs/xemacs-beta/src/c65b0329894b09c08423739508d2775... https://bitbucket.org/xemacs/xemacs-beta/src/c65b0329894b09c08423739508d2775...
On 12/31/2012 7:11 PM, Ben Hoyt wrote:
Interesting idea, though I don't think it's something that should be a Python language extension. For instance, iOS (typically more resource-constrained) sends the application signals when system memory is getting low so it can free stuff -- this is done at the OS level. And I think that's the right place, because this will almost certainly be setup- and system-dependent. For instance, it would depend hugely on whether there's a virtual memory manager, and how it's configured.
I'd say your best bet is to write a little library that does the appropriate thing for your needs (your system or setup). Say starts a thread that checks the system's free memory every so often and sends your application a signal/callback saying "we're getting low, free some of your caches". It could even send a "level flag" to your callback saying "fairly low", "very low", or "critically low" -- I think iOS does this.
I'm concerned that a program that does this will end up as the loser in this scenario: http://blogs.msdn.com/b/oldnewthing/archive/2012/01/18/10257834.aspx (tl;dr, two programs each having a different idea of how much free memory the system should have results in an "unfair" total allocation of memory) I think it's possibly important to avoid using the system's free memory as an input to any such system.
On Mon, Dec 31, 2012 at 7:28 PM, Random832 <random832@fastmail.us> wrote:
On 12/31/2012 7:11 PM, Ben Hoyt wrote:
Interesting idea, though I don't think it's something that should be a Python language extension. For instance, iOS (typically more resource-constrained) sends the application signals when system memory is getting low so it can free stuff -- this is done at the OS level. And I think that's the right place, because this will almost certainly be setup- and system-dependent. For instance, it would depend hugely on whether there's a virtual memory manager, and how it's configured.
I'd say your best bet is to write a little library that does the appropriate thing for your needs (your system or setup). Say starts a thread that checks the system's free memory every so often and sends your application a signal/callback saying "we're getting low, free some of your caches". It could even send a "level flag" to your callback saying "fairly low", "very low", or "critically low" -- I think iOS does this.
I'm concerned that a program that does this will end up as the loser in this scenario:
(tl;dr, two programs each having a different idea of how much free memory the system should have results in an "unfair" total allocation of memory)
I think it's possibly important to avoid using the system's free memory as an input to any such system.
indeed. only look at your own process's consumption vs. some numerical limit you've chosen for yourself. this also means you can adjust your own limit up or down at runtime if desired. (JVM's tend to force you to work this way) -gps
On 12/31/12, Max Moroz <maxmoroz@gmail.com> wrote:
Sometimes, I have the flexibility to reduce the memory used by my program (e.g., by destroying large cached objects, etc.). It would be great if I could ask Python interpreter to notify me when memory is running out, so I can take such actions.
Agreed, provided the overhead isn't too high. Depending on how accurately and precisely you need to track the memory usage, it might be enough to replace Objects/obmalloc.c new_arena with a wrapper that calls your callback before (maybe) allocating a new arena. -jJ
On 01.01.13 00:16, Max Moroz wrote:
But let's say I am willing to do some work to estimate the maximum amount of memory my application can be allowed to use. If I provide that number to Python interpreter, it may be possible for it to notify me when the next memory allocation would exceed this limit by calling a function I provide it (hopefully passing as arguments the amount of memory being requested, as well as the amount currently in use). My callback function could then destroy some objects, and return True to indicate that some objects were destroyed. At that point, the intepreter could run its standard garbage collection routines to release the memory that corresponded to those objects - before proceeding with whatever it was trying to do originally. (If I returned False, or if I didn't provide a callback function at all, the interpreter would simply behave as it does today.) Any memory allocations that happen while the callback function itself is executing, would not trigger further calls to it. The whole mechanism would be disabled for the rest of the session if the memory freed by the callback function was insufficient to prevent going over the memory limit.
Would this be worth considering for a future language extension? How hard would it be to implement?
You can't call a callback function right from memory allocation function. A lot of code in the core, in the standard and third-party extensions rely on the fact that no Python code executed on some C API functions. Violation of this rule will lead to a breakdown of all. Even copying the list could be broken. You allocate the memory of the necessary size for the new list and then copy the elements. If the callback function was called during memory allocation, the size of the original list may change. This will lead to a violation of the integrity and the crash or the wrong result. And there are thousands of such places. Change all of them is impossible, and it will lead to reduction of performance even if callbacks are not used. You can call a callback function only at safe point, at least when GIL is released.
participants (7)
-
Ben Hoyt -
Gregory P. Smith -
Jim Jewett -
Max Moroz -
Random832 -
Serhiy Storchaka -
Stephen J. Turnbull