On Wed, Oct 18, 2017 at 01:39:28PM +0300, Koos Zevenhoven wrote:
I'm writing from my phone now, cause I was dumb enough to try list(count())
You have my sympathies -- I once, due to typo, accidentally ran something like range(10**100) in Python 2.
But should it be fixed in list or in count?
Neither. There are too many other places this can break for it to be effective to try to fix each one in place. e.g. set(xrange(2**64)), or tuple(itertools.repeat([1])) Rather, I think we should set a memory limit that applies to the whole process. Once you try to allocate more memory, you get an MemoryError exception rather than have the OS thrash forever trying to allocate a terabyte on a 4GB machine. (I don't actually understand why the OS can't fix this.) Being able to limit memory use is a fairly common request, e.g. on Stackoverflow: https://stackoverflow.com/questions/30269238/limit-memory-usage https://stackoverflow.com/questions/2308091/how-to-limit-the-heap-size https://community.webfaction.com/questions/15367/setting-max-memory-for-pyth... And Java apparently has a commandline switch to manage memory: https://stackoverflow.com/questions/22887400/is-there-an-equivalent-to-java-... The problems with the resources module are that its effectively an interface to ulimit, which makes it confusing and platform-specific; it is no help to Windows users; it isn't used by default; and not many people know about it. (I know I didn't until tonight.) So here is my suggestion: 1. Let's add a function in sys to set the "maximum memory" available, for some definition of memory that makes the most sense on your platform. Ordinary Python programmers shouldn't have to try to decipher the ulimit interface. 2. Have a command line switch to set that value, e.g.: python3 -L 1073741824 # 1 GiB python3 -L 0 # machine-dependent limit python3 -L -1 # unlimited where the machine-dependent limit is set by the interpreter, depending on the amount of memory it thinks it has available. 3. For the moment, stick to defaulting to -L -1 "unlimited", but with the intention to change to -L 0 "let the interpreter decide" in some future release, after an appropriate transition period. On Linux, we can always run ulimit XXXX python3 but honestly I never know which option to give (maximum stack size? maximum virtual memory? why is there no setting for maximum real memory?) and that doesn't help Windows users. Thoughts? -- Steve