On Fri, Dec 06, 2019 at 11:10:49AM +0000, Paul Moore wrote: [...]
They might end up with a million dynamically created classes, each with a single instance, when what they wanted was a single class with a million instances.
But isn't that the point here? A limit would catch this and prompt them to rewrite the code as
cls = namedtuple("Spam", "fe fi fo fum") for attributes in list_of_attributes: obj = cls(*attributes) values.append(obj)
Indeed, and maybe in the long term their code would be better for it, but in the meantime code that was working is now broken. It's a backwards-incompatable change. This PEP isn't about forcing people to write better code against their wishes :-) This leads me to conclude that: (1) Regardless of what we do for the other resources, "number of classes" may have to be excluded from the PEP. (2) Any such limit on classes needs to be bumped up. (3) Or we need a deprecation period before adding a limit: In release 3.x, the interpreter raises a *warning* when the number of classes reaches a million; in release 3.X+2 or X+5 or whenever, that gets changed to an error. It will need a long deprecation period for the reasons Thomas mentions: the person seeing the warnings might not be the developer who can do anything about it. We have to give people plenty of time to see the warnings and hassle the developers into fixing it. For classes, it might be better to the PEP to increase the desired limit from a million to, let's say, 2**32 (4 billion). Most people are going to run into other limits before they hit that: a bare class inheriting from type uses about 500 bytes on a 32-bit system, and twice that on a 64-bit system: py> sys.getsizeof(type('Klass', (), {})) 1056 so a billion classes uses about a terabyte. In comparison, Windows 10 Home only supports 128 GB memory in total, and while Windows Server 2016 supports up to 26 TB, we surely can agree that we aren't required to allow Python scripts to fill the entire RAM with nothing but classes :-) I think Mark may have been too optimistic to hope for a single limit that is suitable for all seven of his listed resources: * The number of source code lines in a module. * The number of bytecode instructions in a code object. * The sum of local variables and stack usage for a code object. * The number of distinct names in a code object. * The number of constants in a code object. * The number of classes in a running interpreter. * The number of live coroutines in a running interpreter. A million seems reasonable for lines of source code, if we're prepared to tell people using machine generated code to split their humongous .py files into multiple scripts. A small imposition on a small subset of Python users, for the benefit of all. I'm okay with that. Likewise, I guess a million is reasonable for the next four resources, but I'm not an expert and my guess is probably worthless :-) A million seems like it might be too low for the number of classes; perhaps 2**32 is acceptible. And others have suggested that a million is too low for coroutines.
Could there be people doing this deliberately? If so, it must be nice to have so much RAM that we can afford to waste it so prodigiously: a namedtuple with ten items uses 64 bytes, but the associated class uses 444 bytes, plus the sizes of the methods etc. But I suppose there could be a justification for such a design.
You're saying that someone might have a justification for deliberately creating a million classes,
Yes. I *personally* cannot think of an example that wouldn't (in my opinion) be better written another way, but I don't think I'm quite knowledgable enough to categorically state that ALL such uses are bogus. -- Steven