[Python-Dev] Accepting PEP 560 -- Core support for typing module and generic types

[Antoine Pitrou]

On Thu, 14 Dec 2017 16:03:48 -0800
Guido van Rossum <guido at python.org> wrote:
> A slot is pretty expensive, as *every* class in existence will be another 8
> bytes larger (and possibly more due to malloc rounding).

I'm always surprised by the discussions about class object size.
Even imagining you have 10000 classes in memory (a pretty large number,
though I'm sure you can reach that number with a lot of dependencies),
we're talking about a total 800 kB memory growth (let's recall that
each of those classes will probably have code objects, docstrings and what
not attached to it -- i.e. you don't often create empty classes).

Is it really an important concern?

Regards

Antoine.


PS: simple experiment at an IPython prompt, trying to load every large
third-party package I have lying around (I may be forgetting some).

>>> def count_classes():
...:    types = [object]
...:    seen = set(types)
...:    while types:
...:        types = [c for c in itertools.chain.from_iterable(type.__subclasses__(c) for c in types)
...:                 if c not in seen]
...:        seen.update(types)
...:    return len(seen)
...:
>>> import numpy, asyncio, cython, requests, pandas, curio
>>> import django.apps, django.contrib, django.db, django.forms, django.http, django.middleware, django.views
>>> import twisted.internet.reactor, twisted.web
>>> import tornado.ioloop, tornado.gen, tornado.locks
>>> len(sys.modules)
1668
>>> count_classes()
6130

At this point, the IPython process uses 113 MB RSS, which adding a 8-byte
slot to each of those 6130 classes would increase by a mere 49 kB.  And
I'm not even doing anything useful (no user data) with all those modules,
so an actual application using those modules would weigh much more.

(and for the curious, the actual list of classes:
 https://gist.github.com/pitrou/8bd03dbb480f5acbc3abbe6782df5ebd)