On Fri, Jan 20, 2017 at 8:52 PM, Ivan Levkivskyi
On 20 January 2017 at 11:49, INADA Naoki
wrote: * typing may increase memory footprint, through functions __attributes__ and abc. * Can we add option to remove or lazy evaluate __attributes__ ?
This idea already appeared few times. I proposed to introduce a flag (e.g. -OOO) to ignore function and variable annotations in compile.c It was decide to postpone this, but maybe we can get back to this idea.
In 3.6, typing is already (quite heavily) optimized for both speed and space. I remember doing an experiment comparing a memory footprint with and without annotations, the difference was few percent. Do you have such comparison (with and without annotations) for your app? It would be nice to have a realistic number to estimate what would the additional optimization flag save.
-- Ivan
Hi, Ivan. I investigated why our app has so many WeakSet today. We have dozen or hundreds of annotations like Iterable[User] or List[User]. (User is one example of application's domain object. There are hundreds of classes). On the other hand, SQLAlchemy calls isinstance(obj, collections.Iterable) many times, in [sqlalchemy.util._collections.to_list](https://github.com/zzzeek/sqlalchemy/blob/master/lib/sqlalchemy/util/_collec...) method. So there are (# of iterable subclasses) weaksets for negative cache, and each weakset contains (# of column types) entries. That's why WeakSet ate much RAM. It may be slowdown application startup too, because thousands of __subclasscheck_ is called. I gave advice to use 'List[User]' instead of List[User] to the team of the project, if the team think RAM usage or boot speed is important. FWIW, stacktrace is like this: File "/Users/inada-n/local/py37dbg/lib/python3.7/_weakrefset.py", line 84 self.data.add(ref(item, self._remove)) File "/Users/inada-n/local/py37dbg/lib/python3.7/abc.py", line 233 cls._abc_negative_cache.add(subclass) File "/Users/inada-n/local/py37dbg/lib/python3.7/abc.py", line 226 if issubclass(subclass, scls): File "/Users/inada-n/local/py37dbg/lib/python3.7/abc.py", line 226 if issubclass(subclass, scls): File "/Users/inada-n/local/py37dbg/lib/python3.7/abc.py", line 191 return cls.__subclasscheck__(subclass) File "venv/lib/python3.7/site-packages/sqlalchemy/util/_collections.py", line 803 or not isinstance(x, collections.Iterable): File "venv/lib/python3.7/site-packages/sqlalchemy/orm/mapper.py", line 1680 columns = util.to_list(prop) File "venv/lib/python3.7/site-packages/sqlalchemy/orm/mapper.py", line 1575 prop = self._property_from_column(key, prop) File "venv/lib/python3.7/site-packages/sqlalchemy/orm/mapper.py", line 1371 setparent=True) File "venv/lib/python3.7/site-packages/sqlalchemy/orm/mapper.py", line 675 self._configure_properties()<PasteEnd> Regards,