PEP 454 - tracemalloc - accepted
Hi, I'm happy to officially accept PEP 454 aka tracemalloc. The API has substantially improved over the past weeks, and is now both easy to use and suitable as a fundation for high-level tools for memory-profiling. Thanks to Victor for his work! Charles-François
2013/11/21 Charles-François Natali <cf.natali@gmail.com>:
I'm happy to officially accept PEP 454 aka tracemalloc. The API has substantially improved over the past weeks, and is now both easy to use and suitable as a fundation for high-level tools for memory-profiling.
Thanks to Victor for his work!
Thanks Charles-François for your help! Victor
On 22 Nov 2013 07:43, "Charles-François Natali" <cf.natali@gmail.com> wrote:
Hi,
I'm happy to officially accept PEP 454 aka tracemalloc. The API has substantially improved over the past weeks, and is now both easy to use and suitable as a fundation for high-level tools for memory-profiling.
Thanks to Victor for his work!
Huzzah! Thanks to you both for getting this ready for inclusion :) Cheers, Nick.
Charles-François _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
2013/11/21 Nick Coghlan <ncoghlan@gmail.com>:
Huzzah! Thanks to you both for getting this ready for inclusion :)
I now hope that someone will use it :-) By the way, collections.namedtuple has a private _source attribute. This attributes uses something like 676.2 kB in the Python test suite, it the 5th biggest user of memory. Woud you mind if I simply remove it? I'm asking because this *private* attribute is documented, which is surprising for something private. I don't understand the use case. Is it to debug the implementation of the namedtuple? Or I imagined a different usecase: dump the _source into a file to speedup the startup time. But this optimization is not used in CPython whereas we care of the startup time. Can maybe Raymond explain the use case for this attribute? If there is a real use case, I would prefer a real function to get the source code of a function defining a new namedtuple type. I already opened an issue for that: http://bugs.python.org/issue19640 -- The implementation of namedtuple constructor ("factory" ?) looks inefficient. Why do we need to build source code to then parse the code source? Why not having a base namedtuple class and then inherit from this class? It may reduce the memory footprint, allow to check if a type is namedtuple, etc. There are two different concerns, I prefer to only focus on the removal of the _source attribute. But it's also to say the the _source attribute is very specific to the current (inefficient?) implement of the namedtuple constructor. Victor
On 22 Nov 2013 09:02, "Victor Stinner" <victor.stinner@gmail.com> wrote:
2013/11/21 Nick Coghlan <ncoghlan@gmail.com>:
Huzzah! Thanks to you both for getting this ready for inclusion :)
I now hope that someone will use it :-)
By the way, collections.namedtuple has a private _source attribute. This attributes uses something like 676.2 kB in the Python test suite, it the 5th biggest user of memory.
Woud you mind if I simply remove it?
I'm asking because this *private* attribute is documented, which is surprising for something private. I don't understand the use case. Is it to debug the implementation of the namedtuple? Or I imagined a different usecase: dump the _source into a file to speedup the startup time. But this optimization is not used in CPython whereas we care of the startup time.
namedtuple deviates from normal naming conventions to avoid name clashes between field names and the attributes and methods of the object itself (this is covered in the docs for namedtuple). Skipping saving _source under -OO would probably be a good thing, but otherwise it's a public API with the usual backwards compatibility guarantees. Cheers, Nick.
Can maybe Raymond explain the use case for this attribute?
If there is a real use case, I would prefer a real function to get the source code of a function defining a new namedtuple type.
I already opened an issue for that: http://bugs.python.org/issue19640
--
The implementation of namedtuple constructor ("factory" ?) looks inefficient. Why do we need to build source code to then parse the code source? Why not having a base namedtuple class and then inherit from this class? It may reduce the memory footprint, allow to check if a type is namedtuple, etc.
There are two different concerns, I prefer to only focus on the removal of the _source attribute. But it's also to say the the _source attribute is very specific to the current (inefficient?) implement of the namedtuple constructor.
Victor
On Fri, 22 Nov 2013 09:17:14 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
Skipping saving _source under -OO would probably be a good thing, but otherwise it's a public API with the usual backwards compatibility guarantees.
I think skipping saving _source under -OO should be a bugfix. It's terribly weird and suboptimal to remove docstrings but keep the source code of namedtuple instances in some obscure attributes. Regards Antoine.
On Thu, Nov 21, 2013 at 4:17 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Skipping saving _source under -OO would probably be a good thing, but otherwise it's a public API with the usual backwards compatibility guarantees.
One alternative might be to make it a property that re-generates the source (you just need self.__class__.__name__ and self._fields). -eric
Hi, 2013/11/21 Charles-François Natali <cf.natali@gmail.com>:
I'm happy to officially accept PEP 454 aka tracemalloc. The API has substantially improved over the past weeks, and is now both easy to use and suitable as a fundation for high-level tools for memory-profiling.
I pushed the implementation of he PEP 454 (tracemalloc): http://hg.python.org/cpython/rev/6e2089dbc5ad Just after that, I pushed a change in the API to fix a bug: http://hg.python.org/cpython/rev/66db0c66a6ee "Issue #18874: Remove tracemalloc.set_traceback_limit()" Changing the traceback limit while tracemalloc is tracing can leads to bugs or weird behaviour. So I added an option nframe parameter to start instead. I'm sorry, there are pending comments in the issue. I got many reviews the last days, I didn't have time to address them yet. I prefer to push the code before the beta1 because this version marks the feature freeze and I expect more feedback if the module is included in the release. I consider that the comments doesn't change the public API (except the removal of set_tracemalloc_limit which is already done) and so can be addressed later. http://bugs.python.org/issue18874 Anyway, thanks to everyone who helped me to build a better tracemalloc module! And don't hesitate to continue to review the code and test the module. I will try to write an article for the Python Insider blog to show some examples of usage of the tracemalloc module. It would be nice to have similar articles for the other PEPs implemented in Python 3.4, before the release of Python 3.4! http://blog.python.org/ Victor
participants (6)
-
Antoine Pitrou -
Charles-François Natali -
Eric Snow -
Nick Coghlan -
Stefan Krah -
Victor Stinner