[pypy-svn] r58040 - pypy/build/doc

[hpk at codespeak.net]

Author: hpk
Date: Wed Sep 10 17:42:50 2008
New Revision: 58040

Added:
   pypy/build/doc/benchmark_memory.txt   (contents, props changed)
Log:
draft doc about benchmarking memory usage



Added: pypy/build/doc/benchmark_memory.txt
==============================================================================
--- (empty file)
+++ pypy/build/doc/benchmark_memory.txt	Wed Sep 10 17:42:50 2008
@@ -0,0 +1,31 @@
+Considerations regarding Memory Benchmarks
+===============================================
+
+* measure min/max/avg process-private RAM usage of a Benchmark 
+
+* measure (min/max/avg) collection time of an APP/Benchmark 
+
+* measure CPU usage of benchmark
+
+* Microbenchmarks probably with checkpoints in order to have 
+  reproducible RAM usage measurements and to analyse
+  specific object types (strings/dicts/lists/old-style/new-style 
+  instances ...) and scenarios like reading lines in large file. 
+  (reuse some of the speed benchmarks?)
+
+* Scenario/App Benchmarks: 
+  * allocate objects all the time but have mostly constant 
+    number of life objects 
+  * have small number of objects objects, create many, 
+    free them all, repeat 
+   
+  * use several sets of "small" and "many"
+  
+* real APPS: 
+  * some sympy computation/test? 
+  * an app involving c-extensions? 
+  * ask around on pypy-dev for example apps 
+
+* measurements against CPython, pypy-c with various options 
+
+