[pypy-svn] commit/extradoc: 4 new changesets

Bitbucket commits-noreply at bitbucket.org
Thu Mar 3 16:58:54 CET 2011


4 new changesets in extradoc:

http://bitbucket.org/pypy/extradoc/changeset/02e265dc2a8b/
changeset:   r3340:02e265dc2a8b
branch:      extradoc
user:        lac
date:        2011-03-03 15:43:11
summary:     arigo?
This must be one of Armin's because I did not edit this file.
But if I have instead gone and clobbered his changes with an
earlier version a) I apologise and b) I have no clue how this
could happen given that he checked in things later than I.
affected #:  1 file (7 bytes)

--- a/talk/stanford-ee380-2011/talk.html	Wed Mar 02 10:35:16 2011 -0800
+++ b/talk/stanford-ee380-2011/talk.html	Thu Mar 03 15:43:11 2011 +0100
@@ -364,8 +364,8 @@
 print Foo("hello").double().value
 </pre></div>
-<div class="slide" id="in-two-words">
-<h1>In two words</h1>
+<div class="slide" id="in-two-points">
+<h1>In two points</h1><ul class="simple"><li>Strongly, trivially, dynamically typed language</li><li>Ints, floats, longs, string, unicode,
@@ -461,6 +461,16 @@
 <li>Now contains about 200 KLoC, and 150 KLoc of tests</li></ul></div>
+<div class="slide" id="a-bit-of-history">
+<h1>A bit of history</h1>
+<ul class="simple">
+<li>Squeak and Scheme48 are also interpreters written in themselves</li>
+<li>Or more precisely, like PyPy, a subset of themselves</li>
+<li>But in PyPy, the subset is at a higher level</li>
+<li>General rule: <em>every aspect that is independent from the high-level
+description of the interpreter is left out of it</em></li>
+</ul>
+</div><div class="slide" id="what-is-the-point-of-pypy"><h1>What is the point of PyPy?</h1><ul class="simple">
@@ -524,16 +534,6 @@
 <li>See demo</li></ul></div>
-<div class="slide" id="a-bit-of-history">
-<h1>A bit of history</h1>
-<ul class="simple">
-<li>Squeak and Scheme48 are also interpreters written in themselves</li>
-<li>Or more precisely, like PyPy, a subset of themselves</li>
-<li>But in PyPy, the RPython subset is at a higher level</li>
-<li>General rule: <em>every aspect that is independent from the high-level
-description of the interpreter is left out of RPython</em></li>
-</ul>
-</div><div class="slide" id="rpython-is-still-mostly-python"><h1>RPython is still mostly Python</h1><ul class="simple">
@@ -798,7 +798,7 @@
 <ul class="simple"><li>The hard part: finding all pointers to GC objects from local variables
 in the C stack</li>
-<li>ANSI C solution: all pointers are copied to and fro some custom stack</li>
+<li>ANSI C solution: all pointers are copied to and from some custom stack</li><li>Not-ANSI-C-at-all: parse the assembler produced by GCC to build tables</li></ul></div>
@@ -816,9 +816,9 @@
 <div class="slide" id="what-is-a-jit"><h1>What is a JIT</h1><ul class="simple">
-<li>A JIT selects pieces of the user program (say Java) that would benefit
+<li>A JIT selects pieces of the user program (in language P) that would benefit
 from compilation instead of interpretation</li>
-<li>A &quot;method JIT&quot; selects individual Java functions and compiles them,
+<li>A &quot;method JIT&quot; selects individual P functions and compiles them,
 possibly doing some inlining to improve performance (HotSpot, Psyco)</li><li>A &quot;tracing JIT&quot; selects individual code paths from loops and compiles
 them, inlining aggressively (TraceMonkey, PyPy)</li>
@@ -897,7 +897,7 @@
 <li>Turns a trace into machine code</li><li>Simple register allocation (linear code)</li><li>x86, x86-64, (ARM)</li>
-<li>Guards compiled as conditional jumps to code that restore the full state</li>
+<li>Guards compiled as conditional jumps to code that restores the full state</li></ul></div><div class="slide" id="blackhole-interpreter">


http://bitbucket.org/pypy/extradoc/changeset/f25bd2ac1be9/
changeset:   r3341:f25bd2ac1be9
branch:      extradoc
user:        lac
date:        2011-03-03 15:45:32
summary:     intial revision.  notes about the google sf talk
affected #:  1 file (4.3 KB)

--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/talk/ustour2011/rawnotes/googlesf	Thu Mar 03 15:45:32 2011 +0100
@@ -0,0 +1,88 @@
+The building is wonderfully located with views of the Bay.  The cafeteria
+was most pleasant, and the food was good.  Google has 4 floors.  2 of them
+are full of engineers - app engine is here among others.  One of them is
+full of Google staff that aren't engineers.  I don't know what they do.
+The other floor is full of startups, which have some sort of special
+relationship with Google.  I never found out how that works, either.
+
+Several engineers told me that they were really pleased that we chose
+to talk to them in SF as well as at the Googleplex.  It seems that
+most people do not do this, which means they either have to take a
+very long train ride to Mt. View to hear neat stuff, or do without.
+Just by coming to the SF site made us a lot of fans, it seems.
+
+Between 25 and 30 engineers showed up.  Several (many?) of them were
+unknown to either Guido or Wesley Chun, and had to introduce themselves.
+Several of them were primarily C++ users.  They want seamless integration
+with C++ 'natch.  'A way to just import c++ .h files into python' was
+mentioned.  Some of the people there work primarily on the search engine.
+They like prototyping new ideas (in python, I would guess).  They spoke of
+other people who were not there -- who may or may not work at Google --
+who are designing new experimental languages.  Up until this talk they
+did not know that PyPy would be good for implementing them.
+
+Some engineers expressed a desire to come to noisebridge for the sprint.
+The mood was entirely upbeat.  There were no questions of the 'desiged
+to make you look like an idiot, designed to show how much I dislike you,
+your code, your approach sort'.  It sounded like some people were here out
+of sheer intellectual curiosity, but some had real life problems they were 
+interested in using pypy for.
+
+They seemed completely unaware that PyPy was not just python in python and
+could be used to implement any dynamic language.  They think this is bad
+marketing on our part and that we should call the toolchain something else
+and showcase it separately.  
+
+They have lots and lots of SWIG wrapped things at Google.
+
+Questions indicated that most people had at least some idea as to what
+was going on.
+
+"are they all boxed?"
+"how much memory"
+"the gc -- you cannot count on dels happening instantaneously, same as
+Jython, IronPython" -- does this apply to weakrefs also?
+why didn't you use the ref counting gc
+why is ref counting slow?
+people wanted more indepth on the jit how it works
+why are all the blue lines the same size in speed.pypy.org
+what is the difference between us and spidermonkey?
+do you dispatch? pass information between trqces? are they parameterised by 
+the incoming trace information as it comes in?
+is there still hotspot analysis to determine what to jit?
+memory bloat like psyco?
+what is the potential for multithreading?
+stackless
+tail recursion
+release infrastructure so people who want to do language experimentation
+can do it on top of pypy.  you can already? why didn't we know that?
+gameboy got a big laugh
+how can you tell if your cextension will run on pypy?  If you should rewrite
+it in RPython for performance?  Why are some c extensions faster than others
+when linked to pypy?
+SWIG.  can we support SWIG to ctypes?  How hard would it be?
+reflex(?) for importing c++
+include c++ header file
+cern sprint.
+
+So the last impression I got before we went to lunch is that there was one guy
+in particular, John whose name I forgot, who works in search engines and
+writes c++ who is really interested.  Guido and Wesley did not know him,
+and several others who attended.  "People who write C++ all the time in
+Google" may be a new target group for us for funding.  They seem to have
+concrete problems they want to fix.
+
+at lunch we discussed
+sandboxing
+google has some thing for sandboxing c. or 2 things.  open source.
+'nacl' discussing of why pypy was not a good fit there.
+
+funding
+google app engineers can get money out to a santa barbara group that has
+an open source app engine clone 'because they have the .edu' -- somebody
+else who has a different one and is just another OS developer doesn't
+get money.  G + W will think about whether HHU could get money from Google.
+Google apparantly loves to send money to postdocs, but is that only to
+US ones?  G + W will also work on a doc about what we could do to approach 
+other google managers for money.  We will meet in atlanta at pycon and
+discuss.


http://bitbucket.org/pypy/extradoc/changeset/625cd3d0c300/
changeset:   r3342:625cd3d0c300
branch:      extradoc
user:        lac
date:        2011-03-03 16:54:56
summary:     Guido sent me John Plevyak's last name and email.
affected #:  1 file (0 bytes)

--- a/talk/ustour2011/rawnotes/googlesf	Thu Mar 03 15:45:32 2011 +0100
+++ b/talk/ustour2011/rawnotes/googlesf	Thu Mar 03 16:54:56 2011 +0100
@@ -66,7 +66,7 @@
 cern sprint.
 
 So the last impression I got before we went to lunch is that there was one guy
-in particular, John whose name I forgot, who works in search engines and
+in particular, John Plevyak, login jplev who works in search engines and
 writes c++ who is really interested.  Guido and Wesley did not know him,
 and several others who attended.  "People who write C++ all the time in
 Google" may be a new target group for us for funding.  They seem to have


http://bitbucket.org/pypy/extradoc/changeset/73c218ea0fe2/
changeset:   r3343:73c218ea0fe2
branch:      extradoc
user:        lac
date:        2011-03-03 16:58:35
summary:     merge heads
and I hope these are Armins changes I am getting not me clobbering his.
affected #:  0 files (0 bytes)

--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/talk/pycon2011/whyslow/talk.rst	Thu Mar 03 16:58:35 2011 +0100
@@ -0,0 +1,187 @@
+=========================================
+Why is Python slow and how PyPy can help?
+=========================================
+
+What's this talk about?
+-----------------------
+
+* short introduction to JITting
+
+* how does a tracing JIT work
+
+* semantics that make Python slow/hard to optimize
+XXX cross slow
+
+Short introduction to JITting
+-----------------------------
+
+* run code with the interpreter
+
+* observe what it does
+
+* generate optimized machine code for commonly executed paths
+
+* using runtime knowledge (types, paths taken)
+
+Tracing JIT
+-----------
+
+* compiles one loop at a time
+
+* generates linear code paths, recording what the interpreter did
+
+* for each possible branch, generate a guard, that exits assembler on triggering
+
+* if guard fails often enough, start tracing from the failure
+
+Tracing example
+---------------
+
+* we have cool tools!
+
+XXX pic
+
+Part 2 - python semantics
+--------------------------
+
+* we're going to concentrate on a few really annoying things
+
+* frame introspection
+
+* dynamic dispatch
+
+* boxing
+
+* dynamic method lookup
+
+* attribute access
+
+Dynamic dispatch
+----------------
+
+* each operation has to dispatch on the type of the first argument
+
+* ``a + b`` can call integer addition, string concatenation or custom
+  ``__add__`` method
+
+* not much to talk about, tracing JIT deals with this without
+  extra effort
+
+* it can get fairly complex (XXX http://hg.python.org/cpython/file/6910af7df354/Objects/abstract.c#l761)
+
+* all of this logic is constant folded (XXX trace)
+
+Boxing
+------
+
+* for dynamic dispatch to work, each object has to be packed in a box
+
+* ``PyIntObject``, ``PyFloatObject`` etc.
+
+* it's wasteful, because it requires memory allocations (or pooling)
+
+* ideally we would not allocate them unless they escape
+
+* frames get in the way (they escape locals and valuestack)
+
+XXX more traces
+
+Frame introspection
+-------------------
+
+* ``sys._getframe()``, ``sys.exc_info()`` has to work
+
+* require creating python frames on the heap
+
+* not very convinient for fast access or escape analysis
+
+
+Frame introspection - solution
+------------------------------
+
+* so called "virtualizables"
+
+* frames are allocated and unpacked on processor (C) stack
+
+* in case of frame introspection happening, JIT knows where to find necessary values
+
+* reconstructed frame looks like it has always been there
+
+Attribute access
+----------------
+
+* ``obj.attr``
+
+* Look for ``__getattribute__`` on ``obj``
+
+* check ``type(obj).__dict__`` for a descriptor
+
+* check ``obj.__dict__`` for a value
+
+* 3 dict lookups
+
+Map dicts
+-------------
+
+* for a common case reduces 3 dict lookups to a list lookup
+
+* makes objects very compact (same as with ``__slots__``)
+
+* works even for adding attributes later on, after ``__init__``
+
+Map dicts - how it works
+-------------------------
+
+* stores a structure remembering common object shapes
+
+* a dictionary mapping names to numbers in a list
+
+* a list per object
+
+* those dictionary lookups are constant-folded away at the
+  time of JIT compilation
+
+XXX cool pics
+
+Dynamic method lookup
+---------------------
+
+* ``obj.meth()``, what happens?
+
+* 2 parts: attribute lookup, and method call
+
+* check ``__dict__`` of ``obj`` and ``type(obj)`` (and the entire MRO)
+
+* allocate a bound method
+
+* call the bound method
+
+Linking it all together
+-----------------------
+
+* array example
+
+Things we did not talk about
+----------------------------
+
+* regular expressions
+
+* generators
+
+* recursion
+
+* ``map`` and other looping constructs
+
+Future directions
+-----------------
+
+* fast ctypes
+
+* numpy
+
+Thank you
+-----------
+
+* http://pypy.org
+
+* http://morepypy.blogspot.com/


--- a/talk/stanford-ee380-2011/talk.html	Thu Mar 03 16:54:56 2011 +0100
+++ b/talk/stanford-ee380-2011/talk.html	Thu Mar 03 16:58:35 2011 +0100
@@ -523,6 +523,8 @@
 <h1>PyPy's Python interpreter</h1><ul class="simple"><li>A priori similar to CPython, but written in RPython.</li>
+<li>RPython is also valid Python: we test extensively by running
+it on top of CPython</li><li>See demo (py.py)</li></ul></div>
@@ -538,7 +540,8 @@
 <h1>RPython is still mostly Python</h1><ul class="simple"><li>Completely valid Python (can be tested directly)</li>
-<li>Can use lists, dicts, tuples, classes and instances, and so on</li>
+<li>Can use lists, dicts, tuples, classes and instances, and so on,
+but it must be type-safe</li><li>Contains no garbage collection detail (Py_INCREF/Py_DECREF in CPython)</li><li>Really a subset of Python: roughly &quot;how a Java programmer writes his
 first Python program&quot;</li>
@@ -940,6 +943,8 @@
 <li><a class="reference" href="http://pypy.org/">http://pypy.org/</a></li><li><a class="reference" href="http://speed.pypy.org/">http://speed.pypy.org/</a></li><li>irc: <tt class="docutils literal"><span class="pre">#pypy</span><span class="pre">at</span><span class="pre">freenode.net</span></tt></li>
+<li>noisebridge sprint this weekend (from 10am):
+<a class="reference" href="https://www.noisebridge.net/wiki/Getting_Here">https://www.noisebridge.net/wiki/Getting_Here</a></li></ul></div></div>


--- a/talk/stanford-ee380-2011/talk.txt	Thu Mar 03 16:54:56 2011 +0100
+++ b/talk/stanford-ee380-2011/talk.txt	Thu Mar 03 16:58:35 2011 +0100
@@ -237,6 +237,9 @@
 
 * A priori similar to CPython, but written in RPython.
 
+* RPython is also valid Python: we test extensively by running
+  it on top of CPython
+
 * See demo (py.py)
 
 
@@ -255,7 +258,8 @@
 
 * Completely valid Python (can be tested directly)
 
-* Can use lists, dicts, tuples, classes and instances, and so on
+* Can use lists, dicts, tuples, classes and instances, and so on,
+  but it must be type-safe
 
 * Contains no garbage collection detail (Py_INCREF/Py_DECREF in CPython)
 
@@ -762,3 +766,6 @@
 * http://speed.pypy.org/
 
 * irc: ``#pypy at freenode.net``
+
+* noisebridge sprint this weekend (from 10am):
+  https://www.noisebridge.net/wiki/Getting_Here


Binary file talk/stanford-ee380-2011/ui/py-web.png has changed


--- a/talk/stanford-ee380-2011/ui/py.css	Thu Mar 03 16:54:56 2011 +0100
+++ b/talk/stanford-ee380-2011/ui/py.css	Thu Mar 03 16:58:35 2011 +0100
@@ -20,7 +20,7 @@
   background-image: url("py-web.png");
   background-repeat: no-repeat;
   margin: 3px;
-  height: 120px;
+  height: 76px;
   border-bottom: 1px solid black;
 }

Repository URL: https://bitbucket.org/pypy/extradoc/

--

This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.



More information about the Pypy-commit mailing list