FWIW, I tend to run a few project(*) test suites when doing minor releases (2.x.y), just to make sure I don't break things. For major releases, it's a fair bit more work - something like Twisted or Zope3 play at such a low level with the Python interfaces that there's nearly always breakages or massive warning dumps (from memory, zope.interface uses 'with' as an argument name, a lot).
The "community buildbot" idea is a good one, although it should just be possible to write something for buildbot that checks out and builds the latest Python SVN, then installs it to a temporary location, then adds that location to the front of the path. Then each project could just add a "Python SVN buildbot" to their exists bbot install.
(*) Typically includes Zope3 and Twisted, because I have these just lying around.