roccomoretti@netscape.net (Rocco Moretti) writes:
Armin Rigo <arigo@tunes.org> wrote:
On Sat, Jun 07, 2003 at 08:51:51AM +0200, holger krekel wrote:
- how the new test-machinery works so i can make sure that i don't break big stuff while trying to fix/play around
pypy/testall.py runs all the tests, using the object space specified in the environment variable OBJSPACE -- which must be exactly pypy.objspace.std.StdObjSpace to use the standard object space; the >trivial object space is used otherwise.
pypy/testwice.py is a hack around the previous one to run all the tests in both object spaces.
("previous" being the pypy/testall.py file, not any pre-sprint test hook.)
... and pypy/testcts.py is a rather clever piece of code to run tests on StdObjSpace, keep track of the results, and report any difference from run to run.
'clever' is a strange way of spelling 'unfinished' :-) Ideally, this would run after every checkin (or so) and hassle the checker-in (or this list) if any tests broke. It would be nice to get this in place before or early on in the next sprint.
The individual .../test/test_xxx.py files are still runnable. There is a testsupport.py file in all test directories for glueing purposes; running it directly should execute all the tests in that directory.
I've been playing around with it over the weekend, and I feel things (testing wise) need to be clarified slightly. First, I think we should move all pypy-as-a-whole testing related stuff into an approriate subdirectory ("pypy/testing" or some such. As it is now, pieces are scattered in pypy and pypy/interpreter.
Probably, yes.
Secondly, the testing framework as written will only run tests at the interpreter level. Some tests (such as test_exceptcomp and test_exec) should be run at the application level.
Yes. I don't see how to easily and gracefully allow this.
Additionally, there is no provision for running the CPython regression tests automatically within the current testing framework.
The thought occured to me if it would not be easiest to "appropriate" the CPython regrtest.py framework for our purposes. As best I can tell, it's written in a general fashion already, so "all" we would need to do is get it to recognize when to run tests under interpreter level and when to run tests under application level. (Which could potentially be accomplished by a naming convention.)
Hmm, maybe. regrtest has become somewhat crufty over the years, I don't think we need all of it. In a way, I'd prefer to keep things within the unittest framework if possible (although working closely with it for the first time has made its Java heritage unpleasantly clear...). We also need a LOT more tests!
- any "entry points" other than interactive.py?
There is main.py in the same directory. Its purpose is that 'python >main.py script-and-options' should be the same as 'pypy script-and-options' if we >had a working 'pypy' program. I guess that main.py should invoke >interactive.py when started with no argument (it doesn't right now).
Other problems, I assume (from code inspection, not from direct running) are that the commandline arguments are not passed to sys.argv, sys.path may or may not contain the directory of the script, and sys.modules['__main__'] is not set equal to the script module (only a problem, I think, when running unittest.main()).
Yes. It's very much unfinished -- and actually it would make a good project for someone who doesn't get the ins and outs of the standard object space to move this further to completion...
And in keeping with my rearranging mood, I think main.py should probably be placed in the pypy/ directory.
Hmm. Maybe, or maybe the executable that people actually run will look like this: #!/usr/bin/python from pypy.interpreter import main main.main() [...]
Congrats to the spriters - it looks like quite a few good things have been accomplished.
Thanks, I think we really moved things forward. Are you able to come to the next sprint? Cheers, M. -- 48. The best book on programming for the layman is "Alice in Wonderland"; but that's because it's the best book on anything for the layman. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html