Re: [pypy-svn] r5369 - in pypy/trunk/src/pypy: scripttest scripttest/test tool
Hi Rocco, [rocco@codespeak.net Sun, Jun 27, 2004 at 10:53:28PM +0200]
I guess this at least should not live directly in src/pypy but in some subdirectory (e.g. tool) or at 'src/scripttest' or something similar. And btw, it's preferable if you talk to some other developer or best to pypy-dev before adding a whole bunch of things :-)
Can you give some examples of usage? I am a bit of a loss what exactly you are trying to do here. cheers, holger
holger krekel wrote:
Sorry if it's unclear... It started out as a way to run a script at applevel, capture its output, and compare it with a known good result (presumably generated from CPython). I was inspired by the original parts of the CPython regression test framework. The thought was to make a simple way of testing app level functionality, without having to support a whole unittest framework at applevel. app_level_diff.py is the tool that does the running and comparing. The scripts in scripttest/test are unittests that fit in with the general testing framework to run the (applevel) scripts in the scripttest directory proper and compare the results to the pregenerated output files found there. So the usage examples are in the scripttest/test directory. - It's another way of adding tests to pypy, and checking that pypy works like CPython does.
I guess this at least should not live directly in src/pypy but in some subdirectory (e.g. tool) or at 'src/scripttest' or something similar.
Hmmm ... you're probably right that it's a poor place for it to live. But I'm not sure it should be in any particular subdirectory - the tests exercise the whole of pypy, not just a particular part. As for putting them in scr/scripttest or so, I was hoping that this would be able to be a part of the regression testing framework, and would be hit by the test_all.py script. But they are self contained, and can be moved and/or deleted without any consequence, if people so desire. -Rocco P.S. In retrospect, I probably rushed checking in the code. Upon reexamination, it is rather rough, and not as extensible as I would like. If I have time this weekend, I'll see if I can improve it. (This includes adding better documentation.)
[Rocco Moretti Fri, Jul 02, 2004 at 11:29:29AM -0500]
yes, i think this technique makes sense. But did you look into doctests, btw?
well, tool/testit.py is also used by all tests of pypy so i guess something like tool/testexpect or tool/scripttest makes sense.
Yes, i additionally think it's a good idea to discuss with other developers before adding toplevel directories ... cheers, Holger
holger krekel wrote:
The thought was that running doctests/unittests at applevel involves a large amount of code that might not run on pypy. If most of the testing apparatus is at interpreter level, there is less chance the test framework is what is failing, as opposed to the test itself. I guess this is moot now that Seo has shown that the CPython regrtest framework is working now. Add to that the fact that I can't seem to find a good way to create an extensible way of adding tests (currently adding new test scripts requires a substantial amount of cut-and-paste), and its probably best just to delete the whole deal, and get rid of the concomitant namespace pollution. -- If someone thinks it'll be worthwhile later, they can either resurrect it from svn, or recreate it from scratch. Sorry about the dead-end-wild-goose-chase -Rocco
holger krekel wrote:
Sorry if it's unclear... It started out as a way to run a script at applevel, capture its output, and compare it with a known good result (presumably generated from CPython). I was inspired by the original parts of the CPython regression test framework. The thought was to make a simple way of testing app level functionality, without having to support a whole unittest framework at applevel. app_level_diff.py is the tool that does the running and comparing. The scripts in scripttest/test are unittests that fit in with the general testing framework to run the (applevel) scripts in the scripttest directory proper and compare the results to the pregenerated output files found there. So the usage examples are in the scripttest/test directory. - It's another way of adding tests to pypy, and checking that pypy works like CPython does.
I guess this at least should not live directly in src/pypy but in some subdirectory (e.g. tool) or at 'src/scripttest' or something similar.
Hmmm ... you're probably right that it's a poor place for it to live. But I'm not sure it should be in any particular subdirectory - the tests exercise the whole of pypy, not just a particular part. As for putting them in scr/scripttest or so, I was hoping that this would be able to be a part of the regression testing framework, and would be hit by the test_all.py script. But they are self contained, and can be moved and/or deleted without any consequence, if people so desire. -Rocco P.S. In retrospect, I probably rushed checking in the code. Upon reexamination, it is rather rough, and not as extensible as I would like. If I have time this weekend, I'll see if I can improve it. (This includes adding better documentation.)
[Rocco Moretti Fri, Jul 02, 2004 at 11:29:29AM -0500]
yes, i think this technique makes sense. But did you look into doctests, btw?
well, tool/testit.py is also used by all tests of pypy so i guess something like tool/testexpect or tool/scripttest makes sense.
Yes, i additionally think it's a good idea to discuss with other developers before adding toplevel directories ... cheers, Holger
holger krekel wrote:
The thought was that running doctests/unittests at applevel involves a large amount of code that might not run on pypy. If most of the testing apparatus is at interpreter level, there is less chance the test framework is what is failing, as opposed to the test itself. I guess this is moot now that Seo has shown that the CPython regrtest framework is working now. Add to that the fact that I can't seem to find a good way to create an extensible way of adding tests (currently adding new test scripts requires a substantial amount of cut-and-paste), and its probably best just to delete the whole deal, and get rid of the concomitant namespace pollution. -- If someone thinks it'll be worthwhile later, they can either resurrect it from svn, or recreate it from scratch. Sorry about the dead-end-wild-goose-chase -Rocco
participants (2)
-
holger krekel -
Rocco Moretti