[issue18576] Rename and document test.script_helper as test.support.script_helper
New submission from Nick Coghlan: tests.script_helper provides various utilities for sensibly testing scripts in a subprocess. It isn't easy for CPython developers to discover, since it is isn't documented and the file is mixed in with actual tests in the main test directory. As discussed in #15494, it should be: 1. Moved from Lib/test/ to Lib/test/support (and imports in tests adjusted accordingly) 2. Documented in Doc/library/test.rst These changes should be made on both the 3.3 and default branches to avoid spurious merge conflicts. ---------- assignee: docs@python components: Documentation, Tests messages: 193819 nosy: docs@python, ncoghlan priority: normal severity: normal status: open title: Rename and document test.script_helper as test.support.script_helper type: enhancement versions: Python 3.3, Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Changes by Ezio Melotti <ezio.melotti@gmail.com>: ---------- keywords: +easy nosy: +ezio.melotti stage: -> needs patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Seydou Dia added the comment: Since I am already on bugs.python.org/issue18578 I will tackle this issue, if you don't mind. ---------- nosy: +seydou _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Fotis Koutoulakis added the comment: I have finished the changes needed to move script_helper from Lib/test to Lib/test/support, and I have also documented it. Tests run gracefully (but be sure to check out if it works as intended on your machines too) Two notes though: 1. I have only made the changes on the default branch. Will do 3.3 tomorrow. 2. When it came to the documentation, I had to break the 80 characters limit, as going by it was resulting in weird rendering or wrong information depicted from sphinx (for example, when showing that script_helper functions belonged to test.support instead of the correct location, test.support.script_helper). Last but not least, this is one of my first contributions to Python, so I would appreciate your input. ---------- keywords: +patch nosy: +Fotis.Koutoulakis Added file: http://bugs.python.org/file33049/script_helper-default.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Nick Coghlan added the comment: I wouldn't worry about 3.3 at this point - the 3.3 test suite isn't going to see major changes for its final release, so the risk of merge conflicts is low. ---------- versions: -Python 3.3 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Changes by Nick Coghlan <ncoghlan@gmail.com>: ---------- assignee: docs@python -> ncoghlan _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Fotis Koutoulakis added the comment: Hello again. Is everything ok with the patch? Is there something not working as expected? Perhaps an omission or something? Do I need to do something more to get it accepted? Thanks for your time. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Serhiy Storchaka added the comment: Please left test.script_helper as alias to test.support.script_helper. I.e. test/script_helper.py should contains something like: from test.support.script_helper import * And test this with unmodifiable other tests. ---------- nosy: +serhiy.storchaka _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Fotis Koutoulakis added the comment: Ok, here is the second (modified patch) which contains a script so that no modifications are required to existing tests. I am uploading it as a second patch, so that the first one is left as a reference. As a sidenote, I fail to see convincing reasons for why we would want the second solution more than the first. The first one looks cleaner to my eyes. The only downside I can see to the original approach is breaking a couple of tests, which is something that can be fixed easily without even making it to the repo (running the tests in 2-3 could help find all possible broken tests beforehand). Could someone explain to me why the second solution is more desirable? ---------- Added file: http://bugs.python.org/file33055/issue_18576_second.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Serhiy Storchaka added the comment: Third party code can use tests.script_helper. I prefer to do these changes in several steps: 1) Move script_helper.py to Lib/test/support/, document it, and create an alias (with deprecation warning). 2) Ensure that this doesn't break any buildbot. 3) Change tests to use test.support.script_helper instead of test.script_helper. 4) One or two versions later remove an alias. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Fotis Koutoulakis added the comment: Oh, I see. Does the last patch meet your expectations, or would you rather see all these changes implemented at once? ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Serhiy Storchaka added the comment:
Does the last patch meet your expectations, or would you rather see all these changes implemented at once?
I left this for Nick. I have added comments on Rietveld. ---------- stage: needs patch -> patch review _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Fotis Koutoulakis added the comment: Taking the feedback during code review, this is a patch that has the points raised by Serhiy Storchaka fixed. As always, please do note omission or mistakes from my part. Thanks for your help. ---------- Added file: http://bugs.python.org/file33057/issue18576_third_try.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Nick Coghlan added the comment: Note that the test.* namespace has long been declared as unstable, so we don't worry about breaking third party code when rearranging the test suite (as far as I am aware, most Linux distros don't even include the test suite in their Python packages). I'll see how Mercurial goes tracking the file move when the alias is left behind, and decide which way to do the initial change based on whether or not the alias breaks the history tracking. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Nick Coghlan added the comment: The attached patch just moves the file without leaving an alias behind. It turns out we use this one in a fair few places (mostly for "assert_python_ok"). I'm going to sit on this change for the moment - after reading the docs, it's quite clear that the last couple of functions don't really belong in script_helper, but rather in the pkg_helper module discussed in issue 15376. So my current plan is to factor that helper out *first*, then when this one lands, the intent of grouping more of the support code that folks working on the test suite might be interested in using together will hopefully be clearer. ---------- dependencies: +Refactor the test_runpy walk_package support code into a common location Added file: http://bugs.python.org/file33188/issue18576_move_and_document_script_helper.... _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Changes by Eric Snow <ericsnowcurrently@gmail.com>: ---------- nosy: +eric.snow _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Nick Coghlan added the comment: script_helper was moved to the test.support package in issue 9517, so this issue is just about adding the docs now. ---------- dependencies: +Make test.script_helper more comprehensive, and use it in the test suite -Refactor the test_runpy walk_package support code into a common location title: Rename and document test.script_helper as test.support.script_helper -> Document test.support.script_helper versions: +Python 3.5 -Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Nick Coghlan added the comment: Also noting that my draft docs are over a year old now, so they need to be reviewed against the current state of the helper module. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Christie added the comment: I'm going to take a look at bringing these docs up to date and including the changes from issue24033. ---------- nosy: +bobcatfish _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Christie added the comment: @ncoghlan I've taken the test.rst changes from the previous diff and reviewed them against the latest script_helper (including updates from issue24033). Nothing had fallen behind, but I've added what was missing. * I've realized that spawn_python and the helper run_python I added in issue24033 are basically the same so I will reconcile those * Is there some way to load the docstrings from script_helper.py in test.rst instead of writing explanations for each method twice? ---------- Added file: http://bugs.python.org/file39645/issue18576.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Martin Panter added the comment: If you diffed your patch from a public revision in the main repository, there should be a nice “Review” link. Anyway, here are some comments on issue18576.patch: +.. function:: assert_python_ok(*args, **env_vars) + + Runs the interpreter with *args* and optional environment + variables, asserting that *env_vars* succeeds (``rc == 0``). Perhaps you mean *env_vars* are the optional environment variables, and that the interpreter is asserted to have succeeded. Also maybe clarify where the *__cleanenv* and *__isolated* keywords come from? Function signatures? Also applies to assert_python_failure(), run_python/_until_end(). Maybe it is worth mentioning that assert_python_ok/failure() strip whitespace from the stderr stream. I seem to remember being tricked by this (writing a test to ensure stderr ended in a newline or something). + ``stdout`` are configured as binary pipes and ``stderr`` is merged with + ``stdout``.. There are two full stops above here.. I encourage you to start sentences with capital letters (although I admit sometimes other people do not agree with this). Suggestions: + *kw* is passed through to subprocess.Popen as additional keyword + arguments. => The *kw* arguments are passed through to subprocess.Popen. + *source* is a Unicode string which will be written to the file as UTF-8. => The *source* argument is a Unicode string which will be written to the file as UTF-8. + *depth* controls how many deeply nested the package is. => The *depth* argument controls how deeply nested the package is. + Creates a new zip archive in the given directory, containing the specified + Python script/module Needs one of those full stops I mentioned earlier :) ---------- nosy: +vadmium _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Nick Coghlan added the comment: As far as using docstrings for the online docs goes, there are some useful Sphinx add-ons that we've never enabled for the main Python docs (like autodoc, blockdiag, seqdiag) that could be good options to have available. For autodoc in particular, we prefer to avoid it for the main docs because "good docstring" and "good prose documentation" don't necessarily look alike. For ease of maintenance of test.support though, I think the "write a good docstring, get the online docs for free" rationale is a compelling one (since these docs are core contributor facing, rather than being aimed at Python users in general) docs@python.org would be the place to ask about the appropriateness of enabling some Sphinx add-ons. While enabling them seems like a reasonable idea to me, I don't personally know if there are actually technical issues with the idea, or if there just hasn't been a volunteer to add/enable them and update https://docs.python.org/devguide/documenting.html accordingly. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Christie added the comment:
If you diffed your patch from a public revision in the main repository, there should be a nice “Review” link.
Ah, thanks @vadmium!!! I've been confused about why some patches get review links and others do not >.< I'll have a new patch up shortly which is diffed against a public revision and has your suggested changes.
For autodoc in particular, we prefer to avoid it for the main docs because "good docstring" and "good prose documentation" don't necessarily look alike.
@ncoghlan for some reason I thought that the main docs were also auto generated from docstrings in the actual modules, but I can see now that that isn't the case. I'll reach out to docs@python.org to see how they feel about autogenerating the test.support docs. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Changes by Nick Coghlan <ncoghlan@gmail.com>: ---------- assignee: ncoghlan -> _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Christie added the comment: Hey @vadmium! I've finally responded to your review feedback. I've attached a new patch, hopefully this time actually review-able. I've removed the docs for methods I added in @24033 so that this could be merged without it. Let me know if you have any more feedback, or if this could be merged. Thanks! ---------- Added file: http://bugs.python.org/file39827/issue18576.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Martin Panter added the comment: I left some more comments on Reitveld. The main one is I think stderr gets stripped with assert_python_ok(), but I don’t think stdout is touched. Correct me if I’m wrong though. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Changes by Louie Lu <me@louie.lu>: ---------- pull_requests: +1367 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Louie Lu added the comment: The PR on GitHub is based on bobcatfist's patch, addressed on Martin request and some minor change. ---------- nosy: +louielu _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18576> _______________________________________
Julien Palard <julien+python@palard.fr> added the comment: All those functions has already been documented. ---------- dependencies: -Make test.script_helper more comprehensive, and use it in the test suite nosy: +mdk resolution: -> out of date stage: patch review -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue18576> _______________________________________
participants (10)
-
Christie -
Eric Snow
-
Ezio Melotti
-
Fotis Koutoulakis
-
Julien Palard
-
Louie Lu
-
Martin Panter
-
Nick Coghlan
-
Serhiy Storchaka
-
Seydou Dia