Xavier Morel added the comment: Second patch: documenting smtpd-as-a-script in the module's rst ---------- Added file: http://bugs.python.org/file20813/smtpd-as-script-doc.diff _______________________________________ Python tracker http://bugs.python.org/issue11260 _______________________________________

Xavier Morel added the comment:

Do tests currently exist for smtpd run as a script?

I have to confess I didn't think to check.

If not, our experience with converting compileall to argparse indicates a thorough test suite is needed (and even so we missed some things we hadn't thought to test).

OK, so if there is no test suite currently I should create one, and if there is one I should ensure it's complete? I guess I should use compileall as an example of how to test modules-as-scripts if the former? Overall, smtpd-as-a-script is really pretty simple, it totals 28 lines apart from the argument parsing itself (which is a bit under 60 lines ignoring the help text/pattern and gets a bit under 50 including it post-patch), and as I mentioned the only part which actually needed changing is the arguments parsing, which was very well factored out.

In other words, if the current code works, is "updating" it a sufficient reason to change it, considering the chances of introducing new bugs?

I'm not sure, but one of the ways I see standard libraries is not just as "ready to use" code, it's also as a repository of how to do things. And to that end, if argparse is now considered the canonical way to parse command-line arguments (which I would expect), there are very few examples of how to do things in the stdlib itself (though there are examples outside of it, due to the life argparse had pre-stdlib). It also rose to the occasion as I was wondering about the numerous standard library modules-as-scripts which are either undocumented or under-documented: because it had a good command-line documentation and a clean separation between the configuration (options parsing) and the actual execution, but no module documentation (in Doc/) it seemed like a good starting point: if it's not feasible to correctly convert "best cases" (and smtpd is probably one, short of modules using optparse probably) then the whole idea is stillborn: I do not see how it would be possible to fare better on some of the fully undocumented modules using manual options parsing, yet it would have to be expected. ---------- _______________________________________ Python tracker http://bugs.python.org/issue11260 _______________________________________

Xavier Morel added the comment:

any of the undocumented command-line interfaces are intentionally undocumented -- they were there for the convenience of the developer for exercising the module as it was being developed and are not part of the official API.

I can understand that, but it's not clear from the source code which is which, and for several of them third-parties have taken up the convenience. Maybe a more formal process of defining which tools are to be considered "official" and which are easter-eggs is needed? In any case, even for "easter eggs" surely a good command-line documentation is a good idea (even if the module documentation side is not added due to the "easter egg" status) isn't it?

The standard library's primary purpose is to be a library for use in user programs, not to serve as a suite of applications.

Sure, and I don't think most of the module-as-scripts have what it takes to be seen as applications, but many are useful and/or interesting helpers/shortcuts for day to day tasks. http.server and smtpd are good examples of useful modules-as-scripts (http.server being the exact opposite of smtpd in that it has a module-as-script documentation but no documentation whatsoever on the command-line)

Real apps need more features than we usually offer and they need much faster development cycles than supported by our 18-24 month release cycle.

I think my suggestion has been misunderstood: I don't want to turn these into real apps, they're fine as basic scripts/helpers to spend 5mn on a task instead of half an hour. I think they deserve better than to be documented and known through StackOverflow or Reddit "what are the -m stdlib features" lists.

To the extent the standard library serves as a set of examples, that is just side benefit. There are other ways to make code specifically for examples (we include tons of examples in the documentation; we had a Demos directory; there is the wiki; and there is the ASPN Cookbook site; etc)

Sure, but I think it's still an important driver of how things "are done" in that they're *concrete* examples, not abstract (of course the Cookbook is generally concrete as well). I'm not discounting the importance or quality of the rest of the documentation at all, or at least that was not my intention.

All that being said, there are some exceptions and it make may sense to document the interface in some where we really do want a command-line app. I'll look at any patches you want to submit, but try to not go wild turning the library into a suite of applications. For the most part, that is not what the standard library is about.

As I said, my only intention here is be to document (and argparsify/formalize) what is already there. I considered doing more (e.g. for the specific case of smtpd-as-a-script making ports optional even when hosts are specified, that kind of things) but decided against this distraction, I really just want to make existing module-as-script features simpler to discover and use. That said, do you have guidelines of which areas this idea would be most interestingly/efficiently applied? Maybe a list of modules-as-scripts you know are used regularly and could benefit from some improvements in their interface or documentation? ---------- _______________________________________ Python tracker http://bugs.python.org/issue11260 _______________________________________

Barry A. Warsaw added the comment: I'd support updating to argparse *if* you write a test suite for full coverage of the existing cli first. Once that passes, you'll have more confidence in your port. Modernizing the argument parsing in that case would be a useful addition. Not critical, but on the whole I think a good thing. ---------- _______________________________________ Python tracker http://bugs.python.org/issue11260 _______________________________________

Xavier Morel added the comment:

Only "document and formalize" the parts that are well thought out.

I don't believe I have the knowledge, right or ability to make that call for any module or package but a pair of extremely obvious ones (http.server seems a pretty good candidate as it's documented in the module doc, and is just missing a CLI help). Should I create a bug and nosy the people who have been involved in the module to get their opinion for each one? (as with this one, but maybe with a different wording/initial proposal) For this one, David expresses concerns on the stability on the interface (and the point of the idea), you express even bigger concerns with the idea itself and more generally modules-as-scripts in the stdlib (not their existence so much as their official support, which full documentation would imply, if I didn't misunderstand you), and Barry seems to okay the idea as long as an extensive test suite is created beforehand to ensure there is no regression, is that sufficient to go ahead with more work and defer the final decision for when that will be done? Also, if this is to become an actual project if the smtpd stuff ever reaches fruition (though not necessarily a big or impactful one), should there be a meta-bug? (I know they're used in many bugzilla projects and I see the tracker handles dependencies)

There's no harm in adding --help or a usage example, but please avoid making behavior guarantees that we really don't want to have to live with.

I'm not sure what that results in: for "quick hacks" which are not to be officially supported, does this mean fixing the CLI (adding a help for tools missing one) is OK but no more? Is an argparse switch still acceptable? Tests for the CLI tool? I'm guessing documentation in the module would definitely be off limits for those, is my interpretation correct? And again, who would be allowed to make the call on which modules-as-scripts may be considered supported, and can't be? With this one, I created separate patches for the documentation and the CLI parsing alterations, which would allow merging the CLI without adding it to the official documentation (which would I think imply a lack of official support), would that be OK for future works, if this one pans out? ---------- _______________________________________ Python tracker http://bugs.python.org/issue11260 _______________________________________

Changes by Raymond Hettinger : ---------- assignee: rhettinger -> _______________________________________ Python tracker http://bugs.python.org/issue11260 _______________________________________

Nick Coghlan added the comment: I created #14945 as a suggestion to add a simple page to http://docs.python.org/dev/using/index.html that will provide a central reference to the module documentation for modules with officially supported behaviour when used with the "-m" switch. ---------- nosy: +ncoghlan _______________________________________ Python tracker http://bugs.python.org/issue11260 _______________________________________

Changes by Berker Peksag : ---------- nosy: +berker.peksag stage: -> patch review _______________________________________ Python tracker http://bugs.python.org/issue11260 _______________________________________