On Sat, 20 Oct 2012 21:02:58 -0400 Mike Graham <mikegraham@gmail.com> wrote:

On Sat, Oct 20, 2012 at 8:33 PM, Andrew Moffat <andrew.robert.moffat@gmail.com> wrote:

...

Hi,

I'm the author of sh.py, an intuitive interface for launching subprocesses in Linux and OSX http://amoffat.github.com/sh/. It has been maintained on github https://github.com/amoffat/sh for about 10 months and currently has about 25k installs, according to pythonpackages.com (http://pythonpackages.com/package/sh, http://pythonpackages.com/package/pbs)

Andy Grover maintains the Fedora rpm for sh.py http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=94247 and Nick Moffit has submitted an older version of sh.py (which was called pbs) to be included in Debian distros http://pkgs.org/debian-wheezy/debian-main-i386/python-pbs_0.95-1_all.deb.htm...

I'm interested in making sh.py more accessible to help bring Python forward in the area of shell scripting, so I'm interested in seeing if sh would be suitable for the standard library. Is there any other interest in something like this?

Thanks

sh.py strikes me as on the clever side for the stdlib and the lack of Windows support would be very unfortunate for a stdlib module (I don't know if this is relatively easily fixed, though it seems possible)

Ditto for me. The basic concept of the sh module looks like some fancy wrapper around subprocess.check_output: http://docs.python.org/dev/library/subprocess.html#subprocess.check_output The "easy chaining of subprocesses" part does not look that useful to me, or at least the examples aren't very convincing. If I want to sort the results of a shell command, it makes much more sense to me to do so using Python's text processing and sorting capabilities, than trying to find the right invocation of Unix "sort" and other utilities. That said, I do find the "fancy wrapper" part somewhat pretty. Regards Antoine.

The main criticism has been the cleverness of the dynamic lookups. There is also the ability to use a Command object for more explicit calls: cmd = sh.Command("/some/command") cmd(arg) So you have the best of both worlds. If you like the idea of the programs being attributes on the module, you can use the advertised way, if you don't, you can use the more explicit way. Windows support would be a little more difficult. It existed in an old version of sh, when it was merely a wrapper around the subprocess module. Now that sh.py no longer relies on the subprocess module and does fork-exec itself (in order to get more flexible access to the processes), Windows is currently unsupported. My current understanding is that most of the value comes from the linux/OSX folks, but Windows support is scheduled for the future. On Sat, Oct 20, 2012 at 8:02 PM, Mike Graham <mikegraham@gmail.com> wrote:

On Sat, Oct 20, 2012 at 8:33 PM, Andrew Moffat <andrew.robert.moffat@gmail.com> wrote:

...

Hi,

I'm the author of sh.py, an intuitive interface for launching subprocesses in Linux and OSX http://amoffat.github.com/sh/. It has been maintained on github https://github.com/amoffat/sh for about 10 months and currently has about 25k installs, according to pythonpackages.com (http://pythonpackages.com/package/sh, http://pythonpackages.com/package/pbs)

Andy Grover maintains the Fedora rpm for sh.py http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=94247 and Nick Moffit has submitted an older version of sh.py (which was called pbs) to be included in Debian distros

http://pkgs.org/debian-wheezy/debian-main-i386/python-pbs_0.95-1_all.deb.htm...

...

I'm interested in making sh.py more accessible to help bring Python

forward

...

in the area of shell scripting, so I'm interested in seeing if sh would be suitable for the standard library. Is there any other interest in something like this?

Thanks

sh.py strikes me as on the clever side for the stdlib and the lack of Windows support would be very unfortunate for a stdlib module (I don't know if this is relatively easily fixed, though it seems possible)

Mike

On 22.10.12 13:52, Christian Tismer wrote:

On 22.10.12 04:40, Andrew Moffat wrote:

...

The main criticism has been the cleverness of the dynamic lookups. There is also the ability to use a Command object for more explicit calls:

cmd = sh.Command("/some/command") cmd(arg)

So you have the best of both worlds. If you like the idea of the programs being attributes on the module, you can use the advertised way, if you don't, you can use the more explicit way.

Windows support would be a little more difficult. It existed in an old version of sh, when it was merely a wrapper around the subprocess module. Now that sh.py no longer relies on the subprocess module and does fork-exec itself (in order to get more flexible access to the processes), Windows is currently unsupported. My current understanding is that most of the value comes from the linux/OSX folks, but Windows support is scheduled for the future.

This is what I don't like:

subprocess is not used, but you implement stuff yourself. Instead of bypassing subprocess I would improve subprocess and not duplicate the windows problem, which is most of the time _not_ easy to get right.

Can you explain why you went this path?

Sorry, while we are at it: The package name is a problem for me. A two-character name for a package?? That is something that I would never do in the global package namespace. It also is IMHO not nice to have such short names in PyPI. -- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

I would be interested in relicensing and donating. I am able to reach out to the contributors, and I am pretty positive I could reach out and get the signing off from them. I would be more than willing to maintain the package as well...I'm in it for the long haul, it seems to resonated well with the community throughout its development. On Sun, Oct 21, 2012 at 8:35 AM, Christian Heimes <christian@python.org>wrote:

Am 21.10.2012 02:33, schrieb Andrew Moffat:

...

I'm interested in making sh.py more accessible to help bring Python forward in the area of shell scripting, so I'm interested in seeing if sh would be suitable for the standard library. Is there any other interest in something like this?

I like to ignore the technical issues for now and concentrate on the legal and organizational problems.

In order to get sh.py into Python's stdlib you have to relicense and donate the code under the PSF license. You and every contributor must agree on the relicensing. At least you must submit a signed contributor agreement, maybe every contributor. Are you able to get hold of everybody?

Are you willing to maintain your code for several years, at least five years or more?

Regards, Christian

On Mon, Oct 22, 2012 at 9:52 AM, Jasper St. Pierre <jstpierre@mecheye.net>wrote:

On Sat, Oct 20, 2012 at 8:33 PM, Andrew Moffat <andrew.robert.moffat@gmail.com> wrote:

...

Hi,

I'm the author of sh.py, an intuitive interface for launching subprocesses in Linux and OSX http://amoffat.github.com/sh/. It has been maintained on github https://github.com/amoffat/sh for about 10 months and currently has about 25k installs, according to pythonpackages.com (http://pythonpackages.com/package/sh, http://pythonpackages.com/package/pbs)

Andy Grover maintains the Fedora rpm for sh.py http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=94247 and Nick Moffit has submitted an older version of sh.py (which was called pbs) to be included in Debian distros

http://pkgs.org/debian-wheezy/debian-main-i386/python-pbs_0.95-1_all.deb.htm...

...

I'm interested in making sh.py more accessible to help bring Python

forward

...

in the area of shell scripting, so I'm interested in seeing if sh would be suitable for the standard library. Is there any other interest in something like this?

I'm not one for the sugar. Seems like you're stuffing the Python syntax where it doesn't quite belong, as evidenced by the many escape hatches. Basic query of things not covered in the documentation:

If I import a non-existant program, will it give me back a function that will fail or raise an ImportError?

How do I run a program with a - in the name? You say you replace - with _, but thatdoesn't specify what happens in the edge case of "if I have google-chrome and google_chrome, which one wins? What about /usr/bin/google-chrome and /usr/local/bin/google_chrome"? That is, will it exhaust the PATH before trying fallbacks replacements or will it check all replacements at once?

If I have a program that's not on PATH, what do I do? I can manipulate the PATH environment variable, but am I guaranteed that will work? Are you going to double fork forever to guarantee that environment? Can I build a custom prefix, like p = sh.MagicPrefix(path="/opt/android_devtools/bin"), and have that work like the regular sh module? p.gcc("whatever") ? Even with the existence of a regular gcc in the path?

I wonder what happens if you do from sh import *.

Does it block execution before continuing? How can I do parallel execution of four subprocesses, and get notified when all four are done? (Seems like this might be a thing for a Future as well, even in the absence of any scheduler or event loop).

Are newcomers going to be confused by this? What happens if I try and do something like sh.ls("-l -a")? Will you use the POSIX shell parsing algorithm, pass it to bash, or pass it as one parameter? Will some form of injection attack be mitigated by this design?

If you see this magic syntax as your one unique feature, I'd propose that you add it to the subprocess module, and improve the standard subprocess module's interface to cope with the new feature.

But I don't see this as a worthwhile thing to have. -1 on the thing.

...

Thanks

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- Jasper

Hi Jasper, thanks for your questions If I import a non-existant program, will it give me back a function

that will fail or raise an ImportError?

Yes, an exception will be raised How do I run a program with a - in the name? You say you replace -

with _, but thatdoesn't specify what happens in the edge case of "if I have google-chrome and google_chrome, which one wins? What about /usr/bin/google-chrome and /usr/local/bin/google_chrome"? That is, will it exhaust the PATH before trying fallbacks replacements or will it check all replacements at once?

The full PATH will be exhausted for the exact command, as typed, before any kind of "-" replacement is exercised. There hasn't been much concern about this because most people who want to call commands with special characters prefer to use the Command class (e.g. chrome = Command("/usr/bin/google-chrome")), so the documentation makes a note of this on this issue. If I have a program that's not on PATH, what do I do? I can manipulate

the PATH environment variable, but am I guaranteed that will work? Are you going to double fork forever to guarantee that environment? Can I build a custom prefix, like p = sh.MagicPrefix(path="/opt/android_devtools/bin"), and have that work like the regular sh module? p.gcc("whatever") ? Even with the existence of a regular gcc in the path?

You could manipulate the PATH, but a better way would be to use the Command class, which can take a full path of a command. The returned object can be used just like other commands. I wonder what happens if you do from sh import *. "ImportError: Cannot import * from sh. Please import sh or import programs individually." Commands are lazy resolved on sh anyways, so loading from all would be undefined. Does it block execution before continuing? How can I do parallel

execution of four subprocesses, and get notified when all four are done? (Seems like this might be a thing for a Future as well, even in the absence of any scheduler or event loop).

Commands may be run in the background, like this: job1 = sh.tar("-zc", "-f", "archive-name.tar.gz", "/some/directory", _bg=True) job2 = sh.tar(..., _bg=True) job3 = sh.tar(..., _bg=True) job1.wait() job2.wait() job3.wait() Are newcomers going to be confused by this? What happens if I try and

do something like sh.ls("-l -a")? Will you use the POSIX shell parsing algorithm, pass it to bash, or pass it as one parameter? Will some form of injection attack be mitigated by this design?

Bash--nor any shell-- is called into play. Sh.py doesn't do any argument parsing either. Arguments are passed into commands exactly as they're sent through sh.py. Newcomers have loved it so far, and after seeing some examples, there's been minimal confusion about how to use it. My thoughts about the magical-ness of sh.py.. I typically don't support very magical or dynamically resolving modules. I can be a little apprehensive of ORMs for this reason... I like to know how my code behaves explicitly. I think clever/magic can be confusing for people and inexplicit, and it's important to know more or less what's going on under the hood. But I also think that sh.py scratches an itch that a less dynamic approach fails to reach. My goal for sh.py has been to make writing system scripts as easy for Python as it is for Bash. People who write Bash scripts do so for a few reasons, one being that a shell script is pretty portable for *nix systems, and also because it's very easy to call programs with arguments, feed them input, and parse their output. But the shortcomings of shell scripts are how obfuscated and unnecessarily difficult they are to accomplish generic programming tasks. This is somewhere where Python excels. But unfortunately, until sh.py, I have not found any tool that lets you call commands nearly as easily as Bash, that didn't rely on Bash. Subprocess is painful. Other modules are extremely verbose. Sh.py, yes, uses a dynamic lookup mechanism (but it should be noted that you don't have to rely on it *at all* if you don't like it), but it does so with a very specific intention, and that is to make Python more suited and commonplace for writing system shell scripts. My push here to see if there is support is because I believe if sh.py could enter the stdlib, and therefore become more ubiquitous on Linux and OSX, that more shell-style scripts could be written in Python, more new users would be comfortable in using Python, and Bash scripts could go the way of the dodo :) Andrew