<br><br><div class="gmail_quote">On Mon, Oct 22, 2012 at 9:52 AM, Jasper St. Pierre <span dir="ltr"><<a href="mailto:jstpierre@mecheye.net" target="_blank">jstpierre@mecheye.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On Sat, Oct 20, 2012 at 8:33 PM, Andrew Moffat<br>
<<a href="mailto:andrew.robert.moffat@gmail.com">andrew.robert.moffat@gmail.com</a>> wrote:<br>
</div><div><div class="h5">> Hi,<br>
><br>
> I'm the author of sh.py, an intuitive interface for launching subprocesses<br>
> in Linux and OSX <a href="http://amoffat.github.com/sh/" target="_blank">http://amoffat.github.com/sh/</a>. It has been maintained on<br>
> github <a href="https://github.com/amoffat/sh" target="_blank">https://github.com/amoffat/sh</a> for about 10 months and currently has<br>
> about 25k installs, according to <a href="http://pythonpackages.com" target="_blank">pythonpackages.com</a><br>
> (<a href="http://pythonpackages.com/package/sh" target="_blank">http://pythonpackages.com/package/sh</a>,<br>
> <a href="http://pythonpackages.com/package/pbs" target="_blank">http://pythonpackages.com/package/pbs</a>)<br>
><br>
> Andy Grover maintains the Fedora rpm for sh.py<br>
> <a href="http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=94247" target="_blank">http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=94247</a> and Nick<br>
> Moffit has submitted an older version of sh.py (which was called pbs) to be<br>
> included in Debian distros<br>
> <a href="http://pkgs.org/debian-wheezy/debian-main-i386/python-pbs_0.95-1_all.deb.html" target="_blank">http://pkgs.org/debian-wheezy/debian-main-i386/python-pbs_0.95-1_all.deb.html</a><br>
><br>
> I'm interested in making sh.py more accessible to help bring Python forward<br>
> in the area of shell scripting, so I'm interested in seeing if sh would be<br>
> suitable for the standard library. Is there any other interest in something<br>
> like this?<br>
<br>
</div></div>I'm not one for the sugar. Seems like you're stuffing the Python<br>
syntax where it doesn't quite belong, as evidenced by the many escape<br>
hatches. Basic query of things not covered in the documentation:<br>
<br>
If I import a non-existant program, will it give me back a function<br>
that will fail or raise an ImportError?<br>
<br>
How do I run a program with a - in the name? You say you replace -<br>
with _, but thatdoesn't specify what happens in the edge case of "if I<br>
have google-chrome and google_chrome, which one wins? What about<br>
/usr/bin/google-chrome and /usr/local/bin/google_chrome"? That is,<br>
will it exhaust the PATH before trying fallbacks replacements or will<br>
it check all replacements at once?<br>
<br>
If I have a program that's not on PATH, what do I do? I can manipulate<br>
the PATH environment variable, but am I guaranteed that will work? Are<br>
you going to double fork forever to guarantee that environment? Can I<br>
build a custom prefix, like p =<br>
sh.MagicPrefix(path="/opt/android_devtools/bin"), and have that work<br>
like the regular sh module? p.gcc("whatever") ? Even with the<br>
existence of a regular gcc in the path?<br>
<br>
I wonder what happens if you do from sh import *.<br>
<br>
Does it block execution before continuing? How can I do parallel<br>
execution of four subprocesses, and get notified when all four are<br>
done? (Seems like this might be a thing for a Future as well, even in<br>
the absence of any scheduler or event loop).<br>
<br>
Are newcomers going to be confused by this? What happens if I try and<br>
do something like <a href="http://sh.ls" target="_blank">sh.ls</a>("-l -a")? Will you use the POSIX shell parsing<br>
algorithm, pass it to bash, or pass it as one parameter? Will some<br>
form of injection attack be mitigated by this design?<br>
<br>
<br>
If you see this magic syntax as your one unique feature, I'd propose<br>
that you add it to the subprocess module, and improve the standard<br>
subprocess module's interface to cope with the new feature.<br>
<br>
But I don't see this as a worthwhile thing to have. -1 on the thing.<br>
<br>
> Thanks<br>
<div class="HOEnZb"><div class="h5">><br>
> _______________________________________________<br>
> Python-ideas mailing list<br>
> <a href="mailto:Python-ideas@python.org">Python-ideas@python.org</a><br>
> <a href="http://mail.python.org/mailman/listinfo/python-ideas" target="_blank">http://mail.python.org/mailman/listinfo/python-ideas</a><br>
><br>
<br>
</div></div><span class="HOEnZb"><font color="#888888">--<br>
Jasper<br>
</font></span></blockquote></div><br><div><br></div><div><br></div><div>Hi Jasper, thanks for your questions</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<span style="background-color:rgb(255,255,255);color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px">If I import a non-existant program, will it give me back a function<br></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">that will fail or raise an ImportError?</span></blockquote>
<div><br></div><div>Yes, an exception will be raised</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<span style="background-color:rgb(255,255,255);color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px">How do I run a program with a - in the name? You say you replace -<br></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">with _, but thatdoesn't specify what happens in the edge case of "if I<br>
</span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">have google-chrome and google_chrome, which one wins? What about<br></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">/usr/bin/google-chrome and /usr/local/bin/google_chrome"? That is,<br>
</span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">will it exhaust the PATH before trying fallbacks replacements or will<br></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">it check all replacements at once?</span></blockquote>
<div><br></div><div>The full PATH will be exhausted for the exact command, as typed, before any kind of "-" replacement is exercised. There hasn't been much concern about this because most people who want to call commands with special characters prefer to use the Command class (e.g. chrome = Command("/usr/bin/google-chrome")), so the documentation makes a note of this on this issue.</div>
<div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">If I have a program that's not on PATH, what do I do? I can manipulate<br>
</span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">the PATH environment variable, but am I guaranteed that will work? Are<br></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">you going to double fork forever to guarantee that environment? Can I<br>
</span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">build a custom prefix, like p =<br></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">sh.MagicPrefix(path="/opt/</span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">android_devtools/bin"), and have that work<br>
</span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">like the regular sh module? p.gcc("whatever") ? Even with the<br></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">existence of a regular gcc in the path?</span></blockquote>
<div> </div><div>You could manipulate the PATH, but a better way would be to use the Command class, which can take a full path of a command. The returned object can be used just like other commands.</div><div><br></div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">I wonder what happens if you do from sh import *.</span></blockquote><div><br></div><div>"ImportError: Cannot import * from sh. Please import sh or import programs individually." Commands are lazy resolved on sh anyways, so loading from all would be undefined.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">Does it block execution before continuing? How can I do parallel<br>
</span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">execution of four subprocesses, and get notified when all four are<br></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">done? (Seems like this might be a thing for a Future as well, even in<br>
</span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">the absence of any scheduler or event loop).</span></blockquote><div><br></div><div>Commands may be run in the background, like this:</div>
<div><br></div><div>job1 = sh.tar("-zc", "-f", "archive-name.tar.gz", "/some/directory", _bg=True)</div><div>job2 = sh.tar(..., _bg=True)</div><div>job3 = sh.tar(..., _bg=True)</div>
<div><br></div><div>job1.wait()</div><div>job2.wait()</div><div>job3.wait()</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">Are newcomers going to be confused by this? What happens if I try and<br></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">do something like </span><a href="http://sh.ls/" target="_blank" style="color:rgb(17,85,204);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">sh.ls</a><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">("-l -a")? Will you use the POSIX shell parsing<br>
</span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">algorithm, pass it to bash, or pass it as one parameter? Will some<br></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">form of injection attack be mitigated by this design?</span></blockquote>
<div><br></div><div>Bash--nor any shell-- is called into play. Sh.py doesn't do any argument parsing either. Arguments are passed into commands exactly as they're sent through sh.py. Newcomers have loved it so far, and after seeing some examples, there's been minimal confusion about how to use it.</div>
<div><br></div><div>My thoughts about the magical-ness of sh.py.. I typically don't support very magical or dynamically resolving modules. I can be a little apprehensive of ORMs for this reason... I like to know how my code behaves explicitly. I think clever/magic can be confusing for people and inexplicit, and it's important to know more or less what's going on under the hood. But I also think that sh.py scratches an itch that a less dynamic approach fails to reach. My goal for sh.py has been to make writing system scripts as easy for Python as it is for Bash. People who write Bash scripts do so for a few reasons, one being that a shell script is pretty portable for *nix systems, and also because it's very easy to call programs with arguments, feed them input, and parse their output. But the shortcomings of shell scripts are how obfuscated and unnecessarily difficult they are to accomplish generic programming tasks. This is somewhere where Python excels. But unfortunately, until sh.py, I have not found any tool that lets you call commands nearly as easily as Bash, that didn't rely on Bash. Subprocess is painful. Other modules are extremely verbose. Sh.py, yes, uses a dynamic lookup mechanism (but it should be noted that you don't have to rely on it *at all* if you don't like it), but it does so with a very specific intention, and that is to make Python more suited and commonplace for writing system shell scripts.</div>
<div><br></div><div>My push here to see if there is support is because I believe if sh.py could enter the stdlib, and therefore become more ubiquitous on Linux and OSX, that more shell-style scripts could be written in Python, more new users would be comfortable in using Python, and Bash scripts could go the way of the dodo :)</div>
<div><br></div><div><br></div><div>Andrew</div>