Interest in seeing sh.py in the stdlib
Hi, I'm the author of sh.py, an intuitive interface for launching subprocesses in Linux and OSX http://amoffat.github.com/sh/. It has been maintained on github https://github.com/amoffat/sh for about 10 months and currently has about 25k installs, according to pythonpackages.com ( http://pythonpackages.com/package/sh, http://pythonpackages.com/package/pbs) Andy Grover maintains the Fedora rpm for sh.py http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=94247 and Nick Moffit has submitted an older version of sh.py (which was called pbs) to be included in Debian distros http://pkgs.org/debian-wheezy/debian-main-i386/python-pbs_0.95-1_all.deb.htm... I'm interested in making sh.py more accessible to help bring Python forward in the area of shell scripting, so I'm interested in seeing if sh would be suitable for the standard library. Is there any other interest in something like this? Thanks
On Sat, Oct 20, 2012 at 8:33 PM, Andrew Moffat <andrew.robert.moffat@gmail.com> wrote:
sh.py strikes me as on the clever side for the stdlib and the lack of Windows support would be very unfortunate for a stdlib module (I don't know if this is relatively easily fixed, though it seems possible) Mike
On Sat, 20 Oct 2012 21:02:58 -0400 Mike Graham <mikegraham@gmail.com> wrote:
Ditto for me. The basic concept of the sh module looks like some fancy wrapper around subprocess.check_output: http://docs.python.org/dev/library/subprocess.html#subprocess.check_output The "easy chaining of subprocesses" part does not look that useful to me, or at least the examples aren't very convincing. If I want to sort the results of a shell command, it makes much more sense to me to do so using Python's text processing and sorting capabilities, than trying to find the right invocation of Unix "sort" and other utilities. That said, I do find the "fancy wrapper" part somewhat pretty. Regards Antoine.
Antoine Pitrou <solipsis@...> writes:
That said, I do find the "fancy wrapper" part somewhat pretty.
One thing that's not very pretty about it is the need to use "-" prefixed parameters to get special behavior. It might be nicer if the command was an object and you called special methods on it to get special behavior. Ex: wget.background("example.com") sudo.context("extra", "args") Benjamin
The main criticism has been the cleverness of the dynamic lookups. There is also the ability to use a Command object for more explicit calls: cmd = sh.Command("/some/command") cmd(arg) So you have the best of both worlds. If you like the idea of the programs being attributes on the module, you can use the advertised way, if you don't, you can use the more explicit way. Windows support would be a little more difficult. It existed in an old version of sh, when it was merely a wrapper around the subprocess module. Now that sh.py no longer relies on the subprocess module and does fork-exec itself (in order to get more flexible access to the processes), Windows is currently unsupported. My current understanding is that most of the value comes from the linux/OSX folks, but Windows support is scheduled for the future. On Sat, Oct 20, 2012 at 8:02 PM, Mike Graham <mikegraham@gmail.com> wrote:
On 22.10.12 04:40, Andrew Moffat wrote:
This is what I don't like: subprocess is not used, but you implement stuff yourself. Instead of bypassing subprocess I would improve subprocess and not duplicate the windows problem, which is most of the time _not_ easy to get right. Can you explain why you went this path? cheers - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
On 22.10.12 13:52, Christian Tismer wrote:
Sorry, while we are at it: The package name is a problem for me. A two-character name for a package?? That is something that I would never do in the global package namespace. It also is IMHO not nice to have such short names in PyPI. -- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Andrew Moffat <andrew.robert.moffat@...> writes:
The main criticism has been the cleverness of the dynamic lookups.
I would add: * The plethora of special keyword arguments like _bg, _iter, _in, _piped etc. doesn't look good. * Using callbacks for processing stream output makes it harder to do certain kinds of processing on that output.
This isn't good. You may have resorted to bypassing subprocess because it didn't do what you needed, but it certainly wouldn't look good if a proposed stdlib module wasn't eating its own dog food (by which I mean, using subprocess). Though there have been precedents (optparse / argparse), a determined effort was made there to work with the existing stdlib module before giving up on it. From my own experience, subprocess has not been that intractable, so I'm curious - what flexibility of access did you need that subprocess couldn't offer? I would guess things that are essentially non-portable, like tty access to provide pexpect-like behaviour. (I had to eschew this for sarge, in the interests of cross-platform compatibility.)
It seems to me premature to propose sh.py for inclusion in the stdlib before offering Windows support. After all, those who need it can readily get hold of it from PyPI, as the impressive download numbers show. Just as its design has changed a fair bit going from pbs to sh.py, it may change yet more when Windows support is added, and it can be looked at again then. Regards, Vinay Sajip
Am 21.10.2012 02:33, schrieb Andrew Moffat:
I like to ignore the technical issues for now and concentrate on the legal and organizational problems. In order to get sh.py into Python's stdlib you have to relicense and donate the code under the PSF license. You and every contributor must agree on the relicensing. At least you must submit a signed contributor agreement, maybe every contributor. Are you able to get hold of everybody? Are you willing to maintain your code for several years, at least five years or more? Regards, Christian
I would be interested in relicensing and donating. I am able to reach out to the contributors, and I am pretty positive I could reach out and get the signing off from them. I would be more than willing to maintain the package as well...I'm in it for the long haul, it seems to resonated well with the community throughout its development. On Sun, Oct 21, 2012 at 8:35 AM, Christian Heimes <christian@python.org>wrote:
On 22.10.2012 04:40, Andrew Moffat wrote:
Small correction: The contributors would have to sign a contributor agreement with the PSF to enable the PSF to distribute the code under the PSF license: http://www.python.org/psf/contrib/ This usually is much easier to have than a copyright sign-over, since it's only a special license and the copyright remains with the authors.
Are you willing to maintain your code for several years, at least five years or more?
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 22 2012)
2012-09-27: Released eGenix PyRun 1.1.0 ... http://egenix.com/go35 2012-09-26: Released mxODBC.Connect 2.0.1 ... http://egenix.com/go34 2012-09-25: Released mxODBC 3.2.1 ... http://egenix.com/go33 2012-10-23: Python Meeting Duesseldorf ... tomorrow eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
Andrew Moffat <andrew.robert.moffat@...> writes:
I would agree with others who have replied saying that the approach is cute, but a little too magical. Disclosure: this is an area of interest for me, and I maintain a project called sarge [1] which sort of fits in the same space as pbs/sh. It doesn't have the cute shell-command-as-Python-function idiom (which, in my view, buys very little readability), but it does aim to offer some features which (AFAICT) sh doesn't have. I'll just list sarge's features briefly below, if for no other reason than to show that there are other contenders worth considering (should there be a consensus that the stdlib needs batteries in this area). Sarge improves upon subprocess when: * You want to use command pipelines, but using subprocess out of the box often leads to deadlocks because pipe buffers get filled up. * You want to use bash-style pipe syntax on Windows, but some Windows shells don’t support some of the syntax you want to use, like &&, ||, |& and so on. * You want to process output from commands in a flexible way, and communicate() is not flexible enough for your needs – for example, you need to process output a line at a time. * You want to avoid shell injection problems by having the ability to quote your command arguments safely. * subprocess allows you to let stderr be the same as stdout, but not the other way around – and you need to do that. It offers: * A simple run command which allows a rich subset of Bash-style shell command syntax, but parsed and run by sarge so that you can run identically on Windows without cygwin. This includes asynchronous calls (using "&" just as in bash). * The ability to format shell commands with placeholders, such that variables are quoted to prevent shell injection attacks. * The ability to capture output streams without requiring you to program your own threads. You just use a Capture object and then you can read from it as and when you want. A Capture object can capture the output from multiple chained commands. * Delays in commands (e.g. "sleep") are honoured in asynchronous calls. I would also concur with others who've pointed out that stdlib maintenance is a long haul affair. I've been maintaining the logging package for around 10 years now :-) Regards, Vinay Sajip [1] http://sarge.readthedocs.org/
On Sat, Oct 20, 2012 at 8:33 PM, Andrew Moffat <andrew.robert.moffat@gmail.com> wrote:
I'm not one for the sugar. Seems like you're stuffing the Python syntax where it doesn't quite belong, as evidenced by the many escape hatches. Basic query of things not covered in the documentation: If I import a non-existant program, will it give me back a function that will fail or raise an ImportError? How do I run a program with a - in the name? You say you replace - with _, but thatdoesn't specify what happens in the edge case of "if I have google-chrome and google_chrome, which one wins? What about /usr/bin/google-chrome and /usr/local/bin/google_chrome"? That is, will it exhaust the PATH before trying fallbacks replacements or will it check all replacements at once? If I have a program that's not on PATH, what do I do? I can manipulate the PATH environment variable, but am I guaranteed that will work? Are you going to double fork forever to guarantee that environment? Can I build a custom prefix, like p = sh.MagicPrefix(path="/opt/android_devtools/bin"), and have that work like the regular sh module? p.gcc("whatever") ? Even with the existence of a regular gcc in the path? I wonder what happens if you do from sh import *. Does it block execution before continuing? How can I do parallel execution of four subprocesses, and get notified when all four are done? (Seems like this might be a thing for a Future as well, even in the absence of any scheduler or event loop). Are newcomers going to be confused by this? What happens if I try and do something like sh.ls("-l -a")? Will you use the POSIX shell parsing algorithm, pass it to bash, or pass it as one parameter? Will some form of injection attack be mitigated by this design? If you see this magic syntax as your one unique feature, I'd propose that you add it to the subprocess module, and improve the standard subprocess module's interface to cope with the new feature. But I don't see this as a worthwhile thing to have. -1 on the thing.
-- Jasper
On Mon, Oct 22, 2012 at 9:52 AM, Jasper St. Pierre <jstpierre@mecheye.net>wrote:
Hi Jasper, thanks for your questions If I import a non-existant program, will it give me back a function
that will fail or raise an ImportError?
Yes, an exception will be raised How do I run a program with a - in the name? You say you replace -
The full PATH will be exhausted for the exact command, as typed, before any kind of "-" replacement is exercised. There hasn't been much concern about this because most people who want to call commands with special characters prefer to use the Command class (e.g. chrome = Command("/usr/bin/google-chrome")), so the documentation makes a note of this on this issue. If I have a program that's not on PATH, what do I do? I can manipulate
You could manipulate the PATH, but a better way would be to use the Command class, which can take a full path of a command. The returned object can be used just like other commands. I wonder what happens if you do from sh import *. "ImportError: Cannot import * from sh. Please import sh or import programs individually." Commands are lazy resolved on sh anyways, so loading from all would be undefined. Does it block execution before continuing? How can I do parallel
Commands may be run in the background, like this: job1 = sh.tar("-zc", "-f", "archive-name.tar.gz", "/some/directory", _bg=True) job2 = sh.tar(..., _bg=True) job3 = sh.tar(..., _bg=True) job1.wait() job2.wait() job3.wait() Are newcomers going to be confused by this? What happens if I try and
Bash--nor any shell-- is called into play. Sh.py doesn't do any argument parsing either. Arguments are passed into commands exactly as they're sent through sh.py. Newcomers have loved it so far, and after seeing some examples, there's been minimal confusion about how to use it. My thoughts about the magical-ness of sh.py.. I typically don't support very magical or dynamically resolving modules. I can be a little apprehensive of ORMs for this reason... I like to know how my code behaves explicitly. I think clever/magic can be confusing for people and inexplicit, and it's important to know more or less what's going on under the hood. But I also think that sh.py scratches an itch that a less dynamic approach fails to reach. My goal for sh.py has been to make writing system scripts as easy for Python as it is for Bash. People who write Bash scripts do so for a few reasons, one being that a shell script is pretty portable for *nix systems, and also because it's very easy to call programs with arguments, feed them input, and parse their output. But the shortcomings of shell scripts are how obfuscated and unnecessarily difficult they are to accomplish generic programming tasks. This is somewhere where Python excels. But unfortunately, until sh.py, I have not found any tool that lets you call commands nearly as easily as Bash, that didn't rely on Bash. Subprocess is painful. Other modules are extremely verbose. Sh.py, yes, uses a dynamic lookup mechanism (but it should be noted that you don't have to rely on it *at all* if you don't like it), but it does so with a very specific intention, and that is to make Python more suited and commonplace for writing system shell scripts. My push here to see if there is support is because I believe if sh.py could enter the stdlib, and therefore become more ubiquitous on Linux and OSX, that more shell-style scripts could be written in Python, more new users would be comfortable in using Python, and Bash scripts could go the way of the dodo :) Andrew
On Sat, Oct 20, 2012 at 8:33 PM, Andrew Moffat <andrew.robert.moffat@gmail.com> wrote:
sh.py strikes me as on the clever side for the stdlib and the lack of Windows support would be very unfortunate for a stdlib module (I don't know if this is relatively easily fixed, though it seems possible) Mike
On Sat, 20 Oct 2012 21:02:58 -0400 Mike Graham <mikegraham@gmail.com> wrote:
Ditto for me. The basic concept of the sh module looks like some fancy wrapper around subprocess.check_output: http://docs.python.org/dev/library/subprocess.html#subprocess.check_output The "easy chaining of subprocesses" part does not look that useful to me, or at least the examples aren't very convincing. If I want to sort the results of a shell command, it makes much more sense to me to do so using Python's text processing and sorting capabilities, than trying to find the right invocation of Unix "sort" and other utilities. That said, I do find the "fancy wrapper" part somewhat pretty. Regards Antoine.
Antoine Pitrou <solipsis@...> writes:
That said, I do find the "fancy wrapper" part somewhat pretty.
One thing that's not very pretty about it is the need to use "-" prefixed parameters to get special behavior. It might be nicer if the command was an object and you called special methods on it to get special behavior. Ex: wget.background("example.com") sudo.context("extra", "args") Benjamin
The main criticism has been the cleverness of the dynamic lookups. There is also the ability to use a Command object for more explicit calls: cmd = sh.Command("/some/command") cmd(arg) So you have the best of both worlds. If you like the idea of the programs being attributes on the module, you can use the advertised way, if you don't, you can use the more explicit way. Windows support would be a little more difficult. It existed in an old version of sh, when it was merely a wrapper around the subprocess module. Now that sh.py no longer relies on the subprocess module and does fork-exec itself (in order to get more flexible access to the processes), Windows is currently unsupported. My current understanding is that most of the value comes from the linux/OSX folks, but Windows support is scheduled for the future. On Sat, Oct 20, 2012 at 8:02 PM, Mike Graham <mikegraham@gmail.com> wrote:
On 22.10.12 04:40, Andrew Moffat wrote:
This is what I don't like: subprocess is not used, but you implement stuff yourself. Instead of bypassing subprocess I would improve subprocess and not duplicate the windows problem, which is most of the time _not_ easy to get right. Can you explain why you went this path? cheers - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
On 22.10.12 13:52, Christian Tismer wrote:
Sorry, while we are at it: The package name is a problem for me. A two-character name for a package?? That is something that I would never do in the global package namespace. It also is IMHO not nice to have such short names in PyPI. -- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Andrew Moffat <andrew.robert.moffat@...> writes:
The main criticism has been the cleverness of the dynamic lookups.
I would add: * The plethora of special keyword arguments like _bg, _iter, _in, _piped etc. doesn't look good. * Using callbacks for processing stream output makes it harder to do certain kinds of processing on that output.
This isn't good. You may have resorted to bypassing subprocess because it didn't do what you needed, but it certainly wouldn't look good if a proposed stdlib module wasn't eating its own dog food (by which I mean, using subprocess). Though there have been precedents (optparse / argparse), a determined effort was made there to work with the existing stdlib module before giving up on it. From my own experience, subprocess has not been that intractable, so I'm curious - what flexibility of access did you need that subprocess couldn't offer? I would guess things that are essentially non-portable, like tty access to provide pexpect-like behaviour. (I had to eschew this for sarge, in the interests of cross-platform compatibility.)
It seems to me premature to propose sh.py for inclusion in the stdlib before offering Windows support. After all, those who need it can readily get hold of it from PyPI, as the impressive download numbers show. Just as its design has changed a fair bit going from pbs to sh.py, it may change yet more when Windows support is added, and it can be looked at again then. Regards, Vinay Sajip
Am 21.10.2012 02:33, schrieb Andrew Moffat:
I like to ignore the technical issues for now and concentrate on the legal and organizational problems. In order to get sh.py into Python's stdlib you have to relicense and donate the code under the PSF license. You and every contributor must agree on the relicensing. At least you must submit a signed contributor agreement, maybe every contributor. Are you able to get hold of everybody? Are you willing to maintain your code for several years, at least five years or more? Regards, Christian
I would be interested in relicensing and donating. I am able to reach out to the contributors, and I am pretty positive I could reach out and get the signing off from them. I would be more than willing to maintain the package as well...I'm in it for the long haul, it seems to resonated well with the community throughout its development. On Sun, Oct 21, 2012 at 8:35 AM, Christian Heimes <christian@python.org>wrote:
On 22.10.2012 04:40, Andrew Moffat wrote:
Small correction: The contributors would have to sign a contributor agreement with the PSF to enable the PSF to distribute the code under the PSF license: http://www.python.org/psf/contrib/ This usually is much easier to have than a copyright sign-over, since it's only a special license and the copyright remains with the authors.
Are you willing to maintain your code for several years, at least five years or more?
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 22 2012)
2012-09-27: Released eGenix PyRun 1.1.0 ... http://egenix.com/go35 2012-09-26: Released mxODBC.Connect 2.0.1 ... http://egenix.com/go34 2012-09-25: Released mxODBC 3.2.1 ... http://egenix.com/go33 2012-10-23: Python Meeting Duesseldorf ... tomorrow eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
Andrew Moffat <andrew.robert.moffat@...> writes:
I would agree with others who have replied saying that the approach is cute, but a little too magical. Disclosure: this is an area of interest for me, and I maintain a project called sarge [1] which sort of fits in the same space as pbs/sh. It doesn't have the cute shell-command-as-Python-function idiom (which, in my view, buys very little readability), but it does aim to offer some features which (AFAICT) sh doesn't have. I'll just list sarge's features briefly below, if for no other reason than to show that there are other contenders worth considering (should there be a consensus that the stdlib needs batteries in this area). Sarge improves upon subprocess when: * You want to use command pipelines, but using subprocess out of the box often leads to deadlocks because pipe buffers get filled up. * You want to use bash-style pipe syntax on Windows, but some Windows shells don’t support some of the syntax you want to use, like &&, ||, |& and so on. * You want to process output from commands in a flexible way, and communicate() is not flexible enough for your needs – for example, you need to process output a line at a time. * You want to avoid shell injection problems by having the ability to quote your command arguments safely. * subprocess allows you to let stderr be the same as stdout, but not the other way around – and you need to do that. It offers: * A simple run command which allows a rich subset of Bash-style shell command syntax, but parsed and run by sarge so that you can run identically on Windows without cygwin. This includes asynchronous calls (using "&" just as in bash). * The ability to format shell commands with placeholders, such that variables are quoted to prevent shell injection attacks. * The ability to capture output streams without requiring you to program your own threads. You just use a Capture object and then you can read from it as and when you want. A Capture object can capture the output from multiple chained commands. * Delays in commands (e.g. "sleep") are honoured in asynchronous calls. I would also concur with others who've pointed out that stdlib maintenance is a long haul affair. I've been maintaining the logging package for around 10 years now :-) Regards, Vinay Sajip [1] http://sarge.readthedocs.org/
On Sat, Oct 20, 2012 at 8:33 PM, Andrew Moffat <andrew.robert.moffat@gmail.com> wrote:
I'm not one for the sugar. Seems like you're stuffing the Python syntax where it doesn't quite belong, as evidenced by the many escape hatches. Basic query of things not covered in the documentation: If I import a non-existant program, will it give me back a function that will fail or raise an ImportError? How do I run a program with a - in the name? You say you replace - with _, but thatdoesn't specify what happens in the edge case of "if I have google-chrome and google_chrome, which one wins? What about /usr/bin/google-chrome and /usr/local/bin/google_chrome"? That is, will it exhaust the PATH before trying fallbacks replacements or will it check all replacements at once? If I have a program that's not on PATH, what do I do? I can manipulate the PATH environment variable, but am I guaranteed that will work? Are you going to double fork forever to guarantee that environment? Can I build a custom prefix, like p = sh.MagicPrefix(path="/opt/android_devtools/bin"), and have that work like the regular sh module? p.gcc("whatever") ? Even with the existence of a regular gcc in the path? I wonder what happens if you do from sh import *. Does it block execution before continuing? How can I do parallel execution of four subprocesses, and get notified when all four are done? (Seems like this might be a thing for a Future as well, even in the absence of any scheduler or event loop). Are newcomers going to be confused by this? What happens if I try and do something like sh.ls("-l -a")? Will you use the POSIX shell parsing algorithm, pass it to bash, or pass it as one parameter? Will some form of injection attack be mitigated by this design? If you see this magic syntax as your one unique feature, I'd propose that you add it to the subprocess module, and improve the standard subprocess module's interface to cope with the new feature. But I don't see this as a worthwhile thing to have. -1 on the thing.
-- Jasper
On Mon, Oct 22, 2012 at 9:52 AM, Jasper St. Pierre <jstpierre@mecheye.net>wrote:
Hi Jasper, thanks for your questions If I import a non-existant program, will it give me back a function
that will fail or raise an ImportError?
Yes, an exception will be raised How do I run a program with a - in the name? You say you replace -
The full PATH will be exhausted for the exact command, as typed, before any kind of "-" replacement is exercised. There hasn't been much concern about this because most people who want to call commands with special characters prefer to use the Command class (e.g. chrome = Command("/usr/bin/google-chrome")), so the documentation makes a note of this on this issue. If I have a program that's not on PATH, what do I do? I can manipulate
You could manipulate the PATH, but a better way would be to use the Command class, which can take a full path of a command. The returned object can be used just like other commands. I wonder what happens if you do from sh import *. "ImportError: Cannot import * from sh. Please import sh or import programs individually." Commands are lazy resolved on sh anyways, so loading from all would be undefined. Does it block execution before continuing? How can I do parallel
Commands may be run in the background, like this: job1 = sh.tar("-zc", "-f", "archive-name.tar.gz", "/some/directory", _bg=True) job2 = sh.tar(..., _bg=True) job3 = sh.tar(..., _bg=True) job1.wait() job2.wait() job3.wait() Are newcomers going to be confused by this? What happens if I try and
Bash--nor any shell-- is called into play. Sh.py doesn't do any argument parsing either. Arguments are passed into commands exactly as they're sent through sh.py. Newcomers have loved it so far, and after seeing some examples, there's been minimal confusion about how to use it. My thoughts about the magical-ness of sh.py.. I typically don't support very magical or dynamically resolving modules. I can be a little apprehensive of ORMs for this reason... I like to know how my code behaves explicitly. I think clever/magic can be confusing for people and inexplicit, and it's important to know more or less what's going on under the hood. But I also think that sh.py scratches an itch that a less dynamic approach fails to reach. My goal for sh.py has been to make writing system scripts as easy for Python as it is for Bash. People who write Bash scripts do so for a few reasons, one being that a shell script is pretty portable for *nix systems, and also because it's very easy to call programs with arguments, feed them input, and parse their output. But the shortcomings of shell scripts are how obfuscated and unnecessarily difficult they are to accomplish generic programming tasks. This is somewhere where Python excels. But unfortunately, until sh.py, I have not found any tool that lets you call commands nearly as easily as Bash, that didn't rely on Bash. Subprocess is painful. Other modules are extremely verbose. Sh.py, yes, uses a dynamic lookup mechanism (but it should be noted that you don't have to rely on it *at all* if you don't like it), but it does so with a very specific intention, and that is to make Python more suited and commonplace for writing system shell scripts. My push here to see if there is support is because I believe if sh.py could enter the stdlib, and therefore become more ubiquitous on Linux and OSX, that more shell-style scripts could be written in Python, more new users would be comfortable in using Python, and Bash scripts could go the way of the dodo :) Andrew
participants (10)
-
Andrew Moffat -
Antoine Pitrou -
Benjamin Peterson -
Chris Angelico -
Christian Heimes -
Christian Tismer -
Jasper St. Pierre -
M.-A. Lemburg -
Mike Graham -
Vinay Sajip