On Sun, Jul 18, 2010 at 11:30 AM, Tim Golden <mail@timgolden.me.uk> wrote:

On 17/07/2010 11:03 PM, Peng Yu wrote:

...

I don't see that there is a function in the library that mimic the behavior of 'mkdir -p'. If 'makedirs' is used, it will generate an error if the file already exists. There are some functions available on the website to close the gap. But I'd prefer this is in the library. Is there any plan to add a function that mimic the behavior of  'mkdir -p'?

If I were you I'd ping Tarek who's taken ownership of the shutil module. It would go there if it went anywhere.

That said, it's not clear just how far the stdlib should go to mimic every switch and option of shell commands...

Well, it really should have behaved like mkdir -p in the first place. Eric Raymond submitted the code ages ago, and at the time I wasn't familiar with mkdir -p, otherwise, I would have made him change it. I believe every single use of os.mkdirs() I've made goes something like this: if not os.path.exists(x): os.mkdirs(x) or perhaps like this: try: os.mkdirs(x) except os.error: pass (Though the latter masks a bunch of other rare but possible failure modes -- often this is in throw-away code though where it doesn't matter or the next line will raise an exception that's good enough to diagnose the situation anyway.) -- --Guido van Rossum (python.org/~guido)

Agree. Through searching for existing issues, I found there isn't any such request. And I will submit one. The '-p' option of shell's mkdir has tow functions: 1, create parents directories if not exists, 2, suppress "File exists" error if the target has already exists. What we need is the second function. So I think both os.mkdir() and os.makedirs() should add a keyword argument to suppress the "OSError: [Errno 17] File exists". So we can remove many "try...except..." codes surround os.mkdir() and os.makedirs() as Greg said, "The operation of 'make sure this directory and all its parents exist' is very commonly required" -- Ray Allen Best wishes!

Ray Allen wrote:

I think both os.mkdir() and os.makedirs() should add a keyword argument to suppress the "OSError: [Errno 17] File exists".

This could be seen as violating the "no constant arguments" guideline. Maybe separate function would be better? -- Greg

On Tue, 20 Jul 2010 06:49:44 pm 岳帅杰 wrote:

Sorry, I don't know what is the "no constant arguments" guideline refers to. Could you give me some more explanation?

It refers to the guideline that you shouldn't have a single function with two (or more) different behaviour and an argument that does nothing but select between them. Here is a toy example: def change_case(s, make_upper=True): if make_upper: return s.upper() else: return s.lower()

By the way, I feel adding separate functions is not quiet worthy for such a function.

I tend to agree. Perhaps all we need is a recipe in the docs: try: os.makedirs(path) except OSError, e: if e.errno != 17: raise I think that's all it takes to get the behaviour wanted without any false negatives, and four lines is short enough to use in-place, or if you prefer it's easy enough to wrap it in a function in your own module. Not everything needs to be a built-in. I'm -0 on adding an argument to os.makedirs, +0 on adding a variant function to os, and +0.5 on adding the variant to the shutil module. -- Steven D'Aprano

On 20/07/2010 14:43, Nick Coghlan wrote:

On Tue, Jul 20, 2010 at 11:09 PM, Steven D'Aprano<steve@pearwood.info> wrote:

...

I'm -0 on adding an argument to os.makedirs, +0 on adding a variant function to os, and +0.5 on adding the variant to the shutil module.

shutil seems like the place for it to me. The subtlety of getting the error suppression correct makes it worth implementing this once and for all I think.

Really - so we have two almost identical functions with slightly different behaviour in different standard library modules? Aren't the Python APIs for working with files and paths confused enough already. That seems crazy to me and a switch on os.makedirs much more sensible. Michael

Cheers, Nick.

-- http://www.ironpythoninaction.com/

Michael Foord <fuzzyman@voidspace.org.uk> wrote:

On 20/07/2010 14:43, Nick Coghlan wrote:

...

On Tue, Jul 20, 2010 at 11:09 PM, Steven D'Aprano<steve@pearwood.info> wrote:

...

I'm -0 on adding an argument to os.makedirs, +0 on adding a variant function to os, and +0.5 on adding the variant to the shutil module.

shutil seems like the place for it to me. The subtlety of getting the error suppression correct makes it worth implementing this once and for all I think.

Really - so we have two almost identical functions with slightly different behaviour in different standard library modules? Aren't the Python APIs for working with files and paths confused enough already. That seems crazy to me and a switch on os.makedirs much more sensible.

Yep, me too. Bill

Michael

...

Cheers, Nick.

-- http://www.ironpythoninaction.com/

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com

On Tue, Jul 20, 2010 at 11:35 PM, Cameron Simpson <cs@zip.com.au> wrote:

On 20Jul2010 17:49, Michael Foord <fuzzyman@voidspace.org.uk> wrote: | On 20/07/2010 14:43, Nick Coghlan wrote: | >On Tue, Jul 20, 2010 at 11:09 PM, Steven D'Aprano<steve@pearwood.info>  wrote: | >>I'm -0 on adding an argument to os.makedirs, +0 on adding a variant | >>function to os, and +0.5 on adding the variant to the shutil module. | >shutil seems like the place for it to me. The subtlety of getting the | >error suppression correct makes it worth implementing this once and | >for all I think. | | Really - so we have two almost identical functions with slightly | different behaviour in different standard library modules?

Three if you count os.mkdir itself as well.

| Aren't | the Python APIs for working with files and paths confused enough | already. That seems crazy to me and a switch on os.makedirs much | more sensible.

+1 from me. Hasn't the BDFL blessed this particular suggestion too?

Now I have. :-) -1 on a new function (despite the constant-argument guideline) and +1 on a flag. If it weren't for backwards compatibility I'd just change os.makedirs() to act like mkdir -p period, but the last opportunity we had for that was Python 3.0. -- --Guido van Rossum (python.org/~guido)

I'd go with putting it in shutil. We could also add a function there that wraps up the recipe in issue 9311 to work around the quirks of os.access on FreeBSD (and possibly other platforms). -- R. David Murray www.bitdance.com

On 07/27/2010 06:18 PM, Alexander Belopolsky wrote:

On Tue, Jul 20, 2010 at 10:20 AM, R. David Murray<rdmurray@bitdance.com> wrote:

...

I'd go with putting it in shutil.

+1

I would also call it shutil.mktree which will go well with shutil.rmtree next to it.

Note that mktree is not analogous to rmtree - while rmtree removes a directory tree beneath a specified directory, mktree would only create a single "branch", not an entire tree. I'd imagine a mktree function to accept a data structure describing the tree to be created. If you're going for a short name distinctive from mkdir, I propose mksubdirs.

On 7/28/2010 4:42 AM, Ray Allen wrote:

I believe, in design purpose, the os.mkdir() is to match the system call "mkdir()" exactly, the os.makedirs() is a "Super-mkdir", it provides extra convenience for using when we want to create directories. This is the case makedirs() should deal with.

After discussion with Guido, the patch on the tracker leaves makedir alone and catches the exception, or not, in makedirs. -- Terry Jan Reedy

Hrvoje Niksic wrote:

mktree would only create a single "branch", not an entire tree.

Maybe mkbranch, then? -- Greg

On Tue, Jul 20, 2010 at 9:09 AM, Steven D'Aprano <steve@pearwood.info> wrote:

It refers to the guideline that you shouldn't have a single function with two (or more) different behaviour and an argument that does nothing but select between them.

In particular, when that argument is almost never given a variable value, but is specified using a constant at the call site.   -Fred -- Fred L. Drake, Jr.    <fdrake at gmail.com> "A storm broke loose in my mind."  --Albert Einstein

On 07/20/2010 11:47 AM, Ron Adam wrote:

On 07/20/2010 10:43 AM, Fred Drake wrote:

...

On Tue, Jul 20, 2010 at 9:09 AM, Steven D'Aprano<steve@pearwood.info> wrote:

...

It refers to the guideline that you shouldn't have a single function with two (or more) different behaviour and an argument that does nothing but select between them.

In particular, when that argument is almost never given a variable value, but is specified using a constant at the call site.

I was thinking it should have been two functions, but I realized there is more subtleties involved than simply just reusing a directory that already exists.

One possibility might be...

mkdir(path [, allow=None, mode=0777])

Where None can be replaced with one or more of the following.

'exists' dir can already exist (with same permissions as mode) 'case' dir case can be different. (Windows) 'files' dir can have files in it.

Add 'path', Complete path as -p option does.

... or a string containing the initials.

It doesn't fall under the single constant rule if done this way.

Ron

On 07/20/2010 12:00 PM, Fred Drake wrote:

On Tue, Jul 20, 2010 at 12:47 PM, Ron Adam<rrr@ronadam.com> wrote:

...

It doesn't fall under the single constant rule if done this way.

If the value for 'allow' were almost always given as a constant, this would be an argument for three functions instead of one.

The guideline has little to do with the type of the value, but the number of possible values ("small") and whether they're normally given as constants in the code.

If there's more than one, and combinations then to vary, then keeping them as args makes sense.

Also, if we don't know what we want the functionality to be, as you suggest, then worry about that is premature. :-) Let's decide on the required functionality first.

That makes sense. :-) Another things that comes to mind, is it may make sense to choose either strict, and have args to *allow* different cases, or to choose lenient, and have args to *restrict* different cases. That keeps it somewhat less confusing, and doesn't require memorization to remember what the mixed mode default might be. (Unless we use a single combination 99% of the time, then that probably *should* be the default.) Ron

I don't see what would be so bad about adding a new

function for this. Think of it as correcting the mistake of not making makedirs() behave this way from the beginning. If you want to add a new function, then what its name should be? I guess it should be too similar as existing ones. It's confusing for using because they have similar names and do the same major things except exception. For users it's easier to remember one function than tow. If we want to correct a mistake, it's better not to import another mistake. I don't like writing code that depends on particular errno values, because I don't trust it to work cross- platform. I think in this case, the errno is generate by c standard library, which can be seen as cross-platform.

-- Greg

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ysj.ray%2Bpython-dev%40gma...

-- Ray Allen Best wishes!

Greg Ewing <greg.ewing@canterbury.ac.nz> writes:

Ray Allen wrote:

...

I think in this case, the errno is generate by c standard library, which can be seen as cross-platform.

But I'm never sure how standard the actual error numbers are, though.

You can use them by name, and in fact I strongly recommend it: import os import errno try: os.makedirs(path) except OSError, exc: if exc.errno != errno.EEXIST: raise

I tend to think of them as coming from Unix-land, and thus fair game for getting screwed around with on Windows. Am I worrying too much?

There are some that differ between different OSen, true. Using them by name avoids dealing with *different* numbers (since the ‘errno’ module's attributes will use the same name for semantically the same error), leaving only the problem of errors that are *missing* on some platforms. EEXIST is common to all of them though, AFAIK. -- \ “Sittin' on the fence, that's a dangerous course / You can even | `\ catch a bullet from the peace-keeping force” —Dire Straits, | _o__) _Once Upon A Time In The West_ | Ben Finney

岳帅杰 wrote:

Sorry, I don't know what is the "no constant arguments" guideline refers to. Could you give me some more explanation?

It's a rule of thumb that Guido says he uses when designing an API. If in the majority of use cases for a proposed function, one of its arguments would always be a constant that chooses between two or more different behaviours, then it would probably be better to have separate functions for each of the behaviours. -- Greg