More friendly access to chmod

Hi everyone, What do you think about enabling a more friendly interface to chmod information in Python? I believe that currently if I want to get chmod information from a file, I need to do this: my_path.stat().st_mode & 0o777 (I'm using `pathlib`.) (If there's a nicer way than this, please let me know.) This sucks. And then the result is then a number, like 511, which you then have to call `oct` on it to get 0o777. I'm not even happy with getting the octal number. For some of us who live and breathe Linux, seeing a number like 0o440 might be crystal-clear, since your mind automatically translates that to the permissions that user/group/others have, but I haven't reached that level. I would really like an object-oriented approach to chmod, like an object which I can ask "Does group have execute permissions?" and say "Please add read permissions to everyone" etc. Just because Linux speaks in code doesn't mean that we need to. And of course, I'd want that on the `pathlib` module so I could do it all on the path object without referencing another module. What do you think? Ram.

On Sat, Jan 09, 2016 at 05:13:38PM +0200, Ram Rachum wrote:
I think that would make an awesome tool added to your own personal toolbox. Once you are satisfied that it works well, then it would be really good to realise it to the public as a third-party library or recipe on ActiveState or similar. And then we can talk about whether or not it belongs in the stdlib.
And of course, I'd want that on the `pathlib` module so I could do it all on the path object without referencing another module.
What's wrong with referencing other modules? -- Steve

On Sat, Jan 9, 2016 at 7:41 AM, Ram Rachum <ram@rachum.com> wrote:
I often prefer OO structure as well, but you can get that by subclassing Path -- it doesn't need to be in the stdlib. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Sun, Jan 10, 2016 at 2:13 AM, Ram Rachum <ram@rachum.com> wrote:
Have you looked at the 'stat' module? At very least, you can ask questions like "Does group have execute permissions?" like this: my_path.stat().st_mode & stat.S_IXGRP You can also get a printable rwxrwxrwx with: stat.filemode(my_path.stat().st_mode) ChrisA

On Sat, Jan 9, 2016 at 5:59 PM, Chris Angelico <rosuav@gmail.com> wrote:
You can also get a printable rwxrwxrwx with:
stat.filemode(my_path.stat().st_mode)
Thanks for the reference. Personally I think that `my_path.stat().st_mode & stat.S_IXGRP` is not human-readable enough. I'll work on a nicer API. Probably this for the same action you described: 'x' in my_path.chmod()['g']

On Sun, Jan 10, 2016 at 3:06 AM, Ram Rachum <ram@rachum.com> wrote:
Okay. I'm not sure how popular that'll be, but sure. As an alternative API, you could have it return a tuple of permission strings, which you'd use thus: 'gx' in my_path.mode() # Group eXecute permission is set But scratch your own itch, and don't give in to the armchair advisers. ChrisA

I think it's a pretty common itch! Have you seen the boltons <https://pypi.python.org/pypi/boltons> implementation? http://boltons.readthedocs.org/en/latest/fileutils.html#file-permissions Mahmoud github.com/mahmoud On Sat, Jan 9, 2016 at 8:11 AM, Chris Angelico <rosuav@gmail.com> wrote:

On Sun, Jan 10, 2016 at 3:51 AM, Mahmoud Hashemi <mahmoud@hatnote.com> wrote:
I think it's a pretty common itch! Have you seen the boltons implementation? http://boltons.readthedocs.org/en/latest/fileutils.html#file-permissions
Yes it is, and no I haven't; everyone has a slightly different idea of what makes a good API, and that's why I put that caveat onto my suggestion. You can't make everyone happy, and APIs should not be designed by committee :) ChrisA

On 10 January 2016 at 08:19, Chris Angelico <rosuav@gmail.com> wrote:
In the context of Python as a cross-platform language, it's also important to remember that POSIX-style user/group/other permissions are only one form of file level access control - depending on your filesystem and OS, there will be a range of others. That significantly reduces the motivation to try to provide a platform independent abstraction for an inherently platform specific concept (at least in the standard library - PyPI is a different matter). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Jan 11, 2016, at 02:23, Nick Coghlan <ncoghlan@gmail.com> wrote:
Well, not a _huge_ range; as far as I know, the only things you're ever likely to run into besides POSIX permissions or a simple read-only flag are ACLs*. But that's still enough of a range to worry about... * Yes, NT and POSIX ACLs aren't quite identical, and the POSIX standard was never completed and there are some minor differences between the Linux and BSD implementations, and OS X confused things by using the NT design with a POSIX-ish API and completely unique tools, so using ACLs portably isn't trivial. But, except for the problem of representing ACLs for users who don't exist on the system, they're pretty much equivalent for almost anything you care about at the application level.

Hi everyone, I spent some time thinking about this. I come up with a big and impressive API, then figured it's overkill, shelved it and made a simpler one :) Here's my new preferred API. Assume that `path` is a `pathlib.Path` object. Checking the chmod of the file: int(path.chmod) # Get an int 393 which in octal is 0o611 oct(path.chmod) # Get a string '0o611' str(path.chmod) # Get a string 'rw-r--r--' repr(path.chmod) # Get a string '<Chmod: rw-r--r-- / 0o611> Modifying the chmod of the file: path.chmod(0o611) # Set chmod to 0o611 (for backward compatibility) path.chmod = 0o611 # Set chmod to 0o611 path.chmod = 393 # Set chmod to 0o611, which is 393 in decimal path.chmod = other_path.chmod # Set chmod to be the same as that of some other file path.chmod = 'rw-r--r--' # Set chmod to 0o611 path.chmod += '--x--x--x' # Add execute permission to everyone path.chmod -= '----rwx' # Remove all permissions from others I've chosen += and -=, despite the fact they're not set operations, because Python doesn't have __inand__. On an unrelated note, maybe we should have __inand__? (I mean x ^~= y) What do you think? On Sat, Jan 9, 2016 at 6:11 PM, Chris Angelico <rosuav@gmail.com> wrote:

On Mon, Jan 11, 2016 at 11:02 PM, Ram Rachum <ram@rachum.com> wrote:
The one thing I'd do differently is call it "mode" or "permissions" rather than "chmod" (CHange MODe), and drop the callability. If you're going to do it as property assignment, making that property also be callable feels awkward (plus it'll be a pain to implement). But otherwise, yeah! Looks great! ChrisA

If you are doing it OO and trying to create a human-usable API, then, why the hell to stick with octal and string representations from the 1970's? path.chmod.executable could return a named-tuple-like object, with owner=True, group=False, all=False - and conversely, you could have path.chmod.owner to return (read=True, write=True, execute=True) Ad thus one could simply do: if path.chmod.owner.writable: ... On 11 January 2016 at 10:29, Chris Angelico <rosuav@gmail.com> wrote:

On Mon, Jan 11, 2016 at 11:41 PM, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
Because they are compact and readable, even when you have lots of them in a column. $ ll /tmp total 24 drwx------ 2 rosuav rosuav 4096 Jan 8 05:27 gpg-4K37Xk drwxr-xr-x 2 root root 4096 Jan 8 05:36 hsperfdata_root drwxr-xr-x 2 rosuav rosuav 4096 Jan 11 22:35 hsperfdata_rosuav drwx------ 2 root root 4096 Jan 8 05:26 pulse-PKdhtXMmr18n prwxr-xr-x 1 rosuav rosuav 0 Jan 11 22:26 SciTE.1722.in drwx------ 2 rosuav rosuav 4096 Jan 8 05:27 ssh-Feo5RK7e1TV3 drwx------ 3 root root 4096 Jan 8 05:27 systemd-private-843a33883f7e4c5c8e6ff168f853c415-rtkit-daemon.service-cdt00F You can see at a glance which ones are readable by people other than their owners. That's worth keeping. It doesn't have to be the ONLY way to do things, but it's definitely one that I do not want to lose. ChrisA

On Mon, Jan 11, 2016 at 1:02 PM, Ram Rachum <ram@rachum.com> wrote:
There is only one way...? ;) My proposal (used for some time in few private projects, but extracted as standalone few days ago): https://github.com/msztolcman/fileperms -- Marcin Sztolcman :: http://urzenia.net/ :: http://sztolcman.eu/

On Jan 11, 2016, at 04:02, Ram Rachum <ram@rachum.com> wrote:
I've chosen += and -=, despite the fact they're not set operations, because Python doesn't have __inand__.
For a property that acts like a number, and presumably is implemented as a subclass of int, this seems like a horribly confusing idea.
On an unrelated note, maybe we should have __inand__? (I mean x ^~= y)
First, why would you spell inand that way? x ^ ~y is the exclusive or of x and ~y, which is true for ~x and ~y and for x and y. That's completely different from nand, which is true for ~x and ~y, ~x and y, and x and y, but not x and ~y. And neither is what you want, which is true only for x and ~y, which you can easily write as x & ~y. Second, why do you think you need an i-operation for a combined operator? x &= ~y does the same thing you'd expect from x &~= y. And why do you think you need an overridable i-operator in the first place? If you call x |= y and x.__ior__ doesn't exist, it just compiles to the same as x = x | y. And, unless x is mutable (which would be very surprising for something that acts like an int), that's actually the way you want it to be interpreted anyway. All of this implies that adding the 70s bitwise operator syntax for dealing with permissions doesn't help with concise but readable code so much as encourage people who don't actually understand bitwise operations to write things that aren't correct or to misread other people's code. What's wrong with just spelling it "clear"? Or, better, as attribute access ("p.chmod.executable = False" or "p.chmod.group.executable = False") or actual set operations with sets of enums instead of integers? The other advantage of using named operations is that it lets you write things that are useful but can't be expressed in a single bitwise operation. For example, "p.chmod.group = q.chmod.owner" is a lot simpler than "p.chmod = p.chmod & ~0o070 | (q.chmod >> 3) & 0o070". Meanwhile, why are you calling the mode "chmod", which is an abbreviation for "change mode"? That's sort of readable but still weird for the cases when you're modifying it, but completely confusing for cases when you're just reading it off. Have you looked at the existing alternatives on PyPI? If so, why isn't one of them good enough? And meanwhile, why not just put your library on PyPI and see if others take it up and start using it? Is there a reason this has to be in the stdlib (and only available on 3.6+) to be usable?

On Jan 11, 2016, at 08:53, Chris Angelico <rosuav@gmail.com> wrote:
If you read his proposal, he wants oct(path.chmod) to work. That doesn't work on types with __int__. Of course it does work on types with __index__, but that's because the whole point of __index__ is to allow your type to act like an actual int everywhere that Python expects an int, rather than just something coercible to int. The point of PEP 357 was to allow numpy.int64 to act as close to a subtype of int as possible without actually being a subtype. It would be very surprising for, say, IntEnum (which subclasses int), or numpy.int64 (which uses __index__), to offer an __add__ method that actually did an or instead of an add. It will be just as surprising here. And the fact that he wants to make it possible (in fact, _encouraged_) to directly assign an int to the property makes it even more confusing. For example, "p.chmod = q + 0o010" does one thing if q is an integer, and another thing if it's the chmod of another path object. (Of course he also wants to be able to assign a string, but that's not a problem; if you mix up str and int, you get a nice TypeError, as opposed to mixing up int and an int subclass or __index__-using class, where you silently get incorrect behavior.)

On Tue, Jan 12, 2016 at 7:44 AM, Andrew Barnert <abarnert@yahoo.com> wrote:
This is what I get for not actually testing stuff. I thought having __int__ would work for oct. In that case, I would simply recommend dropping that part of the proposal; retrieving the octal representation can be spelled oct(int(x)), or maybe x.octal or x.octal(). This is NOT an integer; it's much closer to a set of bitwise enumeration. ChrisA

Seems like the committee has some designs after all? FilePerms <http://boltons.readthedocs.org/en/latest/fileutils.html#boltons.fileutils.Fi...> is tested, on PyPI, and is even 2/3 compatible. And notice the lack of "chmod" as a noun. ;) Mahmoud github.com/mahmoud On Mon, Jan 11, 2016 at 4:12 PM, Chris Angelico <rosuav@gmail.com> wrote:

On Sat, Jan 09, 2016 at 05:13:38PM +0200, Ram Rachum wrote:
I think that would make an awesome tool added to your own personal toolbox. Once you are satisfied that it works well, then it would be really good to realise it to the public as a third-party library or recipe on ActiveState or similar. And then we can talk about whether or not it belongs in the stdlib.
And of course, I'd want that on the `pathlib` module so I could do it all on the path object without referencing another module.
What's wrong with referencing other modules? -- Steve

On Sat, Jan 9, 2016 at 7:41 AM, Ram Rachum <ram@rachum.com> wrote:
I often prefer OO structure as well, but you can get that by subclassing Path -- it doesn't need to be in the stdlib. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Sun, Jan 10, 2016 at 2:13 AM, Ram Rachum <ram@rachum.com> wrote:
Have you looked at the 'stat' module? At very least, you can ask questions like "Does group have execute permissions?" like this: my_path.stat().st_mode & stat.S_IXGRP You can also get a printable rwxrwxrwx with: stat.filemode(my_path.stat().st_mode) ChrisA

On Sat, Jan 9, 2016 at 5:59 PM, Chris Angelico <rosuav@gmail.com> wrote:
You can also get a printable rwxrwxrwx with:
stat.filemode(my_path.stat().st_mode)
Thanks for the reference. Personally I think that `my_path.stat().st_mode & stat.S_IXGRP` is not human-readable enough. I'll work on a nicer API. Probably this for the same action you described: 'x' in my_path.chmod()['g']

On Sun, Jan 10, 2016 at 3:06 AM, Ram Rachum <ram@rachum.com> wrote:
Okay. I'm not sure how popular that'll be, but sure. As an alternative API, you could have it return a tuple of permission strings, which you'd use thus: 'gx' in my_path.mode() # Group eXecute permission is set But scratch your own itch, and don't give in to the armchair advisers. ChrisA

I think it's a pretty common itch! Have you seen the boltons <https://pypi.python.org/pypi/boltons> implementation? http://boltons.readthedocs.org/en/latest/fileutils.html#file-permissions Mahmoud github.com/mahmoud On Sat, Jan 9, 2016 at 8:11 AM, Chris Angelico <rosuav@gmail.com> wrote:

On Sun, Jan 10, 2016 at 3:51 AM, Mahmoud Hashemi <mahmoud@hatnote.com> wrote:
I think it's a pretty common itch! Have you seen the boltons implementation? http://boltons.readthedocs.org/en/latest/fileutils.html#file-permissions
Yes it is, and no I haven't; everyone has a slightly different idea of what makes a good API, and that's why I put that caveat onto my suggestion. You can't make everyone happy, and APIs should not be designed by committee :) ChrisA

On 10 January 2016 at 08:19, Chris Angelico <rosuav@gmail.com> wrote:
In the context of Python as a cross-platform language, it's also important to remember that POSIX-style user/group/other permissions are only one form of file level access control - depending on your filesystem and OS, there will be a range of others. That significantly reduces the motivation to try to provide a platform independent abstraction for an inherently platform specific concept (at least in the standard library - PyPI is a different matter). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Jan 11, 2016, at 02:23, Nick Coghlan <ncoghlan@gmail.com> wrote:
Well, not a _huge_ range; as far as I know, the only things you're ever likely to run into besides POSIX permissions or a simple read-only flag are ACLs*. But that's still enough of a range to worry about... * Yes, NT and POSIX ACLs aren't quite identical, and the POSIX standard was never completed and there are some minor differences between the Linux and BSD implementations, and OS X confused things by using the NT design with a POSIX-ish API and completely unique tools, so using ACLs portably isn't trivial. But, except for the problem of representing ACLs for users who don't exist on the system, they're pretty much equivalent for almost anything you care about at the application level.

Hi everyone, I spent some time thinking about this. I come up with a big and impressive API, then figured it's overkill, shelved it and made a simpler one :) Here's my new preferred API. Assume that `path` is a `pathlib.Path` object. Checking the chmod of the file: int(path.chmod) # Get an int 393 which in octal is 0o611 oct(path.chmod) # Get a string '0o611' str(path.chmod) # Get a string 'rw-r--r--' repr(path.chmod) # Get a string '<Chmod: rw-r--r-- / 0o611> Modifying the chmod of the file: path.chmod(0o611) # Set chmod to 0o611 (for backward compatibility) path.chmod = 0o611 # Set chmod to 0o611 path.chmod = 393 # Set chmod to 0o611, which is 393 in decimal path.chmod = other_path.chmod # Set chmod to be the same as that of some other file path.chmod = 'rw-r--r--' # Set chmod to 0o611 path.chmod += '--x--x--x' # Add execute permission to everyone path.chmod -= '----rwx' # Remove all permissions from others I've chosen += and -=, despite the fact they're not set operations, because Python doesn't have __inand__. On an unrelated note, maybe we should have __inand__? (I mean x ^~= y) What do you think? On Sat, Jan 9, 2016 at 6:11 PM, Chris Angelico <rosuav@gmail.com> wrote:

On Mon, Jan 11, 2016 at 11:02 PM, Ram Rachum <ram@rachum.com> wrote:
The one thing I'd do differently is call it "mode" or "permissions" rather than "chmod" (CHange MODe), and drop the callability. If you're going to do it as property assignment, making that property also be callable feels awkward (plus it'll be a pain to implement). But otherwise, yeah! Looks great! ChrisA

If you are doing it OO and trying to create a human-usable API, then, why the hell to stick with octal and string representations from the 1970's? path.chmod.executable could return a named-tuple-like object, with owner=True, group=False, all=False - and conversely, you could have path.chmod.owner to return (read=True, write=True, execute=True) Ad thus one could simply do: if path.chmod.owner.writable: ... On 11 January 2016 at 10:29, Chris Angelico <rosuav@gmail.com> wrote:

On Mon, Jan 11, 2016 at 11:41 PM, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
Because they are compact and readable, even when you have lots of them in a column. $ ll /tmp total 24 drwx------ 2 rosuav rosuav 4096 Jan 8 05:27 gpg-4K37Xk drwxr-xr-x 2 root root 4096 Jan 8 05:36 hsperfdata_root drwxr-xr-x 2 rosuav rosuav 4096 Jan 11 22:35 hsperfdata_rosuav drwx------ 2 root root 4096 Jan 8 05:26 pulse-PKdhtXMmr18n prwxr-xr-x 1 rosuav rosuav 0 Jan 11 22:26 SciTE.1722.in drwx------ 2 rosuav rosuav 4096 Jan 8 05:27 ssh-Feo5RK7e1TV3 drwx------ 3 root root 4096 Jan 8 05:27 systemd-private-843a33883f7e4c5c8e6ff168f853c415-rtkit-daemon.service-cdt00F You can see at a glance which ones are readable by people other than their owners. That's worth keeping. It doesn't have to be the ONLY way to do things, but it's definitely one that I do not want to lose. ChrisA

On Mon, Jan 11, 2016 at 1:02 PM, Ram Rachum <ram@rachum.com> wrote:
There is only one way...? ;) My proposal (used for some time in few private projects, but extracted as standalone few days ago): https://github.com/msztolcman/fileperms -- Marcin Sztolcman :: http://urzenia.net/ :: http://sztolcman.eu/

On Jan 11, 2016, at 04:02, Ram Rachum <ram@rachum.com> wrote:
I've chosen += and -=, despite the fact they're not set operations, because Python doesn't have __inand__.
For a property that acts like a number, and presumably is implemented as a subclass of int, this seems like a horribly confusing idea.
On an unrelated note, maybe we should have __inand__? (I mean x ^~= y)
First, why would you spell inand that way? x ^ ~y is the exclusive or of x and ~y, which is true for ~x and ~y and for x and y. That's completely different from nand, which is true for ~x and ~y, ~x and y, and x and y, but not x and ~y. And neither is what you want, which is true only for x and ~y, which you can easily write as x & ~y. Second, why do you think you need an i-operation for a combined operator? x &= ~y does the same thing you'd expect from x &~= y. And why do you think you need an overridable i-operator in the first place? If you call x |= y and x.__ior__ doesn't exist, it just compiles to the same as x = x | y. And, unless x is mutable (which would be very surprising for something that acts like an int), that's actually the way you want it to be interpreted anyway. All of this implies that adding the 70s bitwise operator syntax for dealing with permissions doesn't help with concise but readable code so much as encourage people who don't actually understand bitwise operations to write things that aren't correct or to misread other people's code. What's wrong with just spelling it "clear"? Or, better, as attribute access ("p.chmod.executable = False" or "p.chmod.group.executable = False") or actual set operations with sets of enums instead of integers? The other advantage of using named operations is that it lets you write things that are useful but can't be expressed in a single bitwise operation. For example, "p.chmod.group = q.chmod.owner" is a lot simpler than "p.chmod = p.chmod & ~0o070 | (q.chmod >> 3) & 0o070". Meanwhile, why are you calling the mode "chmod", which is an abbreviation for "change mode"? That's sort of readable but still weird for the cases when you're modifying it, but completely confusing for cases when you're just reading it off. Have you looked at the existing alternatives on PyPI? If so, why isn't one of them good enough? And meanwhile, why not just put your library on PyPI and see if others take it up and start using it? Is there a reason this has to be in the stdlib (and only available on 3.6+) to be usable?

On Jan 11, 2016, at 08:53, Chris Angelico <rosuav@gmail.com> wrote:
If you read his proposal, he wants oct(path.chmod) to work. That doesn't work on types with __int__. Of course it does work on types with __index__, but that's because the whole point of __index__ is to allow your type to act like an actual int everywhere that Python expects an int, rather than just something coercible to int. The point of PEP 357 was to allow numpy.int64 to act as close to a subtype of int as possible without actually being a subtype. It would be very surprising for, say, IntEnum (which subclasses int), or numpy.int64 (which uses __index__), to offer an __add__ method that actually did an or instead of an add. It will be just as surprising here. And the fact that he wants to make it possible (in fact, _encouraged_) to directly assign an int to the property makes it even more confusing. For example, "p.chmod = q + 0o010" does one thing if q is an integer, and another thing if it's the chmod of another path object. (Of course he also wants to be able to assign a string, but that's not a problem; if you mix up str and int, you get a nice TypeError, as opposed to mixing up int and an int subclass or __index__-using class, where you silently get incorrect behavior.)

On Tue, Jan 12, 2016 at 7:44 AM, Andrew Barnert <abarnert@yahoo.com> wrote:
This is what I get for not actually testing stuff. I thought having __int__ would work for oct. In that case, I would simply recommend dropping that part of the proposal; retrieving the octal representation can be spelled oct(int(x)), or maybe x.octal or x.octal(). This is NOT an integer; it's much closer to a set of bitwise enumeration. ChrisA

Seems like the committee has some designs after all? FilePerms <http://boltons.readthedocs.org/en/latest/fileutils.html#boltons.fileutils.Fi...> is tested, on PyPI, and is even 2/3 compatible. And notice the lack of "chmod" as a noun. ;) Mahmoud github.com/mahmoud On Mon, Jan 11, 2016 at 4:12 PM, Chris Angelico <rosuav@gmail.com> wrote:
participants (9)
-
Andrew Barnert
-
Chris Angelico
-
Chris Barker
-
Joao S. O. Bueno
-
Mahmoud Hashemi
-
Marcin Sztolcman
-
Nick Coghlan
-
Ram Rachum
-
Steven D'Aprano