Working with Path objects: p-strings?
So, let's start a new thread about how to deal with pathlib.Path objects, involving how/whether to convert between str and Path. On Fri, Mar 25, 2016 at 7:14 PM, Sven R. Kunze <srkunze@mail.de> wrote:
On 24.03.2016 22:06, Koos Zevenhoven wrote:
Or even better, that you could do p"filename.txt", which would give you a Path string object. Has this been discussed?
Interesting. I had this thought half a year ago. Quite recently I mentioned this to Andrew in a private conversation.
Yeah, I first thought about this at some point last summer when I had written the little np package for creating numpy arrays as np[3,4,5] (which I am planning to polish and maybe advertise), probably also inspired by the f-string discussions on this list. And now that you asked me about dealing with Paths... Anyway, as you say, if different people have thought about this independently, maybe its worth discussing.
p'/etc/hosts' would make a perfect path which subclassed from str.
Yeah, and regarding the "`to_file()` method for strings" thread, one could simply do p"targetfile.txt".write_text(string_to_write) using the write_text method introduced to Path objects in 3.5. [And of course there's also read_text(), write_bytes(...) and read_bytes()] One downside to p-strings like this is that they don't help you when you have the path in a plain str variable (i.e. not a literal), which has led me to think Path perhaps should be in builtins. But one thing that just occured to me now is that, for instance in interactive sessions, you could do p"" / path_as_str instead of `import pathlib; pathlib.Path(path_as_str)`. Not sure if this is the most beautiful thing ever seen, but it would work for quick scripting at least.
The p-string idea has not been discussed in the PEP I think. The subclassing thing however was and I think its resolution was a mistake. The explanation, at least from my point of view, is a bit weird.
Yeah, having Path subclass str would immediately solve a lot of compatibility issues. One question is how large a part of those issues would be solved by just making the builtin open(...) and a few places here and there in the stdlib accept Path objects. Maybe there are implementation issues too, although having them lead to bad decisions would be sad. Anyway, I think the question of whether Path should subclass str is partly separate from the p-string concept. After all, b'byte strings' are not subclassed from str either. - Koos
Le 25/03/2016 21:20, Koos Zevenhoven a écrit :
So, let's start a new thread about how to deal with pathlib.Path objects, involving how/whether to convert between str and Path.
On Fri, Mar 25, 2016 at 7:14 PM, Sven R. Kunze <srkunze@mail.de> wrote:
On 24.03.2016 22:06, Koos Zevenhoven wrote:
Or even better, that you could do p"filename.txt", which would give you a Path string object. Has this been discussed?
Interesting. I had this thought half a year ago. Quite recently I mentioned this to Andrew in a private conversation.
Yeah, I first thought about this at some point last summer when I had written the little np package for creating numpy arrays as np[3,4,5] (which I am planning to polish and maybe advertise), probably also inspired by the f-string discussions on this list. And now that you asked me about dealing with Paths...
Anyway, as you say, if different people have thought about this independently, maybe its worth discussing.
p'/etc/hosts' would make a perfect path which subclassed from str.
Yeah, and regarding the "`to_file()` method for strings" thread, one could simply do
p"targetfile.txt".write_text(string_to_write)
It's nice. It remains me of the proposal for odereddict with o{}. There is a limited number of prefix we can use though.
using the write_text method introduced to Path objects in 3.5. [And of course there's also read_text(), write_bytes(...) and read_bytes()]
One downside to p-strings like this is that they don't help you when you have the path in a plain str variable (i.e. not a literal), which has led me to think Path perhaps should be in builtins. But one thing that just occured to me now is that, for instance in interactive sessions, you could do
p"" / path_as_str
instead of `import pathlib; pathlib.Path(path_as_str)`. Not sure if this is the most beautiful thing ever seen, but it would work for quick scripting at least.
The p-string idea has not been discussed in the PEP I think. The subclassing thing however was and I think its resolution was a mistake. The explanation, at least from my point of view, is a bit weird.
Yeah, having Path subclass str would immediately solve a lot of compatibility issues. One question is how large a part of those issues would be solved by just making the builtin open(...) and a few places here and there in the stdlib accept Path objects. Maybe there are implementation issues too, although having them lead to bad decisions would be sad.
Although path.py, which I have been using for years now (and still prefer to pathlib) subclass str and never caused any problem whatsoever. So really, we should pinpoint where it could be an issue and see if this is a frequent problem, because right now, it seems more a decision based on purity than practicality.
Anyway, I think the question of whether Path should subclass str is partly separate from the p-string concept. After all, b'byte strings' are not subclassed from str either.
- Koos _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 25.03.2016 22:56, Michel Desmoulin wrote:
Although path.py, which I have been using for years now (and still prefer to pathlib) subclass str and never caused any problem whatsoever. So really, we should pinpoint where it could be an issue and see if this is a frequent problem, because right now, it seems more a decision based on purity than practicality.
I agree. The PEP says: """ No confusion with builtins ---------------------------------- In this proposal, the path classes do not derive from a builtin type. This contrasts with some other Path class proposals which were derived from str . They also do not pretend to implement the sequence protocol: if you want a path to act as a sequence, you have to lookup a dedicated attribute (the parts attribute). Not behaving like one of the basic builtin types also minimizes the potential for confusion if a path is combined by accident with genuine builtin types. """" I have to admit I cannot follow these statements but they should have appeared to be necessary back then. As experience shows the PyPI module fared far better here. I am great a fan of theory over practice as what has been proven in theory cannot be proven wrong in practice. However, this only holds if we talk about hard proof. For soft things like "human interaction", "mutual understanding" or "potential for confusion", only the practice of many many people can "prove" what's useful, what's "practical". Best, Sven
Le 29/03/2016 09:50, Sven R. Kunze a écrit :
On 25.03.2016 22:56, Michel Desmoulin wrote:
Although path.py, which I have been using for years now (and still prefer to pathlib) subclass str and never caused any problem whatsoever. So really, we should pinpoint where it could be an issue and see if this is a frequent problem, because right now, it seems more a decision based on purity than practicality.
I agree. The PEP says:
""" No confusion with builtins ---------------------------------- In this proposal, the path classes do not derive from a builtin type. This contrasts with some other Path class proposals which were derived from str . They also do not pretend to implement the sequence protocol: if you want a path to act as a sequence, you have to lookup a dedicated attribute (the parts attribute).
Not behaving like one of the basic builtin types also minimizes the potential for confusion if a path is combined by accident with genuine builtin types. """"
I have to admit I cannot follow these statements but they should have appeared to be necessary back then. As experience shows the PyPI module fared far better here.
I am great a fan of theory over practice as what has been proven in theory cannot be proven wrong in practice. However, this only holds if we talk about hard proof. For soft things like "human interaction", "mutual understanding" or "potential for confusion", only the practice of many many people can "prove" what's useful, what's "practical".
+1. Can somebody defend, with practical examples, the imperative to refuse to inherit from str ? And weight it against the benefits of it ? We have been doing it for years, and so far it's been really, really nice. I can't recall problem I had because of it ever. I can recall numerous editions of my code because I forgot str() on pathlib.Path.
Best, Sven
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 29 March 2016 at 10:43, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
+1. Can somebody defend, with practical examples, the imperative to refuse to inherit from str ? And weight it against the benefits of it ?
I don't intend to get sucked into the role of "defender of not inheriting", but one point that should probably be considered is that on Unix, it is possible to have paths which are *not* valid Python strings. How would those work in an inherit-from-str environment? I suspect that using surrogateescape is the answer here, but I don't know. Also, has anyone *converted* code from os.path manipulation of strings to a str-subclass path object like path.py? If you write your code from scratch intending to use path.py it's probably easier to avoid objects getting silently downgraded from path to str than if you're retrofitting a path library to legacy code. That's a consideration that is more applicable to the standard library than to a 3rd party module. But it may be that now is the right time to revisit the choice to not inherit from str. I don't know, personally. I'm not even sure if a pathlib.Path that inherited from str is something I'd use more often than I currently do. For open source work, the need to support Python 2 means I can't use pathlib (without adding the backport as a dependency). For personal code, it's just a case of mental inertia... Paul
Le 29/03/2016 12:36, Paul Moore a écrit :
On 29 March 2016 at 10:43, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
+1. Can somebody defend, with practical examples, the imperative to refuse to inherit from str ? And weight it against the benefits of it ?
I don't intend to get sucked into the role of "defender of not inheriting", but one point that should probably be considered is that on Unix, it is possible to have paths which are *not* valid Python strings. How would those work in an inherit-from-str environment? I suspect that using surrogateescape is the answer here, but I don't know.
How does str(path) does it ? Because ultimatly, we pass strings to open() and os.stuff. So we need it as strings anyway.
Also, has anyone *converted* code from os.path manipulation of strings to a str-subclass path object like path.py? If you write your code from scratch intending to use path.py it's probably easier to avoid objects getting silently downgraded from path to str than if you're retrofitting a path library to legacy code. That's a consideration that is more applicable to the standard library than to a 3rd party module.
I do it all the time. I'm a path.py/begin/request/arrow/pytest addict and try to convert coworkers as I meet them, so for path.py I'll usually take a small part of their project with intensive FS manipulations and demonstrate how path.py integrate in it. It's doesn't take a lot to convince them, the one good reason people don't want to is the additional dependency, which I understand perfectly. The only hard part to sell is the need to always pass unicode strings to Path(), because in Python 2.7 it tries to automatically convert b'' to u"" with the usual UnicodeDecodeError pitfalls. It's a good habit anyway, yet people are attached to their shortcuts : it's the same with arrow because of the default to UTC instead of local time. it's the same with pytest which requires python setup.py develop and explicit -s. request and begin are is easier since urllib and argparse are so hard. Like with all changes, you need a little adaptation. But all in all, HTTP requests, FS manipulation, datetime handling, unit tests and parameter parsing are so common tasks it make sense to have tools that gets the job done in a few lines and gets out of the way. You want to concentrate of the real value of your code, not those. Now I was sad when I learned request would never be part of the stdlib, and I'm still sad asyncio doesn't have a good way to do a simple GET with the stdlib, even more with http2 comming our way. But I get it: the web moves was, the stdlib moves slowly. It's not True with FS manipulations, they are are old as it gets, and we can improve on them without fearing to be left behind.
But it may be that now is the right time to revisit the choice to not inherit from str. I don't know, personally. I'm not even sure if a pathlib.Path that inherited from str is something I'd use more often than I currently do. For open source work, the need to support Python 2 means I can't use pathlib (without adding the backport as a dependency). For personal code, it's just a case of mental inertia...
Paul
On Tue, Mar 29, 2016 at 12:57:02PM +0200, Michel Desmoulin wrote:
Le 29/03/2016 12:36, Paul Moore a écrit :
I don't intend to get sucked into the role of "defender of not inheriting", but one point that should probably be considered is that on Unix, it is possible to have paths which are *not* valid Python strings. How would those work in an inherit-from-str environment? I suspect that using surrogateescape is the answer here, but I don't know.
How does str(path) does it ? Because ultimatly, we pass strings to open() and os.stuff. So we need it as strings anyway.
Actually, open and os.* will accept bytes paths, at least on Linux. Unless I'm missing something, pathlib doesn't support bytes filenames. So the question doesn't come up. Any path you can use with pathlib is a valid Python string, because you can only create paths using Python strings. So if I have a Linux file b'\xD8\x01', I can't create a path object to work with it. -- Steve
On 29 March 2016 at 12:24, Sven R. Kunze <srkunze@mail.de> wrote:
On 29.03.2016 13:21, Steven D'Aprano wrote:
So if I have a Linux file b'\xD8\x01', I can't create a path object to work with it.
I actually like this limitation. It forces a cleaner API and cleaner code as it is the one way.
However, it does mean that certain use cases are not supported by pathlib. One of the constant issues with core/stdlib development is having to consider how the design affects people with needs you don't personally have. This one's not such a big problem "pathlib doesn't support non-Unicode filenames" is a simple enough to express limitation. But that doesn't help the poor guy debugging an issue with his filesystem-scanning program where a user has a mangled non-Unicode filename somewhere in the bowels of his profile - does he declare that user's situation is unsupported, or does he abandon pathlib in favour of the old os APIs? (And if the , what's the likelihood
Bah, unexpected keyboard shortcut - sorry for the premature send... On 29 March 2016 at 13:10, Paul Moore <p.f.moore@gmail.com> wrote:
However, it does mean that certain use cases are not supported by pathlib. One of the constant issues with core/stdlib development is having to consider how the design affects people with needs you don't personally have. This one's not such a big problem "pathlib doesn't support non-Unicode filenames" is a simple enough to express limitation. But that doesn't help the poor guy debugging an issue with his filesystem-scanning program where a user has a mangled non-Unicode filename somewhere in the bowels of his profile - does he declare that user's situation is unsupported, or does he abandon pathlib in favour of the old os APIs? (And if the , what's the likelihood
(And either way, what's the likelihood of him ever wanting to use pathlib again?) Paul
Patching your mails: On 29.03.2016 14:10, Paul Moore wrote:
However, it does mean that certain use cases are not supported by pathlib. One of the constant issues with core/stdlib development is having to consider how the design affects people with needs you don't personally have. This one's not such a big problem "pathlib doesn't support non-Unicode filenames" is a simple enough to express limitation. But that doesn't help the poor guy debugging an issue with his filesystem-scanning program where a user has a mangled non-Unicode filename somewhere in the bowels of his profile - does he declare that user's situation is unsupported, or does he abandon pathlib in favour of the old os APIs?
Perfect example. As I said, I've encountered this situation and most likely will encounter it again. We have mostly control over the file names. So, we might be able to fix a problematic file name. The issue you are bringing up (which is perfectly valid) is when you as a developer don't have control over the file name (aka fixing them). In this regard, I might be pragmatic and say: "let's see what happens". If this is a frequent issue, then this is one of those warts pathlib needs to loose in order to become mainstream. But why not open another thread with a solution to this issue as has Koos done with p-strings?
(And either way, what's the likelihood of him ever wanting to use pathlib again?)
You mean like myself? As I wanted to use pathlib but found myself writing awkward code with it? (btw. that was not the path->str issue) That as well needs to be fixed but it's no argument for delaying other improvements, right? Best, Sven
On 29 March 2016 at 13:31, Sven R. Kunze <srkunze@mail.de> wrote:
But why not open another thread with a solution to this issue as has Koos done with p-strings?
Agreed, that's fine. I personally don't have any solutions to offer, so I'll wait to comment on any that do arise (as I've been doing on the p-string proposal).
(And either way, what's the likelihood of him ever wanting to use pathlib again?)
You mean like myself? As I wanted to use pathlib but found myself writing awkward code with it? (btw. that was not the path->str issue)
That as well needs to be fixed but it's no argument for delaying other improvements, right?
Certainly. However, I don't think p-strings are an improvement - I'm not suggesting delaying them because other things need to be fixed, I'm suggesting that the whole proposal shouldn't be implemented because it's not a good idea. If you want custom string-like syntaxes, PEP 501 is where you should be getting involved. Adding a single-purpose custom string prefix is a proposal that *should* be delayed until the fate of a more general proposal of which it's a subset is decided. Because once syntax is added it's nigh-on impossible to get rid of. Paul
On 29.03.2016 15:08, Paul Moore wrote:
On 29 March 2016 at 13:31, Sven R. Kunze <srkunze@mail.de> wrote:
That as well needs to be fixed but it's no argument for delaying other improvements, right? Certainly. However, I don't think p-strings are an improvement - I'm not suggesting delaying them because other things need to be fixed, I'm suggesting that the whole proposal shouldn't be implemented because it's not a good idea.
Alright. Got it. :) p-strings are not the most pressing issue of pathlib I assume. Wrapping an (f-)string into a Path might work for now although I would love see the convenient p-string in some future Python version.
If you want custom string-like syntaxes, PEP 501 is where you should be getting involved. Adding a single-purpose custom string prefix is a proposal that *should* be delayed until the fate of a more general proposal of which it's a subset is decided. Because once syntax is added it's nigh-on impossible to get rid of.
Agreed. First things first. Note on PEP 501. Not sure if I like it (sorry Nick) but to me it feels a bit artificial/impracticable. Its examples seem very awkward to me. This additional pair of parentheses kills it for me. runquery & runcommand should do the wrapping themselves. If done, i-strings look like f-strings. runquery(sql(i"SELECT {column} FROM {table};")) runcommand(sh(i"cat {filename}")) This means for p-string: Path(f'/whereever/{youwant}/{togo}') == p'/whereever/{youwant}/{togo}' So, if Path is powerful enough, I think most people can live with that for now (but those parentheses .... don't we already have the quotation marks ... anyway) ;-) Best, Sven
Le 29/03/2016 14:31, Sven R. Kunze a écrit :
Patching your mails:
On 29.03.2016 14:10, Paul Moore wrote:
However, it does mean that certain use cases are not supported by pathlib. One of the constant issues with core/stdlib development is having to consider how the design affects people with needs you don't personally have. This one's not such a big problem "pathlib doesn't support non-Unicode filenames" is a simple enough to express limitation. But that doesn't help the poor guy debugging an issue with his filesystem-scanning program where a user has a mangled non-Unicode filename somewhere in the bowels of his profile - does he declare that user's situation is unsupported, or does he abandon pathlib in favour of the old os APIs?
Perfect example.
As I said, I've encountered this situation and most likely will encounter it again. We have mostly control over the file names. So, we might be able to fix a problematic file name. The issue you are bringing up (which is perfectly valid) is when you as a developer don't have control over the file name (aka fixing them).
In this regard, I might be pragmatic and say: "let's see what happens". If this is a frequent issue, then this is one of those warts pathlib needs to loose in order to become mainstream.
yes. It's a very rare use case. If you create a program that needs to be handling this, you still got the old API. For the 99% of the rest of the world, you have the modern one. Let's not try to make pathlib handle everything, it's a convenience wrapper, not a silver bullet.
But why not open another thread with a solution to this issue as has Koos done with p-strings?
(And either way, what's the likelihood of him ever wanting to use pathlib again?)
You mean like myself? As I wanted to use pathlib but found myself writing awkward code with it? (btw. that was not the path->str issue)
That as well needs to be fixed but it's no argument for delaying other improvements, right?
Exactly. pathlib needs improvements, and there is plenty of prior art to make suggestions about this, but if it's a central API, it will be on the spotlight to do so.
Best, Sven _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Mar 29, 2016 at 10:21 PM, Steven D'Aprano <steve@pearwood.info> wrote:
Unless I'm missing something, pathlib doesn't support bytes filenames. So the question doesn't come up. Any path you can use with pathlib is a valid Python string, because you can only create paths using Python strings.
So if I have a Linux file b'\xD8\x01', I can't create a path object to work with it.
Actually you can. Here's an annotated interactive session illustrating byte smuggling:
# Create a file using a bytes name ... with open(b'\xD8\x01',"w") as f: f.write("Demo text\n") ... 10 import os, pathlib # By default, you get text strings. ... os.listdir() ['\udcd8\x01'] os.listdir(".") ['\udcd8\x01'] # But you can request byte strings. ... os.listdir(b".") [b'\xd8\x01'] # pathlib works with text strings the same way. ... list(pathlib.Path(".").glob("*")) [PosixPath('\udcd8\x01')] # And you can cast that to str and open it. ... open(str(pathlib.Path("\uDCD8\x01"))).read() 'Demo text\n'
So it looks like the only missing piece of the puzzle is a way to construct a Path from a bytes, which would do the same transformation of \xD8 to \uDCD8. ChrisA
On Tue, Mar 29, 2016, at 07:21, Steven D'Aprano wrote:
Unless I'm missing something, pathlib doesn't support bytes filenames. So the question doesn't come up. Any path you can use with pathlib is a valid Python string, because you can only create paths using Python strings.
So if I have a Linux file b'\xD8\x01', I can't create a path object to work with it.
Any reason you can't use surrogateescape and "\uDCD8\x01"?
offtopic On 29.03.2016 12:57, Michel Desmoulin wrote:
Now I was sad when I learned request would never be part of the stdlib,
Looking at http://docs.python-requests.org/en/master/dev/philosophy/#standard-library Seems to be true. I wonder if it would be possible/necessary for the stdlib to include a set of libraries marked as "featured". Not like "provisional" but more like "You should use this one and be aware that it may upgrade faster than normal stdlib modules". Basically like a "pull module". Just a third-party module pulled from the source and upgraded with each Python version. That would maintain the maintainers flexibility and the increase the availability of these "defacto stdlib" modules. Just wondering. Best, Sven
On Tue, Mar 29, 2016 at 2:22 PM, Sven R. Kunze <srkunze@mail.de> wrote:
offtopic
I wonder if it would be possible/necessary for the stdlib to include a set of libraries marked as "featured". Not like "provisional" but more like "You should use this one and be aware that it may upgrade faster than normal stdlib modules".
Basically like a "pull module". Just a third-party module pulled from the source and upgraded with each Python version. That would maintain the maintainers flexibility and the increase the availability of these "defacto stdlib" modules.
+1. It is a different thing to find a random module on PyPI and depend on it, than to depend on something that is ~officially recommended. Or maybe some kind of package reputation system on PyPI? More threads? :) - Koos
On 29 March 2016 at 16:48, Koos Zevenhoven <k7hoven@gmail.com> wrote:
On Tue, Mar 29, 2016 at 2:22 PM, Sven R. Kunze <srkunze@mail.de> wrote:
offtopic
I wonder if it would be possible/necessary for the stdlib to include a set of libraries marked as "featured". Not like "provisional" but more like "You should use this one and be aware that it may upgrade faster than normal stdlib modules".
Basically like a "pull module". Just a third-party module pulled from the source and upgraded with each Python version. That would maintain the maintainers flexibility and the increase the availability of these "defacto stdlib" modules.
+1.
It is a different thing to find a random module on PyPI and depend on it, than to depend on something that is ~officially recommended. Or maybe some kind of package reputation system on PyPI? More threads? :)
Been discussed a few times previously (look for things like "sumo distribution", "breaking out the stdlib", "recommended packages"). So yes, again, should be a new thread and should also start with a bit of research into previous discussions. Paul
Le 29/03/2016 17:48, Koos Zevenhoven a écrit :
On Tue, Mar 29, 2016 at 2:22 PM, Sven R. Kunze <srkunze@mail.de> wrote:
offtopic
I wonder if it would be possible/necessary for the stdlib to include a set of libraries marked as "featured". Not like "provisional" but more like "You should use this one and be aware that it may upgrade faster than normal stdlib modules".
Basically like a "pull module". Just a third-party module pulled from the source and upgraded with each Python version. That would maintain the maintainers flexibility and the increase the availability of these "defacto stdlib" modules.
+1.
It is a different thing to find a random module on PyPI and depend on it, than to depend on something that is ~officially recommended. Or maybe some kind of package reputation system on PyPI? More threads? :)
Stop growing the stdlib and make Python a distribution is a topic I already read here. It may be time to revive it again. Last time, it was deeply linked with packaging and battery included philosophy.
- Koos _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
In days of yore, before package managers, people used to download source code and read it. Maybe not all of it, but enough to feel not terribly scared when running that code. In modern times, with centralized package repositories and convenient installer tools, we want a better way to know "what’s the right package to use for this task?" It's a "first-world problem" in a sense. There are too many products in my supermarket aisle! On a personal note, I have on occasion spent twenty minutes choosing a toothpaste at Target. If I care enough, I'll take a moment to look at how many downloads have been counted recently, how many issues there are (usually on GitHub), how many contributors, etc. I'll read the docs. I might even poke around in the source. I'll also check Google rankings to see if people are chatting about the module and linking to it. I'm not sure if there's a good centralized solution to this problem, but it's a question many people are asking: How do I know which non-stdlib module to use? Back at Georgia Tech, my professor [0] once told me that the way to get rich is to invent an index. He was referring to Richard Florida's "Creative Class" book and the subsequent "Creativity Index" consulting that Florida provided to various municipalities. People who score high on the index pay you to speak. People who score low on the index pay you to consult. There are a few companies who sell a Python package reputation service, along with some distribution tools. Continuum's Anaconda, Enthought's Canopy, and ActiveState's ActivePython come to mind. There's clearly value in helping people answer this question. [0] http://www.coa.gatech.edu/people/david-sawicki
On Mar 29, 2016, at 1:13 PM, Michel Desmoulin <desmoulinmichel@gmail.com> wrote: Le 29/03/2016 17:48, Koos Zevenhoven a écrit :
On Tue, Mar 29, 2016 at 2:22 PM, Sven R. Kunze <srkunze@mail.de> wrote:
offtopic
I wonder if it would be possible/necessary for the stdlib to include a set of libraries marked as "featured". Not like "provisional" but more like "You should use this one and be aware that it may upgrade faster than normal stdlib modules".
Basically like a "pull module". Just a third-party module pulled from the source and upgraded with each Python version. That would maintain the maintainers flexibility and the increase the availability of these "defacto stdlib" modules.
+1.
It is a different thing to find a random module on PyPI and depend on it, than to depend on something that is ~officially recommended. Or maybe some kind of package reputation system on PyPI? More threads? :)
Stop growing the stdlib and make Python a distribution is a topic I already read here. It may be time to revive it again.
Last time, it was deeply linked with packaging and battery included philosophy.
On 30 March 2016 at 06:51, Michael Selik <mike@selik.org> wrote:
In days of yore, before package managers, people used to download source code and read it. Maybe not all of it, but enough to feel not terribly scared when running that code. In modern times, with centralized package repositories and convenient installer tools, we want a better way to know "what’s the right package to use for this task?" It's a "first-world problem" in a sense. There are too many products in my supermarket aisle! On a personal note, I have on occasion spent twenty minutes choosing a toothpaste at Target.
If I care enough, I'll take a moment to look at how many downloads have been counted recently, how many issues there are (usually on GitHub), how many contributors, etc. I'll read the docs. I might even poke around in the source. I'll also check Google rankings to see if people are chatting about the module and linking to it.
I'm not sure if there's a good centralized solution to this problem, but it's a question many people are asking: How do I know which non-stdlib module to use?
Back at Georgia Tech, my professor [0] once told me that the way to get rich is to invent an index. He was referring to Richard Florida's "Creative Class" book and the subsequent "Creativity Index" consulting that Florida provided to various municipalities. People who score high on the index pay you to speak. People who score low on the index pay you to consult.
There are a few companies who sell a Python package reputation service, along with some distribution tools. Continuum's Anaconda, Enthought's Canopy, and ActiveState's ActivePython come to mind. There's clearly value in helping people answer this question.
And so do Linux distributions. There's a reason there are so many of the latter: "good" choices depend a great deal on what you're doing, which means there's no point in trying to come up with the "one true reputation system" for software components. djangopackages.com does a relatively good job for the Django ecosystem, but the simple fact of using Django puts you far enough down the path towards solving a particular kind of problem (web service design with a rich user permission system backed by a relational database) that a community driven rating system can work. By contrast, I think Python itself covers too many domains for a common rating system to be feasible - "good for education" is not the same as "good for sysadmin tasks" is not the same as "good for data analysis" is not the same as "good for network service development", etc. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 4/1/2016 09:07, Nick Coghlan wrote:
And so do Linux distributions.
There's a reason there are so many of the latter: "good" choices depend a great deal on what you're doing, which means there's no point in trying to come up with the "one true reputation system" for software components. djangopackages.com does a relatively good job for the Django ecosystem, but the simple fact of using Django puts you far enough down the path towards solving a particular kind of problem (web service design with a rich user permission system backed by a relational database) that a community driven rating system can work.
By contrast, I think Python itself covers too many domains for a common rating system to be feasible - "good for education" is not the same as "good for sysadmin tasks" is not the same as "good for data analysis" is not the same as "good for network service development", etc.
Cheers, Nick. Not that this was the original proposal, but there can be such a thing as a universal 'bad' package, though. So about the only thing that a universal package rating system can do effectively is shame developers. I don't think we want that.
2016-03-29 22:51 GMT+02:00, Michael Selik <mike@selik.org>: [...]
Back at Georgia Tech, my professor [0] once told me that the way to get rich is to invent an index.
2016-04-01 16:42 GMT+02:00, Alexander Walters <tritium-list@sdamon.com>:
On 4/1/2016 09:07, Nick Coghlan wrote: [...]
By contrast, I think Python itself covers too many domains for a common rating system to be feasible - "good for education" is not the same as "good for sysadmin tasks" is not the same as "good for data analysis" is not the same as "good for network service development", etc.
Cheers, Nick. Not that this was the original proposal, but there can be such a thing as a universal 'bad' package, though. So about the only thing that a universal package rating system can do effectively is shame developers. I don't think we want that.
1. There is not only good-bad dimension. beauty-ugly, simple-complex, flat-nested (and others from PEP20) trusted-untrusted is one from others. 2. Art critics are not to shame artists. Constructive criticism could help authors too. It is not benefit only for "customers". One who wants invent index to get rich could probably invent this job (Python critic) too. 3. pypi has number of downloads for day, week and month. If there is public api to data - one could make graphs, trends, etc and public them. If they will be nice then they probably could be included on pypi site too.
On Sat, Apr 2, 2016 at 8:16 PM, Pavol Lisy <pavol.lisy@gmail.com> wrote:
2. Art critics are not to shame artists. Constructive criticism could help authors too. It is not benefit only for "customers". One who wants invent index to get rich could probably invent this job (Python critic) too.
A reputation system is simply a score, though. If you are low on the rankings, you don't automatically know why - and you can't look at the high ranked results to figure out what to do better. That's why the inventor of the index gets a lot of money - the consultancy fees to help people get higher on the index will be gladly paid, because there's no other way to learn.
3. pypi has number of downloads for day, week and month. If there is public api to data - one could make graphs, trends, etc and public them. If they will be nice then they probably could be included on pypi site too.
That's popularity, which is different again from reputation. It suffers from several brutal limitations: 1) By definition, it cannot count *usage*. If I download something only to find that it completely fails me, it's still a download; if I download something and use it every day for years, it's still only one download. 2) It can only count downloads from the ultimate origin. If a Python package gets included in a downstream collection such as the Debian repositories, people who apt-get the package won't ting the stats. 3) Installations cascade into their dependencies. Does that count? What if you 'pip freeze' and now explicitly list all those dependencies in your requirements.txt - should they NOW count? 4) Even if you solve all of those problems, popularity begets popularity in many ways. Two competing packages may fight for a while, but one of them will "win", and the other languish - and it's virtually impossible for a new competitor to get to the point where it can be properly respected. Choosing which thing to use based on how many other people use it is sometimes important (the network effect - using Python rather than Bob's Obscure Scripting Language means it's a lot easier to find collaborators, no matter how technically superior BOSL might be), but it's almost completely orthogonal to quality. ChrisA
On 29.03.2016 12:36, Paul Moore wrote:
I don't intend to get sucked into the role of "defender of not inheriting", but one point that should probably be considered is that on Unix, it is possible to have paths which are *not* valid Python strings.
I don't know how other system developers are working in this field but that might be something serious I encounter from time to time. As a developer and API designer, one has two options (off the top of my head): 1) design and code with Unicode/UTF-8 as file names 2) deal with arbitrary bytes in file names I like 1) more because it gives me the ability to extract useful data/text from paths, which as far as I can tell is quite some usecase. 2) is more like a black box variant to me, but then operations like "/" or "+" or "suffix" etc. have far less meaning (at least to me).
How would those work in an inherit-from-str environment? I suspect that using surrogateescape is the answer here, but I don't know.
Could you explain what this means?
Also, has anyone *converted* code from os.path manipulation of strings to a str-subclass path object like path.py? If you write your code from scratch intending to use path.py it's probably easier to avoid objects getting silently downgraded from path to str than if you're retrofitting a path library to legacy code. That's a consideration that is more applicable to the standard library than to a 3rd party module.
Good point. It's quite problematic to convert a large system at once, so a transition is necessary. Maybe, that is a reason why more people want path->str.
But it may be that now is the right time to revisit the choice to not inherit from str. I don't know, personally. I'm not even sure if a pathlib.Path that inherited from str is something I'd use more often than I currently do.
It's only one of its warts. In order to get a greater mass to make the transition, most of these issues needs to be removed. Quite recently, there has been a discussion to add something to pathlib (I can't remember now) but it failed to get through because of "purity over practicability". A wasted chance IIRC. That thing was exactly what the Zen warns about.
For open source work, the need to support Python 2 means I can't use pathlib (without adding the backport as a dependency). For personal code, it's just a case of mental inertia...
Adding the dependency works for me. Best, Sven
On 03/29/2016 03:36 AM, Paul Moore wrote:
On 29 March 2016 at 10:43, Michel Desmoulin wrote:
+1. Can somebody defend, with practical examples, the imperative to refuse to inherit from str ? And weight it against the benefits of it ?
I cannot. Pathlib seems far too complicated to me. (And this coming from the guy whose antipathy [1] path library has both bytes and unicode inheritance and supports 2.7 and 3.3+ in the same code base.)
I don't intend to get sucked into the role of "defender of not inheriting", but one point that should probably be considered is that on Unix, it is possible to have paths which are *not* valid Python strings. How would those work in an inherit-from-str environment? I suspect that using surrogateescape is the answer here, but I don't know.
With antipathy's Path you get either a bPath if the incoming object was from bytes (str in 2.x) or a uPath if it was str (unicode in 2.x). The only problem I have ever hit is with older libraries that were doing type-checks instead of isinstance-checks on their incoming arguments.
Also, has anyone *converted* code from os.path manipulation of strings to a str-subclass path object like path.py? If you write your code from scratch intending to use path.py it's probably easier to avoid objects getting silently downgraded from path to str than if you're retrofitting a path library to legacy code. That's a consideration that is more applicable to the standard library than to a 3rd party module.
If I have to do any other work with the code, then I convert the os.path portions to Path first. It just makes it so much clearer.
But it may be that now is the right time to revisit the choice to not inherit from str.
Is it still marked provisional? The pain of using it with any other portion of the stdlib that uses paths makes it a non-starter for me. -- ~Ethan~ [1] https://pypi.python.org/pypi/antipathy/
On Tue, 29 Mar 2016 at 02:45 Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
Le 29/03/2016 09:50, Sven R. Kunze a écrit :
On 25.03.2016 22:56, Michel Desmoulin wrote:
Although path.py, which I have been using for years now (and still prefer to pathlib) subclass str and never caused any problem whatsoever. So really, we should pinpoint where it could be an issue and see if this is a frequent problem, because right now, it seems more a decision based on purity than practicality.
I agree. The PEP says:
""" No confusion with builtins ---------------------------------- In this proposal, the path classes do not derive from a builtin type. This contrasts with some other Path class proposals which were derived from str . They also do not pretend to implement the sequence protocol: if you want a path to act as a sequence, you have to lookup a dedicated attribute (the parts attribute).
Not behaving like one of the basic builtin types also minimizes the potential for confusion if a path is combined by accident with genuine builtin types. """"
I have to admit I cannot follow these statements but they should have appeared to be necessary back then. As experience shows the PyPI module fared far better here.
I am great a fan of theory over practice as what has been proven in theory cannot be proven wrong in practice. However, this only holds if we talk about hard proof. For soft things like "human interaction", "mutual understanding" or "potential for confusion", only the practice of many many people can "prove" what's useful, what's "practical".
+1. Can somebody defend, with practical examples, the imperative to refuse to inherit from str ? And weight it against the benefits of it ?
We have been doing it for years, and so far it's been really, really nice. I can't recall problem I had because of it ever. I can recall numerous editions of my code because I forgot str() on pathlib.Path.
I may regret this, but I will give this a shot. :) I wrote a blog post at http://www.snarky.ca/why-pathlib-path-doesn-t-inherit-from-str because my response was getting a bit long. I have pasted it below in CommonMark. ----- Over on [python-ideas](https://mail.python.org/mailman/listinfo/python-ideas) a discussion has broken out about somehow trying to make `p'/some/path/to/a/file` return an [instance of `pathlib.Path`]( https://docs.python.org/3/library/pathlib.html#pathlib.Path). This led to a splinter discussion as to why `pathlib.Path` doesn't inherit from `str`? I figured instead of burying my response to this question in the thread I'd blog about it to try and explain one approach to API design. I think the key question in all of this is whether paths are semantically equivalent to a sequence of characters? Obviously the answer is "no" since paths have structure, directly represent something like files and directories, etc. Now paths do have a serialized representation as strings -- at least most of the time, but I'm ignoring Linux and the crazy situation of binary paths -- which is why C APIs take in `char *` as the representation of paths (and because C tends to make one try to stick with the types that are part of the C standard). So not all strings represent paths, but all paths can be represented by strings. I think we can all agree with that. OK, so if all paths can be represented as strings, why don't we just make `pathlib.Path` subclass `str`? Well, as I said earlier, not all strings are paths. You can't concatenate a string representing a path with some other random string and expect to get back a string that still represents a valid string (and I'm not talking about "valid" as in "the file doesn't exist", I'm talking about "syntactically not possible"). This is what [PEP 428]( https://www.python.org/dev/peps/pep-0428/) is talking about when it says:
Not behaving like one of the basic builtin types [list str] also minimizes the potential for confusion if a path is combined by accident with genuine builtin types [like str].
Now at this point someone someone will start saying, "but Brett, '[practicality beats purity](https://www.python.org/dev/peps/pep-0020/)' and all of those pre-existing, old OS APIs want a string as an argument!" And you're right, in terms of practicality it would be easier for `pathlib.Path` to inherit from `str` ... for the short/medium term. One thing people are forgetting is that "[explicit is better than implicit]( https://www.python.org/dev/peps/pep-0020/)" as well and as with most design decisions there's not one clear answer. Implicitly treating a path as a string currently works, but that's just because we have inherited a suboptimal representation for paths from C. Now if you use an explicit representation of paths like `pathlib.Path` then you gain semantic separation and understanding of what you are working with. You also avoid any issues that come with implicit `str` compatibility as I pointed out earlier. And before anyone says, "so what?", think about this: Python 2/3. While I'm sure some will say I'm overblowing the comparison, but Python 3 came about because the implicit compatibility of binary and textual data in Python 2 caused major headaches to the point that we made a backwards-incompatible change that has caused widespread ramifications (but which we seem to be coming out on the other side of). By not inheriting from `str`, `pathlib.Path` has avoided a similar potential issue from the get-go. Or another way of looking at it is to ask why doesn't `dict` inherit from `str` so that it's easier to make it easier to stick in the body of an HTTP response so it can be implicitly treated as JSON? If you going, "eww, no because they are different types" then you at least understand the argument I'm trying to make (if you're going "I like that idea", then I think [Perl]( https://www.perl.org/) might be more to your liking than Python and that's fine since the two languages just have different designs). Now back to that "practicality beats purity" side of this. Obviously there are lots of APIs out there that take a string as an argument for a file path. And I understand people don't like using `str(path)` or the upcoming/new [`getattr(path, 'path', path)` idiom]( https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.path) to work around the limitations forced upon them by other APIs that only take a `str` because they occasionally forget it. For this point, I have two answers. One is still "explicit is better than implicit" and you should have tests to begin with to catch when you forget to convert your `pathlib.Path` objects to `str` as needed. I'm just one of those programmers who's willing to type a bit more for easy-to-read code that's less error-prone within reason (and I obviously view this as reasonable). But more importantly, why don't you work with the code and/or projects that are forcing you to convert to `str` to start accepting `pathlib` objects as well for those same APIs? If projects start to work on making `pathlib` objects acceptable anywhere a path is accepted then once Python 3.4 is the oldest version of Python that is supported by the project then they can start considering dropping support for `str` as paths in new releases (obviously this also includes dropping support for Python 2 unless people use [pathlib2](https://pypi.python.org/pypi/pathlib2/)). And I fully admit the stdlib is not exempt from a place that needs updating, so for `importlib` I have [opened an issue](http://bugs.python.org/issue26667) to update it to show this isn't just talk on my end (unfortunately there's some unique bootstrapping problems to import where stuff like `sys.path` probably can't be updated since that has to exist before you can import `pathlib` and that sort of thing ripples out, but I'm hoping I can update at least some things and this is a very unique case of potentially needing to stick with `str`). Hopefully if people start asking for `pathlib` support from projects they will add it, eventually leading to an alleviation of the desire to have `pathlib.Path` inherit from `str`.
On Tue, Mar 29, 2016, at 14:51, Brett Cannon wrote:
OK, so if all paths can be represented as strings, why don't we just make `pathlib.Path` subclass `str`? Well, as I said earlier, not all strings are paths. You can't concatenate a string representing a path with some other random string and expect to get back a string that still represents a valid string
I assume you mean a valid path. You can't add an int to a float and expect to get an integer back, either. If you're adding it to a random string you're clearly not using it in a path context, and the type of the result won't be Path anyway.
On Tue, 29 Mar 2016 at 11:59 Random832 <random832@fastmail.com> wrote:
On Tue, Mar 29, 2016, at 14:51, Brett Cannon wrote:
OK, so if all paths can be represented as strings, why don't we just make `pathlib.Path` subclass `str`? Well, as I said earlier, not all strings are paths. You can't concatenate a string representing a path with some other random string and expect to get back a string that still represents a valid string
I assume you mean a valid path.
Yep, it's a typo.
You can't add an int to a float and expect to get an integer back, either.
Sure, but that's because int and float are both part of the same type hierarchy (https://www.python.org/dev/peps/pep-3141/).
If you're adding it to a random string you're clearly not using it in a path context, and the type of the result won't be Path anyway.
You're assuming it's always clear. If that was always true then no one would ever find a bug in their code when porting to Python 3. We can all assume we won't screw up, but the fact bugs exist shows that's not true.
On Tue, Mar 29, 2016, at 17:37, Brett Cannon wrote:
You can't add an int to a float and expect to get an integer back, either.
Sure, but that's because int and float are both part of the same type hierarchy (https://www.python.org/dev/peps/pep-3141/).
...so would str and path, in the scenario being discussed. The argument presented here against allowing paths to be strings is also an argument against allowing ints to be Numbers.
On 03/29/2016 11:51 AM, Brett Cannon wrote:
I wrote a blog post at http://www.snarky.ca/why-pathlib-path-doesn-t-inherit-from-str because my response was getting a bit long. I have pasted it below in CommonMark.
Thanks for that. For myself at least, I also find the complexity of pathlib completely unnecessary: I see the structure of paths as loose at best -- at least in regards to the kinds of things we do with paths: - changing file names either in part or in whole, - changing directory names or making new directories from portions of the names of previous directories, - looking for directory or file names that have some substring as part of their names - etc. And yes, it is possible to incorrectly combine two path pieces (or a path and a string) together and get something that is semantically invalid -- but you can do that with any thing more complex than a random jumble of characters: - sentences - names - phrases - paragraphs - etc. But oh well -- what's done is done and we're probably stuck with it now. -- ~Ethan~
On 29.03.2016 20:51, Brett Cannon wrote:
I may regret this, but I will give this a shot. :)
Never, Brett. I think everybody here appreciate a decent discussion. Who knows maybe, somebody advocating for path->str misses an important piece which in turn misses the PEP to describe clearly.
I wrote a blog post at http://www.snarky.ca/why-pathlib-path-doesn-t-inherit-from-str because my response was getting a bit long. I have pasted it below in CommonMark.
Thanks +1 So, let's see:
Over on [python-ideas](https://mail.python.org/mailman/listinfo/python-ideas) a discussion has broken out about somehow trying to make `p'/some/path/to/a/file` return an [instance of `pathlib.Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path). This led to a splinter discussion as to why `pathlib.Path` doesn't inherit from `str`? I figured instead of burying my response to this question in the thread I'd blog about it to try and explain one approach to API design.
I think the key question in all of this is whether paths are semantically equivalent to a sequence of characters? Obviously the answer is "no" since paths have structure, directly represent something like files and directories, etc. Now paths do have a serialized representation as strings -- at least most of the time, but I'm ignoring Linux and the crazy situation of binary paths -- which is why C APIs take in `char *` as the representation of paths (and because C tends to make one try to stick with the types that are part of the C standard). So not all strings represent paths, but all paths can be represented by strings. I think we can all agree with that.
OK, so if all paths can be represented as strings, why don't we just make `pathlib.Path` subclass `str`? Well, as I said earlier, not all strings are paths. You can't concatenate a string representing a path with some other random string and expect to get back a string that still represents a valid string (and I'm not talking about "valid" as in "the file doesn't exist", I'm talking about "syntactically not possible"). This is what [PEP 428](https://www.python.org/dev/peps/pep-0428/) is talking about when it says:
Not behaving like one of the basic builtin types [list str] also minimizes the potential for confusion if a path is combined by accident with genuine builtin types [like str].
Valid string? I assume you mean path, right?
Now at this point someone someone will start saying, "but Brett, '[practicality beats purity](https://www.python.org/dev/peps/pep-0020/)' and all of those pre-existing, old OS APIs want a string as an argument!" And you're right, in terms of practicality it would be easier for `pathlib.Path` to inherit from `str` ... for the short/medium term. One thing people are forgetting is that "[explicit is better than implicit](https://www.python.org/dev/peps/pep-0020/)" as well and as with most design decisions there's not one clear answer. Implicitly treating a path as a string currently works, but that's just because we have inherited a suboptimal representation for paths from C. Now if you use an explicit representation of paths like `pathlib.Path` then you gain semantic separation and understanding of what you are working with. You also avoid any issues that come with implicit `str` compatibility as I pointed out earlier.
I agree with the concatenation issue when using plain strings for paths. However, something that I cannot leave uncommented is "suboptimal representation for paths". What would your optimal representation for paths look like? I cannot believe that the current representation is so bad and has been for so long and nobody, really nobody, has anything done about it.
And before anyone says, "so what?", think about this: Python 2/3. While I'm sure some will say I'm overblowing the comparison, but Python 3 came about because the implicit compatibility of binary and textual data in Python 2 caused major headaches to the point that we made a backwards-incompatible change that has caused widespread ramifications (but which we seem to be coming out on the other side of). By not inheriting from `str`, `pathlib.Path` has avoided a similar potential issue from the get-go. Or another way of looking at it is to ask why doesn't `dict` inherit from `str` so that it's easier to make it easier to stick in the body of an HTTP response so it can be implicitly treated as JSON? If you going, "eww, no because they are different types" then you at least understand the argument I'm trying to make (if you're going "I like that idea", then I think [Perl](https://www.perl.org/) might be more to your liking than Python and that's fine since the two languages just have different designs).
I think most "practicality beats purity" folks don't want that either. They are just bloody lazy. They actually want the benefits of both, the pure datastructure with its convenience methods and the dirty str-like thing with its convenience methods. People don't like it when somebody takes one bag of convenience away from them if they easily can have both. ;-) Better sell it differently. So, let's read on. ;)
Now back to that "practicality beats purity" side of this. Obviously there are lots of APIs out there that take a string as an argument for a file path. And I understand people don't like using `str(path)` or the upcoming/new [`getattr(path, 'path', path)` idiom](https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.path) to work around the limitations forced upon them by other APIs that only take a `str` because they occasionally forget it. For this point, I have two answers.
Sorry to interrupt your here but you missed one important piece (didn't we agree on selling this better?): people do nasty little string manipulations/analysis/regex with their paths for whatever reasons. Right now, it's easier than ever because, well, because paths are strings. :) Selling pathlib with the "getattr(path, 'path', path)" idiom is not going to help those fellows. I mean, are you serious? It looks ridiculous. :-/ Especially the point that a "path" has a "path" is not really getting into my head. Either a "path" is a "path" or it better be named "file/directory" which has a "path". path.path vs file.path/dir.path You need to work on your selling skills, Brett. ;-)
One is still "explicit is better than implicit" and you should have tests to begin with to catch when you forget to convert your `pathlib.Path` objects to `str` as needed. I'm just one of those programmers who's willing to type a bit more for easy-to-read code that's less error-prone within reason (and I obviously view this as reasonable).
But more importantly, why don't you work with the code and/or projects that are forcing you to convert to `str` to start accepting `pathlib` objects as well for those same APIs? If projects start to work on making `pathlib` objects acceptable anywhere a path is accepted then once Python 3.4 is the oldest version of Python that is supported by the project then they can start considering dropping support for `str` as paths in new releases (obviously this also includes dropping support for Python 2 unless people use [pathlib2](https://pypi.python.org/pypi/pathlib2/) <https://pypi.python.org/pypi/pathlib2/%29>). And I fully admit the stdlib is not exempt from a place that needs updating, so for `importlib` I have [opened an issue](http://bugs.python.org/issue26667) to update it to show this isn't just talk on my end (unfortunately there's some unique bootstrapping problems to import where stuff like `sys.path` probably can't be updated since that has to exist before you can import `pathlib` and that sort of thing ripples out, but I'm hoping I can update at least some things and this is a very unique case of potentially needing to stick with `str`). Hopefully if people start asking for `pathlib` support from projects they will add it, eventually leading to an alleviation of the desire to have `pathlib.Path` inherit from `str`.
I think updating the stdlib looks sufficiently simple to start hacking CPython. So, I would be willing to help here once the github repo is up and running. :) Nice post, Brett. Seems you covered the most important point quite well and also gave an explanation of why people request path->str. Best, Sven
On 29 March 2016 at 22:00, Sven R. Kunze <srkunze@mail.de> wrote:
However, something that I cannot leave uncommented is "suboptimal representation for paths". What would your optimal representation for paths look like? I cannot believe that the current representation is so bad and has been for so long and nobody, really nobody, has anything done about it.
Well, pathlib.Path :-) The point here is that C's char* representation is a serialisation of a path object, just like 123 is a serialisation of an integer object. We don't object to having to convert user input to a string if we need to, why object to having to convert it to a Path if appropriate? Paul
On 29.03.2016 23:57, Paul Moore wrote:
On 29 March 2016 at 22:00, Sven R. Kunze <srkunze@mail.de> wrote:
However, something that I cannot leave uncommented is "suboptimal representation for paths". What would your optimal representation for paths look like? I cannot believe that the current representation is so bad and has been for so long and nobody, really nobody, has anything done about it. Well, pathlib.Path :-)
The point here is that C's char* representation is a serialisation of a path object, just like 123 is a serialisation of an integer object. We don't object to having to convert user input to a string if we need to, why object to having to convert it to a Path if appropriate?
I think there is a misunderstanding here. Let me quote myself: '''I think most "practicality beats purity" folks don't want that either. They are just bloody lazy. They actually want the benefits of both, the pure datastructure with its convenience methods and the dirty str-like thing with its convenience methods.''' The desire is not "str vs. Path"; its "Path + str". (at least from what I can tell) Best, Sven
On Mar 30, 2016 3:47 AM, "Sven R. Kunze" <srkunze@mail.de> wrote:
On 29.03.2016 23:57, Paul Moore wrote:
On 29 March 2016 at 22:00, Sven R. Kunze <srkunze@mail.de> wrote:
However, something that I cannot leave uncommented is "suboptimal representation for paths". What would your optimal representation for
look like? I cannot believe that the current representation is so bad and has been for so long and nobody, really nobody, has anything done about it.
Well, pathlib.Path :-)
The point here is that C's char* representation is a serialisation of a path object, just like 123 is a serialisation of an integer object. We don't object to having to convert user input to a string if we need to, why object to having to convert it to a Path if appropriate?
I think there is a misunderstanding here. Let me quote myself:
'''I think most "practicality beats purity" folks don't want that either. They are just bloody lazy. They actually want the benefits of both, the
paths pure datastructure with its convenience methods and the dirty str-like thing with its convenience methods.'''
The desire is not "str vs. Path"; its "Path + str". (at least from what I
can tell) so, like str but without +, %, .format ? class pathstr(str): __add__ __mod__ format python3: pathstr(str)? python2: pathstr(unicode)?
Best, Sven
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 30.03.2016 10:56, Wes Turner wrote:
so, like str but without +, %, .format ?
class pathstr(str): __add__ __mod__ format
python3: pathstr(str)? python2: pathstr(unicode)?
I am not sure I understand. Where is the Path in your code? To make it more explicit: like "all methods of Path" + "all methods of str". And no, format, % and + are nice to have in case you need them; so why removing them? Best, Sven
On 30 March 2016 at 09:45, Sven R. Kunze <srkunze@mail.de> wrote:
On 29.03.2016 23:57, Paul Moore wrote:
On 29 March 2016 at 22:00, Sven R. Kunze <srkunze@mail.de> wrote:
However, something that I cannot leave uncommented is "suboptimal representation for paths". What would your optimal representation for paths look like? I cannot believe that the current representation is so bad and has been for so long and nobody, really nobody, has anything done about it.
Well, pathlib.Path :-)
The point here is that C's char* representation is a serialisation of a path object, just like 123 is a serialisation of an integer object. We don't object to having to convert user input to a string if we need to, why object to having to convert it to a Path if appropriate?
I think there is a misunderstanding here. Let me quote myself:
'''I think most "practicality beats purity" folks don't want that either. They are just bloody lazy. They actually want the benefits of both, the pure datastructure with its convenience methods and the dirty str-like thing with its convenience methods.'''
The desire is not "str vs. Path"; its "Path + str". (at least from what I can tell)
I understand all that. I thought you were referring to Brett's comment that C's "char *" format was "suboptimal" and asking what would be an optimal representation (you said "your optimal representation", meaning Brett's, so only he can answer to that but I took the question more generally as what would be *an* optimal representation). My reply was intended to say that I view something like pathlib.Path as optimal. I don't personally see "working like a string" as being optimal, precisely because it mixes the "structured object" level with the "serialised representation" level. That's something Brett's article clarified for me, so I suspect he'd have the same view. I doubt the people arguing for Path being a subclass of string would be swayed by an argument like the above, though, as they would view it as "purity over practicality". Personally, I find that careful separation of concerns is a practicality benefit, in terms of maintainability and clarity of code - a lesson I learned from having to do far too many bytes/unicode migrations where separating out the concepts was a vital step. Anyway, this is pretty off topic, so I'll leave it there. Paul
On Wed, Mar 30, 2016 at 4:20 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 29.03.2016 23:57, Paul Moore wrote:
On 29 March 2016 at 22:00, Sven R. Kunze <srkunze@mail.de> wrote:
However, something that I cannot leave uncommented is "suboptimal representation for paths". What would your optimal representation for paths look like? I cannot believe that the current representation is so bad
and
has been for so long and nobody, really nobody, has anything done about it.
Well, pathlib.Path :-)
The point here is that C's char* representation is a serialisation of a path object, just like 123 is a serialisation of an integer object. We don't object to having to convert user input to a string if we need to, why object to having to convert it to a Path if appropriate?
I think there is a misunderstanding here. Let me quote myself:
'''I think most "practicality beats purity" folks don't want that either. They are just bloody lazy. They actually want the benefits of both, the
On 30 March 2016 at 09:45, Sven R. Kunze <srkunze@mail.de> wrote: pure
datastructure with its convenience methods and the dirty str-like thing with its convenience methods.'''
The desire is not "str vs. Path"; its "Path + str". (at least from what I can tell)
I understand all that. I thought you were referring to Brett's comment that C's "char *" format was "suboptimal" and asking what would be an optimal representation (you said "your optimal representation", meaning Brett's, so only he can answer to that but I took the question more generally as what would be *an* optimal representation). My reply was intended to say that I view something like pathlib.Path as optimal.
Really, these are actual graph paths sequences Because: * When you have to abspath, normpath, check for '../.../../.$var./..', you have to parse it by splitting on /, anyway, which indicates that a list/tuple would be more optimal in some use cases.
I don't personally see "working like a string" as being optimal, precisely because it mixes the "structured object" level with the "serialised representation" level. That's something Brett's article clarified for me, so I suspect he'd have the same view.
" To e.g. rsync, this is a different thing: ./path/ ./path
I doubt the people arguing for Path being a subclass of string would be swayed by an argument like the above, though, as they would view it as "purity over practicality". Personally, I find that careful separation of concerns is a practicality benefit, in terms of maintainability and clarity of code - a lesson I learned from having to do far too many bytes/unicode migrations where separating out the concepts was a vital step.
Practically, because path.py subclasses unicode in python2 and str in python 3, path.py should work with all existing standard library methods: https://github.com/jaraco/path.py/blob/master/path.py
Anyway, this is pretty off topic, so I'll leave it there. Paul _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
. They actually want the benefits of both, the pure datastructure with its convenience methods and the dirty str-like thing with its convenience methods.'''
The desire is not "str vs. Path"; its "Path + str". (at least from what I can tell)
I don't think so. ( though I agree with the lazy part). People want paths to be a strings so that they will work with all the code that already works with strings. That's the primary motivator. If early versions of Python had a path object, there'd be no problem. But whatever happened to duck typing? Paths don't need to BE strings. Rather, everything that needs a path needs to accept anything that acts like a path. I suppose a __path__ magic method would be the "proper" way to do this, but it's really not necessary, it can be handled by the libs themselves. I think there is a fine plan in place, but if more flexibility is required, something like numpy's asarray() would be handy -- it passes actual Numpy arrays through untouched, and makes various efforts to make an array out of anything else. It's remarkably easy and effective to write functions that take virtually anything remotely array like -- lists, tuples, array.arrays, custom objects, etc. -CHB
On 30 March 2016 at 15:26, Chris Barker - NOAA Federal <chris.barker@noaa.gov> wrote:
. They actually want the benefits of both, the pure datastructure with its convenience methods and the dirty str-like thing with its convenience methods.'''
The desire is not "str vs. Path"; its "Path + str". (at least from what I can tell)
I don't think so. ( though I agree with the lazy part).
I'm not sure I follow your point here.
People want paths to be a strings so that they will work with all the code that already works with strings.
Correct. That's the prime motivation. But you then say
But whatever happened to duck typing? Paths don't need to BE strings. Rather, everything that needs a path needs to accept anything that acts like a path.
But "all the code that already works with strings" doesn't do that. If we're allowed to change that code then a simple patharg = getattr(patharg, 'path', patharg) is sufficient to work with path objects or strings. Of course this doesn't address functions that *return* paths (as strings). There the caller has to wrap the return value in Path(). Or the function changes to return Path objects, which won't be backward compatible (whether that matters depends on what the code is).
I suppose a __path__ magic method would be the "proper" way to do this, but it's really not necessary, it can be handled by the libs themselves.
Well, the 'path' attribute is basically the same as a __path__ magic method. (That may be what you mean by "can be handled by the libs" - I'm not sure how you'd handle it *other* than in the libraries that currently assume a string).
I think there is a fine plan in place, but if more flexibility is required, something like numpy's asarray() would be handy -- it passes actual Numpy arrays through untouched, and makes various efforts to make an array out of anything else.
It's remarkably easy and effective to write functions that take virtually anything remotely array like -- lists, tuples, array.arrays, custom objects, etc.
Again, this seems to be backwards - the problem isn't treating things that aren't paths as paths (just do p = Path(p) and that's sorted - paths are immutable so there's no need for the complexity that I imagine asarray has to manage). The problem is existing code that expects and/or returns strings. Until that is changed or replaced with code that expects and returns Path objects, people will always need to convert to and from strings. And of course code that has to deal with *both* Path and string objects (for compatibility reasons) can easily enough handle anything it receives, but has a decision to make about what to return - if it returns Path objects, it won't be backward compatible. But if it returns string objects, we'll never get away from the need to convert strings to paths in our code at some point. As a simple case in point, what should the appdirs module (https://pypi.python.org/pypi/appdirs) do? Continue to return strings, and the user needs to wrap them in Path(), or switch to returning Path objects and break backward compatibility? Or maintain a messy API that caters (somehow) for both possibilities? It may be that we're talking at cross purposes here. I'm thinking very much of users working with library code on from PyPI and things like that. You may be looking at this from the perspective of a user who controls the majority of the code they are working with. I'm not sure. A transition like this is never simple, and libraries pretty much have to wait for end user code to switch first. And paths as string subclasses doesn't really count as switching, it simply makes it easier to ignore the issue. Arguably being able to pass a path object to code that expects a string means that you don't have to change your code twice, first to handle path objects because the library code doesn't, then again to switch back when the library code is fixed. But then there is zero motivation for the library code to change. And if code *accepting* paths doesn't change, then code *producing* paths won't either. And we're stuck where we are at the moment, with the string representation as the lowest common denominator representation of a path, and everyone converting back and forth if they want the richer Path interface (or deciding it's not worth it, and sticking to the old os.path routines...). Paul
On 03/30/2016 08:18 AM, Paul Moore wrote:
And of course code that has to deal with *both* Path and string objects (for compatibility reasons) can easily enough handle anything it receives, but has a decision to make about what to return - if it returns Path objects, it won't be backward compatible. But if it returns string objects, we'll never get away from the need to convert strings to paths in our code at some point.
A solution here is to return what you receive: - str coming in? str going out - path coming in? path going out Of course, this gets muddied when multiple path/str arguments are accepted coming in. -- ~Ethan~
On 30.03.2016 18:49, Ethan Furman wrote:
A solution here is to return what you receive:
- str coming in? str going out - path coming in? path going out
Of course, this gets muddied when multiple path/str arguments are accepted coming in.
I have bad memories about this kind of approach. It reminds me of the peculiarity of os(.path)-related functions in Python 2 that return either bytes or unicode depending on the argument's type. That's utterly confusing and I remember us hunting bugs related to this for hours (basically looking for "where does this byte string come from?"). Best, Sven
On 31 March 2016 at 10:09, Sven R. Kunze <srkunze@mail.de> wrote:
On 30.03.2016 18:49, Ethan Furman wrote:
A solution here is to return what you receive:
- str coming in? str going out - path coming in? path going out
Of course, this gets muddied when multiple path/str arguments are accepted coming in.
I have bad memories about this kind of approach. It reminds me of the peculiarity of os(.path)-related functions in Python 2 that return either bytes or unicode depending on the argument's type.
That's utterly confusing and I remember us hunting bugs related to this for hours (basically looking for "where does this byte string come from?").
Any suggestions for a better design? It might be nice for someone to write a blog entry or posting somewhere on "how to upgrade your 3rd party library to support pathlib" and one of the harder questions will be "how do I decide whether to return a string or a path?" If we had a good recommendation for this, it might help adoption. Paul
On 31.03.2016 11:15, Paul Moore wrote:
Any suggestions for a better design? It might be nice for someone to write a blog entry or posting somewhere on "how to upgrade your 3rd party library to support pathlib" and one of the harder questions will be "how do I decide whether to return a string or a path?" If we had a good recommendation for this, it might help adoption.
Good point. This discussion has been long already so it might make sense to write something down in a concise manner. I am not sure if I can make a good suggestion here because I am still trapped at the point where I need to sort out if a path more like a dict or another structured datatype, or if it is more a monolithic object. That will be the next blog post topic for me. "how do I decide whether to return a string or a path?" << very good question btw; my initial thought would be; make two functions (maybe the same name in two different namespaces), so it's crystal clear what this function takes a arguments and what it returns. But I need to think more about this. Best, Sven
On Thu, Mar 31, 2016 at 12:46 PM, Sven R. Kunze <srkunze@mail.de> wrote:
On 31.03.2016 11:15, Paul Moore wrote:
Any suggestions for a better design? It might be nice for someone to write a blog entry or posting somewhere on "how to upgrade your 3rd party library to support pathlib" and one of the harder questions will be "how do I decide whether to return a string or a path?" If we had a good recommendation for this, it might help adoption.
True. When we do have the answer, we need to let people know about it.
Good point. This discussion has been long already so it might make sense to write something down in a concise manner.
I am not sure if I can make a good suggestion here because I am still trapped at the point where I need to sort out if a path more like a dict or another structured datatype, or if it is more a monolithic object. That will be the next blog post topic for me.
While discussing, pondering and experimenting with this, I have formed a quite clear view on how I think we should move forward with this. I'm working on a proposal that attempts to not be in conflict with any of the goals. I hope it is not in conflict with your present thoughts about structured/vs monolithic objects either :)
"how do I decide whether to return a string or a path?" << very good question btw; my initial thought would be; make two functions (maybe the same name in two different namespaces), so it's crystal clear what this function takes a arguments and what it returns. But I need to think more about this.
Best, Sven
On 31.03.2016 13:31, Koos Zevenhoven wrote:
On Thu, Mar 31, 2016 at 12:46 PM, Sven R. Kunze <srkunze@mail.de> wrote:
I am not sure if I can make a good suggestion here because I am still trapped at the point where I need to sort out if a path more like a dict or another structured datatype, or if it is more a monolithic object. That will be the next blog post topic for me. While discussing, pondering and experimenting with this, I have formed a quite clear view on how I think we should move forward with this. I'm working on a proposal that attempts to not be in conflict with any of the goals. I hope it is not in conflict with your present thoughts about structured/vs monolithic objects either :)
Not at all. The following post it just a reflection of this whole discussion to make it clear to myself and hopefully to others about where several thoughts emerged from and why this whole topic started in the first place. Basically, a structured collection of thoughts, mainly mine: http://srkunze.blogspot.com/2016/03/what-is-path.html I pasted the interesting meat below: *What is a path in the first place?* Brett Cannon made a good point of why PEP 428 (that's the one which introduced pathlib as a stdlib module) deliberately chose Path not to inherit from string. I pondered over it for a while and saw that from his perspective a path is actually not a string but rather a complex data-structure consisting of parts with distinct meanings: literally the steps (of which a path consists) to a resource. The string which represents a path as most people know it is just that: a representation of a more complex object, just like a dict or a list. Let me make this a bit clearer. I think we agree on the following: writing down 21 characters in a row is a string, right? So, what about these 21 characters? {1: 11, 2: 12, 3: 13} If you see that in a Python program (and presumably in many other modern programming languages), you associate that with a dictionary, mapping, hash, etc. So, these 21 characters are a mere representation of a complex object with a very rich functionality. The following paragraphs summarizes what makes the discussion about paths and strings to hard. Depending on whom you ask there are different interpretations of what a path actually is. *Paths as complex objects and strings as their representation* Let's put this analogy to work with paths. If you come from a language that treats strings as file paths (like Python), you can imagine and categorize the facilities of pathlib like so: * pure path - operating on the path string * concrete path - operating on files corresponding to the given path The classic "extract the file extension" issue is done easily with the pure path methods. Writing to a file is also easily done with concrete path operations. So, it seems paths are pretty complex objects with some internal structure and a lot of functionality. *Paths as monolithic object for addressing resources* The previous interpretation is not the only one. Despite all the fine functionality of extracting file extensions, concatenating parts to a larger path, etc., building a path is not an end in itself. When you got a path, it addresses a resource on a machine. When doing so for reading or writing that resource, you actually don't care about whether the path consists of parts or not. To you, it's a monolithic structure, an address. But, you might say, each part of a path represents a directory in hierarchical file systems. Sure that is true for many file systems but not for all. Moreover, how often do you really care about the underlying directory structure? It needs to be there to make things work, of course. When it's there, you mostly don't care. How often do you need to create a subtree in an directory in order to create a single config file? I encounter this once in a while and to be honest: it sucks. ```touch /home/me/on/your/ssd.conf``` will fail if the directory "on/your/" has not been created by somebody before me. Especially for me, as a Web developer, it's quite hard to understand what purpose this restriction serves. Within a Web application the hierarchy of URLs is an emergent property not a prerequisite. Users of git are accustomed to not-committing directories in. Why? Because it's unnecessary and the directory structure is again emerging from the files names themselves (aka from the content). This said, it's rather cumbersome to attribute semantics to the parts of a string that happens to be separated by "/" or "\". At least to me, a path made of one piece. *What about security then?* One can further argue that Web development and git repositories are different here. There is a clear boundary where a path can lead. A URL path cannot address a foreign resource on another domain. git file paths are contained within the repository root. See the common theme? There is a container from where the path of a resource cannot escape. If you have a complete file system available at your fingertips, a lot harm can be done when malicious user input is concatenated unattendedly as a subpath; actually to address a resource within a container but misused to gain access to the complete file system. I cannot say if the container pattern would work for everybody but it's definitely worth exploring as there are some prominent working examples out there. *Conclusion* I really like pathlib since it solves many frequently asked questions in the right way once and for all. But I don't like using it as an argument again inheriting paths from strings saying paths have internal structure in contrast to strings. At least to me, they do not. That, on the other hand, does not necessarily mean inheriting path from string is a good idea but it makes it no worse one than it was before. That's it. Best, Sven
That's a very interesting perspective, and one I'd not fully appreciated. Thanks for posting it. There's a lot I still have to think about in your post, but there's one point I'd like to comment on already: On 31 March 2016 at 15:43, Sven R. Kunze <srkunze@mail.de> wrote:
When doing so for reading or writing that resource, you actually don't care about whether the path consists of parts or not.
While that's true for *existing* resources (files) it's not so true for when you're creating a file. Like it or not, the filesystem imposes a set of container-containee relationships that is the directory structure. Your mental model (the application domain view) may not care about this, but at the "path" level, we're representing the filesystem structure, where directories are real, and so a/b/c being a set of 3 nested objects, 2 directories and a leaf object (which may be a file or itself a directory) is the reality being modeled. So I'd suggest that your "resource address" is a higher level abstraction, layered on top of the filesystem. In many ways, it's a URI - and as such may map to a filesystem path or to something else. This also clarifies (at least to me) why I am uncomfortable about the idea of merging pathlib and URIs - they are at 2 different abstraction levels. It's somewhat interesting to me that the "resource address" level of abstraction is a higher level that the "filesystem path" level, but has *less* structure. That seems reasonable to me (you abstract away the details). And it makes the string representation at the "lowest level" the really odd case, as it has *no* structure, and yet it's representing a detail level of object structure. Maybe that's an indication of why it's useful to have a structured Path object at this level rather than working solely with strings? Paul
On Mar 31, 2016 10:02 AM, "Paul Moore" <p.f.moore@gmail.com> wrote:
That's a very interesting perspective, and one I'd not fully appreciated. Thanks for posting it.
There's a lot I still have to think about in your post, but there's one point I'd like to comment on already:
On 31 March 2016 at 15:43, Sven R. Kunze <srkunze@mail.de> wrote:
When doing so for reading or writing that resource, you actually don't
care
about whether the path consists of parts or not.
While that's true for *existing* resources (files) it's not so true for when you're creating a file. Like it or not, the filesystem imposes a set of container-containee relationships that is the directory structure. Your mental model (the application domain view) may not care about this, but at the "path" level, we're representing the filesystem structure, where directories are real, and so a/b/c being a set of 3 nested objects, 2 directories and a leaf object (which may be a file or itself a directory) is the reality being modeled.
So I'd suggest that your "resource address" is a higher level abstraction, layered on top of the filesystem. In many ways, it's a URI - and as such may map to a filesystem path or to something else. This also clarifies (at least to me) why I am uncomfortable about the idea of merging pathlib and URIs - they are at 2 different abstraction levels.
It's somewhat interesting to me that the "resource address" level of abstraction is a higher level that the "filesystem path" level, but has *less* structure. That seems reasonable to me (you abstract away the details). And it makes the string representation at the "lowest level" the really odd case, as it has *no* structure, and yet it's representing a detail level of object structure. Maybe that's an indication of why it's useful to have a structured Path object at this level rather than working solely with strings?
it would seem that the utility here includes having a @memoize-d parsed copy of the path; and then not doubling forward/backslashes ... and then checking for '../' traversals and any chroot-like relpath predicates w/ e.g. fnmatch); and not shell quoting None = gethostname() os.path.sep.join(['file:', None, '/home/user/']) os.path.sep.join(['file:', None, ('/home', '/user/')]) p = Path('/home/user/') p.parts # ... URLObject + path.py
Paul _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Mar 31, 2016 10:36 AM, "Wes Turner" <wes.turner@gmail.com> wrote:
On Mar 31, 2016 10:02 AM, "Paul Moore" <p.f.moore@gmail.com> wrote:
That's a very interesting perspective, and one I'd not fully appreciated. Thanks for posting it.
There's a lot I still have to think about in your post, but there's one point I'd like to comment on already:
On 31 March 2016 at 15:43, Sven R. Kunze <srkunze@mail.de> wrote:
When doing so for reading or writing that resource, you actually
don't care
about whether the path consists of parts or not.
While that's true for *existing* resources (files) it's not so true for when you're creating a file. Like it or not, the filesystem imposes a set of container-containee relationships that is the directory structure. Your mental model (the application domain view) may not care about this, but at the "path" level, we're representing the filesystem structure, where directories are real, and so a/b/c being a set of 3 nested objects, 2 directories and a leaf object (which may be a file or itself a directory) is the reality being modeled.
So I'd suggest that your "resource address" is a higher level abstraction, layered on top of the filesystem. In many ways, it's a URI - and as such may map to a filesystem path or to something else. This also clarifies (at least to me) why I am uncomfortable about the idea of merging pathlib and URIs - they are at 2 different abstraction levels.
It's somewhat interesting to me that the "resource address" level of abstraction is a higher level that the "filesystem path" level, but has *less* structure. That seems reasonable to me (you abstract away the details). And it makes the string representation at the "lowest level" the really odd case, as it has *no* structure, and yet it's representing a detail level of object structure. Maybe that's an indication of why it's useful to have a structured Path object at this level rather than working solely with strings?
it would seem that the utility here includes having a @memoize-d parsed copy of the path; and then not doubling forward/backslashes ... and then checking for '../' traversals and any chroot-like relpath predicates w/ e.g. fnmatch); and not shell quoting
None = gethostname() os.path.sep.join(['file:', None, '/home/user/']) os.path.sep.join(['file:', None, ('/home', '/user/')])
p = Path('/home/user/') p.parts
# ... URLObject + path.py
URLObject has a .path attr for getattr, too. - char*: bytestring, unicode/str
Paul _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 31.03.2016 17:02, Paul Moore wrote:
That's a very interesting perspective, and one I'd not fully appreciated. Thanks for posting it.
You're welcome.
There's a lot I still have to think about in your post, but there's one point I'd like to comment on already:
When doing so for reading or writing that resource, you actually don't care about whether the path consists of parts or not. While that's true for *existing* resources (files) it's not so true for when you're creating a file. Like it or not, the filesystem imposes a set of container-containee relationships that is the
On 31 March 2016 at 15:43, Sven R. Kunze <srkunze@mail.de> wrote: directory structure. Your mental model (the application domain view) may not care about this, but at the "path" level, we're representing the filesystem structure, where directories are real, and so a/b/c being a set of 3 nested objects, 2 directories and a leaf object (which may be a file or itself a directory) is the reality being modeled.
So I'd suggest that your "resource address" is a higher level abstraction, layered on top of the filesystem. In many ways, it's a URI - and as such may map to a filesystem path or to something else. This also clarifies (at least to me) why I am uncomfortable about the idea of merging pathlib and URIs - they are at 2 different abstraction levels.
It's somewhat interesting to me that the "resource address" level of abstraction is a higher level that the "filesystem path" level, but has *less* structure. That seems reasonable to me (you abstract away the details). And it makes the string representation at the "lowest level" the really odd case, as it has *no* structure, and yet it's representing a detail level of object structure. Maybe that's an indication of why it's useful to have a structured Path object at this level rather than working solely with strings?
That all makes perfect sense now. I didn't expect such a quick and enlightening response. That's awesome. Thanks a lot. That might also influence how we look at the other thread about including better URL handling to Python. It's just another abstraction level and the main question is: is Python ready for that level or better use a third party library for it. Best, Sven
On 03/31/2016 08:56 AM, Sven R. Kunze wrote:
That might also influence how we look at the other thread about including better URL handling to Python. It's just another abstraction level and the main question is: is Python ready for that level or better use a third party library for it.
The actual questions are: 1) Should URL abstraction be merged in to pathlib? 2) If no, does it /need/ to be in the stdlib? -- ~Ethan~
Paul Moore <p.f.moore@...> writes:
People want paths to be a strings so that they will work with all the code that already works with strings.
Correct. That's the prime motivation. But you then say
But whatever happened to duck typing? Paths don't need to BE strings. Rather, everything that needs a path needs to accept anything that acts like a path.
But "all the code that already works with strings" doesn't do that. If we're allowed to change that code then a simple
patharg = getattr(patharg, 'path', patharg)
is sufficient to work with path objects or strings.
I think you have it backwards here: the problem is not updating code that you can change, it's using any external library that uses paths as strings. When I use a library from pypi, I won't add a new line in every function using a string representing a path. When if I use path.py, I can directly pass paths to any library without changing its code.
On Tue, 29 Mar 2016 at 14:00 Sven R. Kunze <srkunze@mail.de> wrote:
On 29.03.2016 20:51, Brett Cannon wrote:
I may regret this, but I will give this a shot. :)
Never, Brett. I think everybody here appreciate a decent discussion. Who knows maybe, somebody advocating for path->str misses an important piece which in turn misses the PEP to describe clearly.
I wrote a blog post at http://www.snarky.ca/why-pathlib-path-doesn-t-inherit-from-str because my response was getting a bit long. I have pasted it below in CommonMark.
Thanks +1
So, let's see:
Over on [python-ideas](https://mail.python.org/mailman/listinfo/python-ideas) a discussion has broken out about somehow trying to make `p'/some/path/to/a/file` return an [instance of `pathlib.Path`]( https://docs.python.org/3/library/pathlib.html#pathlib.Path). This led to a splinter discussion as to why `pathlib.Path` doesn't inherit from `str`? I figured instead of burying my response to this question in the thread I'd blog about it to try and explain one approach to API design.
I think the key question in all of this is whether paths are semantically equivalent to a sequence of characters? Obviously the answer is "no" since paths have structure, directly represent something like files and directories, etc. Now paths do have a serialized representation as strings -- at least most of the time, but I'm ignoring Linux and the crazy situation of binary paths -- which is why C APIs take in `char *` as the representation of paths (and because C tends to make one try to stick with the types that are part of the C standard). So not all strings represent paths, but all paths can be represented by strings. I think we can all agree with that.
OK, so if all paths can be represented as strings, why don't we just make `pathlib.Path` subclass `str`? Well, as I said earlier, not all strings are paths. You can't concatenate a string representing a path with some other random string and expect to get back a string that still represents a valid string (and I'm not talking about "valid" as in "the file doesn't exist", I'm talking about "syntactically not possible"). This is what [PEP 428](https://www.python.org/dev/peps/pep-0428/) is talking about when it says:
Not behaving like one of the basic builtin types [list str] also minimizes the potential for confusion if a path is combined by accident with genuine builtin types [like str].
Valid string? I assume you mean path, right?
Yes.
Now at this point someone someone will start saying, "but Brett, '[practicality beats purity](https://www.python.org/dev/peps/pep-0020/)' and all of those pre-existing, old OS APIs want a string as an argument!" And you're right, in terms of practicality it would be easier for `pathlib.Path` to inherit from `str` ... for the short/medium term. One thing people are forgetting is that "[explicit is better than implicit](https://www.python.org/dev/peps/pep-0020/)" as well and as with most design decisions there's not one clear answer. Implicitly treating a path as a string currently works, but that's just because we have inherited a suboptimal representation for paths from C. Now if you use an explicit representation of paths like `pathlib.Path` then you gain semantic separation and understanding of what you are working with. You also avoid any issues that come with implicit `str` compatibility as I pointed out earlier.
I agree with the concatenation issue when using plain strings for paths.
However, something that I cannot leave uncommented is "suboptimal representation for paths". What would your optimal representation for paths look like?
I don't know, but I do like pathlib.Path more than a plain string.
I cannot believe that the current representation is so bad and has been for so long and nobody, really nobody, has anything done about it.
I never said it was bad, just suboptimal. If you want you can say using strings as paths is mediocre. I really don't want to argue about the label, just that pathlib.Path is a better representation of a path than a plain string and that strings on their own are nothing that special in terms of representing a string that demands we defend it on its own grounds as such.
And before anyone says, "so what?", think about this: Python 2/3. While I'm sure some will say I'm overblowing the comparison, but Python 3 came about because the implicit compatibility of binary and textual data in Python 2 caused major headaches to the point that we made a backwards-incompatible change that has caused widespread ramifications (but which we seem to be coming out on the other side of). By not inheriting from `str`, `pathlib.Path` has avoided a similar potential issue from the get-go. Or another way of looking at it is to ask why doesn't `dict` inherit from `str` so that it's easier to make it easier to stick in the body of an HTTP response so it can be implicitly treated as JSON? If you going, "eww, no because they are different types" then you at least understand the argument I'm trying to make (if you're going "I like that idea", then I think [Perl](https://www.perl.org/) might be more to your liking than Python and that's fine since the two languages just have different designs).
I think most "practicality beats purity" folks don't want that either. They are just bloody lazy. They actually want the benefits of both, the pure datastructure with its convenience methods and the dirty str-like thing with its convenience methods.
People don't like it when somebody takes one bag of convenience away from them if they easily can have both. ;-)
That's fine, but I also don't want to have the worry of latent bugs lurking in code either due to implicit type compatibility and so that clashes in this instance.
Better sell it differently. So, let's read on. ;)
Now back to that "practicality beats purity" side of this. Obviously there are lots of APIs out there that take a string as an argument for a file path. And I understand people don't like using `str(path)` or the upcoming/new [`getattr(path, 'path', path)` idiom]( https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.path) to work around the limitations forced upon them by other APIs that only take a `str` because they occasionally forget it. For this point, I have two answers.
Sorry to interrupt your here but you missed one important piece (didn't we agree on selling this better?): people do nasty little string manipulations/analysis/regex with their paths for whatever reasons. Right now, it's easier than ever because, well, because paths are strings. :)
Sure, but that doesn't mean the situation can't be improved. You can also get the string out of Path objects to hack on, but that doesn't mean you should be shipping paths around in your code as strings either. I'm not saying you *can't* use strings to represent a path, just that it isn't the best option available from a higher-level perspective.
Selling pathlib with the "getattr(path, 'path', path)" idiom is not going to help those fellows. I mean, are you serious? It looks ridiculous. :-/
I disagree.
Especially the point that a "path" has a "path" is not really getting into my head. Either a "path" is a "path" or it better be named "file/directory" which has a "path".
path.path vs file.path/dir.path
You need to work on your selling skills, Brett. ;-)
One is still "explicit is better than implicit" and you should have tests to begin with to catch when you forget to convert your `pathlib.Path` objects to `str` as needed. I'm just one of those programmers who's willing to type a bit more for easy-to-read code that's less error-prone within reason (and I obviously view this as reasonable).
But more importantly, why don't you work with the code and/or projects that are forcing you to convert to `str` to start accepting `pathlib` objects as well for those same APIs? If projects start to work on making `pathlib` objects acceptable anywhere a path is accepted then once Python 3.4 is the oldest version of Python that is supported by the project then they can start considering dropping support for `str` as paths in new releases (obviously this also includes dropping support for Python 2 unless people use [pathlib2](https://pypi.python.org/pypi/pathlib2/) <https://pypi.python.org/pypi/pathlib2/%29>). And I fully admit the stdlib is not exempt from a place that needs updating, so for `importlib` I have [opened an issue](http://bugs.python.org/issue26667) to update it to show this isn't just talk on my end (unfortunately there's some unique bootstrapping problems to import where stuff like `sys.path` probably can't be updated since that has to exist before you can import `pathlib` and that sort of thing ripples out, but I'm hoping I can update at least some things and this is a very unique case of potentially needing to stick with `str`). Hopefully if people start asking for `pathlib` support from projects they will add it, eventually leading to an alleviation of the desire to have `pathlib.Path` inherit from `str`.
I think updating the stdlib looks sufficiently simple to start hacking CPython. So, I would be willing to help here once the github repo is up and running. :)
I suspect simple fixes like updating the docs and using `getattr(p, 'path', p)` in the right places will be the kind of quick fixes the GitHub migration will encourage. -Brett
Nice post, Brett. Seems you covered the most important point quite well and also gave an explanation of why people request path->str.
Best, Sven
On 30.03.2016 00:03, Brett Cannon wrote:
I cannot believe that the current representation is so bad and has been for so long and nobody, really nobody, has anything done about it.
[...] pathlib.Path is a better representation of a path than a plain string and that strings on their own are nothing that special in terms of representing a string that demands we defend it on its own grounds as such.
I agree with the first part. I disagree with the second (assuming you mean "representing a path"). Strings are extremely useful for informal working like exploration, debugging etc.
I think most "practicality beats purity" folks don't want that either. They are just bloody lazy. They actually want the benefits of both, the pure datastructure with its convenience methods and the dirty str-like thing with its convenience methods.
People don't like it when somebody takes one bag of convenience away from them if they easily can have both. ;-)
That's fine, but I also don't want to have the worry of latent bugs lurking in code either due to implicit type compatibility and so that clashes in this instance.
Could you give an example here?
Sure, but that doesn't mean the situation can't be improved. You can also get the string out of Path objects to hack on, but that doesn't mean you should be shipping paths around in your code as strings either. I'm not saying you *can't* use strings to represent a path, just that it isn't the best option available from a higher-level perspective.
Let me think of a recommendation here. I personally find the proposed idiom quite cumbersome and ugly but that's just me. However, it is basically not better than "latent bugs lurking in code" due to inheritance. Wrapping stuff up just because we don't know if it's a string or a path is always a bad idea. In pathlibs current form, this would give use a recommendation for application developers: 1) pass a path if you need path methods 2) pass a path.path if you want to do string manipulations 3) don't try both at the same time Would this makes sense?
Selling pathlib with the "getattr(path, 'path', path)" idiom is not going to help those fellows. I mean, are you serious? It looks ridiculous. :-/
I disagree.
That's fine but that's something I would call toiled paper programming. Wrapping things up until it doesn't smell anymore. Seems like here I am the purist. ;-) So, having only one place (the stdlib) dealing with this inconvenience would make things less worse I suppose.
I suspect simple fixes like updating the docs and using `getattr(p, 'path', p)` in the right places will be the kind of quick fixes the GitHub migration will encourage.
I look forward to it. :) Best, Sven
On 30 March 2016 at 10:58, Sven R. Kunze <srkunze@mail.de> wrote:
That's fine, but I also don't want to have the worry of latent bugs lurking in code either due to implicit type compatibility and so that clashes in this instance.
Could you give an example here?
def fix_path(p): # Old code, possibly in a module. # Quite possibly it does a lot more than # simply calling abspath, so it might not # be easy to review return os.path.abspath(p) # Elsewhere p = Path(sys.argv[1]) if not p.is_absolute(): p = fix_path(p) # Now a relative p is no longer a Path object. Later in your code, possibly somewhere # miles away if you rely heavily on paths being interchangeable with strings, you get # an AttributeError when you try to use a Path method on p. Obviously the issue here is trivial to spot. But in real code, it could be a mess of complex logic, with a single os.path call somewhere in an obscure conditional branch. Possibly fix_path has even been updated to work with pathlib, but that one os.path call was accidentally missed and there's no test covering it. Before you say so, of course this is a made up scenario, and of course it'll likely be rare. The question isn't whether such things are common, so much as whether the cost of requiring explicit conversion to string is worth the benefit of avoiding situations like this. And that's a hard judgement call. For casual scripters, and people doing research, the answer is "clearly not". For teaching new users, probably not (new users won't make this sort of mistake, but equally having to teach what a string subclass is relatively early is maybe an issue). For people writing mission critical software, very definitely yes (as a pip developer, I'd hate to switch pip over to a str-subclass path object, whereas switching to a non-subclass one is much more conceivable[1]). Paul [1] Backward compatibility, not wanting another dependency (on the pathlib backport) and the fact that it's change for no actual benefit mean we'd probably not bother anyway, but that's a different decision.
On 24.03.2016 22:06, Koos Zevenhoven wrote:
Or even better, that you could do p"filename.txt", which would give you a Path string object.
That would tie Path objects deeply into the parser and compiler, which I'm not sure is a good idea. Also, it would be stretching the string-prefix concept considerably. Currently, the prefixes just represent different ways of specifying a string -- the end result is still always an instance of str. In this proposal, it would be a different type of object with greatly different behaviour. Not sure whether I feel positively or negatively about this, so +0j. -- Greg
On Mar 25, 2016, at 5:59 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Currently, the prefixes just represent different ways of specifying a string -- the end result is still always an instance of str.
While I’m not sure if the p”” string is a good idea, this isn’t exactly true. b”” creates a bytes object, not a str object. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Sat, Mar 26, 2016 at 8:59 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Also, it would be stretching the string-prefix concept considerably. Currently, the prefixes just represent different ways of specifying a string -- the end result is still always an instance of str. In this proposal, it would be a different type of object with greatly different behaviour.
There's b"..." vs u"...", which do represent entirely different objects, plus f"..." which isn't even a literal at all, but more like a special syntax for an expression (it's more akin to a list display than to a string literal). So there is precedent. If Path objects had universal support in the stdlib *and* significant support in third-party libraries, they could be the one obvious way to do pretty much anything involving paths. At the moment, they're a cute [1] way of getting just slightly more functionality than strings give. The question is: Should syntax precede or follow extensive usage? ChrisA [1] Overloading division doesn't really do anything for you, other than the way it looks similar to the normal path sep. It's really more of a concatenation operation, which would normally be + not /.
On Sat, Mar 26, 2016 at 1:56 AM, Chris Angelico <rosuav@gmail.com> wrote: [...]
If Path objects had universal support in the stdlib *and* significant support in third-party libraries, they could be the one obvious way to do pretty much anything involving paths.
True. I suppose many third-party libraries will just work when the stdlib functions they use [like open(...) and os.path.join(...)] have started accepting Path objects. Then, if a third-party library returns a string, one would need to convert it into a path. That is, unless you just directly pass it to open(...) or something so you don't care. How many reasonably things can a library do with paths without just passing them to stdlib functions? [...]
Should syntax precede or follow extensive usage?
Does anything learned from the evolution of string formatting apply here?
[1] Overloading division doesn't really do anything for you, other than the way it looks similar to the normal path sep. It's really more of a concatenation operation, which would normally be + not /.
I agree that joining paths with / is the least interesting feature of pathlib.Path. Some of this can of course be done with os.path.* too, but here's the list of methods and properties on Path: ['absolute', 'anchor', 'as_posix', 'as_uri', 'chmod', 'cwd', 'drive', 'exists', 'expanduser', 'glob', 'group', 'home', 'is_absolute', 'is_block_device', 'is_char_device', 'is_dir', 'is_fifo', 'is_file', 'is_reserved', 'is_socket', 'is_symlink', 'iterdir', 'joinpath', 'lchmod', 'lstat', 'match', 'mkdir', 'name', 'open', 'owner', 'parent', 'parents', 'parts', 'read_bytes', 'read_text', 'relative_to', 'rename', 'replace', 'resolve', 'rglob', 'rmdir', 'root', 'samefile', 'stat', 'stem', 'suffix', 'suffixes', 'symlink_to', 'touch', 'unlink', 'with_name', 'with_suffix', 'write_bytes', 'write_text'] - Koos
Le 26/03/2016 11:35, Koos Zevenhoven a écrit :
On Sat, Mar 26, 2016 at 1:56 AM, Chris Angelico <rosuav@gmail.com <mailto:rosuav@gmail.com>> wrote: [...]
If Path objects had universal support in the stdlib *and* significant support in third-party libraries, they could be the one obvious way to do pretty much anything involving paths.
True. I suppose many third-party libraries will just work when the stdlib functions they use [like open(...) and os.path.join(...)] have started accepting Path objects. Then, if a third-party library returns a string, one would need to convert it into a path. That is, unless you just directly pass it to open(...) or something so you don't care. How many reasonably things can a library do with paths without just passing them to stdlib functions?
[...]
Should syntax precede or follow extensive usage?
Does anything learned from the evolution of string formatting apply here?
[1] Overloading division doesn't really do anything for you, other than the way it looks similar to the normal path sep. It's really more of a concatenation operation, which would normally be + not /.
I agree that joining paths with / is the least interesting feature of pathlib.Path.
Quite the contratry: - it saves "+" so that "+" still returns a string if needed an not a Path object, preserving compatibility. - on the other hand, "/" is garantied to return a Path object, meaning you can chain the method you listed : Path('test/') / 'foo').method - it ensure your don't have double "/" around :
Path('test/') / 'foo' PosixPath('test/foo')
Path('test') / 'foo' PosixPath('test/foo')
- it will insert "\" if you need it to. Dealing with "\" is really annoying because it's an escape caracter, so it abstract that for you. - it marks clearly in the code that you are doing a concatenation of directories and files, and not something like a prefix, a suffix. You can spot quickly the places in the code where you deal with hierarchy. It's a subtile but quite awesome feature.
Some of this can of course be done with os.path.* too, but here's the list of methods and properties on Path:
['absolute', 'anchor', 'as_posix', 'as_uri', 'chmod', 'cwd', 'drive', 'exists', 'expanduser', 'glob', 'group', 'home', 'is_absolute', 'is_block_device', 'is_char_device', 'is_dir', 'is_fifo', 'is_file', 'is_reserved', 'is_socket', 'is_symlink', 'iterdir', 'joinpath', 'lchmod', 'lstat', 'match', 'mkdir', 'name', 'open', 'owner', 'parent', 'parents', 'parts', 'read_bytes', 'read_text', 'relative_to', 'rename', 'replace', 'resolve', 'rglob', 'rmdir', 'root', 'samefile', 'stat', 'stem', 'suffix', 'suffixes', 'symlink_to', 'touch', 'unlink', 'with_name', 'with_suffix', 'write_bytes', 'write_text']
- Koos
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
I love argparse. I really do. But it's one of those lib forcing me to RTFM everytime I need it, for the last 10 years. We had pathlib added to the stdlib to help with open/os/shutil API and we could do the same with argparse. There are many libs helping with argparsing (clize, click, argh, docopt, etc), and I'd like to talk about the API of one of them, begins. You define the entry point of your script as a regular function you decorate:
import begin @begin.start ... def run(name='Arther', quest='Holy Grail', colour='blue', *knights): ... "tis but a scratch!"
@begin.start will do "if __name__ == '__main__'" for you (alhough you can opt out), and parse the function arguments then generate a argparse parser. When you call the script, it parses the arguments, and call your function with them passed as parameters. It also generates the --help: usage: example.py [-h] [-n NAME] [-q QUEST] [-c COLOUR] [knights [knights ...]] tis but a scratch! positional arguments: knights optional arguments: -h, --help show this help message and exit -n NAME, --name NAME (default: Arther) -q QUEST, --quest QUEST (default: Holy Grail) -c COLOUR, --colour COLOUR (default: blue) Of course, you can define more behavior using other decorators such as repetition, type, checks and sub commands. It's not a huge lib, but's it makes writting script out of the box a breeze. The only problem is : you usually don't want to do a pip install just to run a script. That's why I things it would be great to have something similar in the stdlib. What's more, it's really not going to evolve much (you can't get more old school than cmd), so having it not be updated for years after including it (or something like it) in the stdlib is not an issue. And we can even add some features that would be useful to benefit from the last Python goodness, such as an option to auto-spawn an event loop. This way you can create a quick script with just an async def function. Indeed, the boiler plate of getting the event loop, adding a coroutine and starting the loop is getting annoying very fast when you do a lot of quick things with any aio libs.
(Full disclosure: I'm the developer of clize, another argument parser)
you usually don't want to do a pip install just to run a script
I feel like this should be corrected instead. While I'm not advocating script developers auto-install packages, it'd be nice to help "I just want to ship a single 50-line file" people make better use of dependencies. What if they want to use `requests` (of which the maintainer is, as I understand it, leaning on the side of not including it in the stdlib) ? It's even more likely than wishing for a different argument parser. I'm sure there are other examples. Including another lib in the stdlib (of which there are already 3 alternatives in!) is merely delaying this general issue. -- Yann Kaiser kaiser.yann@gmail.com yann.kaiser@efrei.net +33 6 51 64 01 89 https://github.com/epsy
On 26 March 2016 at 11:21, Yann Kaiser <kaiser.yann@gmail.com> wrote:
I feel like this should be corrected instead. While I'm not advocating script developers auto-install packages, it'd be nice to help "I just want to ship a single 50-line file" people make better use of dependencies.
You may want to look at zipapp (and in general the support that's been in Python since 2.6) for zipped applications. For pure-Python dependencies you can put them in a directory alongside your application script (renamed as __main__.py), add the dependency directory to sys.path, and zip it all up and have a standalone file runnable with Python. Paul
On 26.03.2016 12:02, Michel Desmoulin wrote:
I love argparse. I really do. But it's one of those lib forcing me to RTFM everytime I need it, for the last 10 years.
You may want to look at: https://pypi.python.org/pypi/argh#examples It's dead simple. Each function can become a cmdline command since it just re-uses the functions' signatures. :) Best, Sven
Forget about it. My thunderbird again screwed up my threading view. On 29.03.2016 10:14, Sven R. Kunze wrote:
On 26.03.2016 12:02, Michel Desmoulin wrote:
I love argparse. I really do. But it's one of those lib forcing me to RTFM everytime I need it, for the last 10 years.
You may want to look at: https://pypi.python.org/pypi/argh#examples
It's dead simple. Each function can become a cmdline command since it just re-uses the functions' signatures. :)
Best, Sven _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Chris Angelico wrote:
There's b"..." vs u"...", which do represent entirely different objects,
That's true, but they're not entirely unrelated -- bytes objects are the py3 version of what str was in py2. Also, they're both pretty fundamental to the language, and they're essentially pure representations of data, with not much semantics attached. Path objects, on the other hand, come with a pile of semantics relating to a particular application area -- manipulation of the file system. Moreover, those semantics vary depending on what operating system you're running on. So it's hard so see them as being part of the language the way str and bytes are. This feels like a slippery slope to me. If we include special syntax for pathnames, why shouldn't we have it for dates and times? Regular expressions? URLs? JSON data? SQL queries? XML data? Where do we draw the line?
plus f"..." which isn't even a literal at all, but more like a special syntax for an expression
It still evaluates to a str, though. -- Greg
On Sun, Mar 27, 2016 at 1:13 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
[...] This feels like a slippery slope to me. If we include special syntax for pathnames, why shouldn't we have it for dates and times? Regular expressions? URLs? JSON data? SQL queries? XML data? Where do we draw the line?
OT: To be honest, I do think it feels like URL:s are becoming (or have become) just as important as paths, and that pathlib.Path should in the future work with URLs just like it now works with windows and posix paths. The difference between "http://domain.xyz/" and "C:\\" is not huge. I also think there should be a Python type (stdlib or builtin), which handles JSON objects nicer than dicts do and has its own literal syntax. - Koos
On Sat, Mar 26, 2016 at 6:51 PM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
On Sun, Mar 27, 2016 at 1:13 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
[...] This feels like a slippery slope to me. If we include special syntax for pathnames, why shouldn't we have it for dates and times? Regular expressions? URLs? JSON data? SQL queries? XML data? Where do we draw the line?
OT:
To be honest, I do think it feels like URL:s are becoming (or have become) just as important as paths, and that pathlib.Path should in the future work with URLs just like it now works with windows and posix paths. The difference between "http://domain.xyz/" and "C:\\" is not huge.
Here's some preliminary research (w/ links) that may be of use here: https://www.reddit.com/r/Python/comments/1r7h1t/python_objects_for_working_w... * URLObject (+1 from me) * rdflib.term.URIRef * fs.path Differences between fs paths and URIs: * urlencode IDK why I seem to remember having concluded that it would make sense to store paths as tuples / lists and join by os.path.sep
I also think there should be a Python type (stdlib or builtin), which handles JSON objects nicer than dicts do and has its own literal syntax.
collections.OrderedDict almost works here, except that .keys() and .values() must be casted to e.g. lists (because otherwise the comparison is between KeysView / ValuesView ~iterators) to_json = _repr_json_ # "IPython repr method examples" # https://gist.github.com/westurner/be22dba8110be099a35e#file-ordereddefaultdi... Then, if Path is not a subclass of str (or, py2 __builtin__.unicode, as in https://github.com/jaraco/path.py/blob/master/path.py ), comparisons between JSON-ified path objects will fail.
- Koos
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Koos Zevenhoven writes:
To be honest, I do think it feels like URL:s are becoming (or have become) just as important as paths, and that pathlib.Path should in the future work with URLs just like it now works with windows and posix paths.
+1 ... to the concept, but: Is there a semantic difference between a RFC 3986 path component (Section 3.3), and a pathlib path? If there is, this could be a difficult project. (May as well start now, though!)
On 27.03.2016 19:36, Stephen J. Turnbull wrote:
Koos Zevenhoven writes:
To be honest, I do think it feels like URL:s are becoming (or have become) just as important as paths, and that pathlib.Path should in the future work with URLs just like it now works with windows and posix paths.
+1
... to the concept, but: Is there a semantic difference between a RFC 3986 path component (Section 3.3), and a pathlib path? If there is, this could be a difficult project. (May as well start now, though!)
I would interpret the path of pathlib as a subset of functionality of path of a URI. """ The path component contains data, usually organized in hierarchical form [...] serves to identify a resource within the scope of the URI's scheme and naming authority (if any). """ URI's scheme for path of pathlib could implicitly be: "file://" Relative paths of pathlib are a subset of relative references (Section 4.1). The only practical issue, I can think of is, how to distinguish (in the sense of avoiding hidden bugs) file paths and url paths. And name clashes when communicating with your fellow programmers: "I got that url path working" "the url path or the url path path?" Best, Sven
On 27.03.2016 00:51, Koos Zevenhoven wrote:
On Sun, Mar 27, 2016 at 1:13 AM, Greg Ewing <greg.ewing@canterbury.ac.nz <mailto:greg.ewing@canterbury.ac.nz>> wrote:
[...] This feels like a slippery slope to me. If we include special syntax for pathnames, why shouldn't we have it for dates and times? Regular expressions? URLs? JSON data? SQL queries? XML data? Where do we draw the line?
Just because you want to draw a line does not necessarily mean, we want to draw one. ;) I don't think we (as the Python community) can allow ourselves to stand still. We need to have tools that are simple and available. If YAML is the new standard for everything, Python should move. If $ before variable names are the new standard (I hope they never will again), Python should move. I understand that there are preservers who don't want anything to change. So, compromises will be necessary.
OT:
To be honest, I do think it feels like URL:s are becoming (or have become) just as important as paths, and that pathlib.Path should in the future work with URLs just like it now works with windows and posix paths. The difference between "http://domain.xyz/" and "C:\\" is not huge. I also think there should be a Python type (stdlib or builtin), which handles JSON objects nicer than dicts do and has its own literal
That even occurred to me after we talked about the p-string (mainly because I am working in this field, so I basically need both file paths and URIs). I agree with the URI/IRI idea. It feels natural and sometimes it is necessary to extract specific parts from an URL according to RFC 3986 or 3987. So, +1 from me. Just for the record: "Path" might not be the most correct wording. There is a "file://" scheme which identifies locally located files. So, paths are basically a subset of URLs speaking functionality-wise. Thus, a better/more generic name would be "URL", "URI", "Link" or the like in order to avoid confusing of later generations. However, I think I could live with Path. Another thought: requesting URLs. Basically the same as p'/etc/hosts'.write_text(secret). It's really important to have a dead simple library which is able to work with URLs. So, if I could do: p'https://mysite.com/{page}'.get() that'll be awesome. Best, Sven
On 29 March 2016 at 09:44, Sven R. Kunze <srkunze@mail.de> wrote:
I don't think we (as the Python community) can allow ourselves to stand still. We need to have tools that are simple and available. If YAML is the new standard for everything, Python should move. If $ before variable names are the new standard (I hope they never will again), Python should move.
I understand that there are preservers who don't want anything to change. So, compromises will be necessary.
I agree 100% that Python should not stand still. However, every addition to the core language involves a backward compatibility cost that must be very carefully weighed. If we built XML into the language back when that was the standard, then we'd now be looking at adding YAML, but we wouldn't be able to remove XML without significant pain. So a conservative approach *to core language syntax* is entirely reasonable. Having said that, Python has a very good means of adding new *functionality* in the form of modules, and an very good path for gradually standardising modules in the form of personal library -> PyPI package -> stdlib package. It seems to me that this route is appropriate here. At the moment, pathlib has been included as a stdlib library, based on the experience gained from the various PyPI packages developed beforehand. There still seem to be some reservations in the community over the adoption of pathlib, and the first thing we need to do is to resolve those, before we even think about promoting the library to syntax (after all, regular expressions have been round *forever*, and still aren't built in syntax, in spite of the precedents in Perl, Javascript, and other languages - why are Path objects so much more deserving of a fast track?) I'm much more in favour of discussions on how to address the perceived shortcomings of pathlib over alternatives like path.py and pylib's path object. If it's all about subclassing str, then let's by all means reopen that debate - making Path a str subclass is something that could be done as a stdlib change if the decision made in the original PEP was agreed to be flawed (I'm not sure it was, personally, but I'm open to a discussion on that, as the need to convert to and from strings certainly is a nuisance).
To be honest, I do think it feels like URL:s are becoming (or have become) just as important as paths, and that pathlib.Path should in the future work with URLs just like it now works with windows and posix paths. The difference between "http://domain.xyz/" and "C:\\" is not huge. I also think there should be a Python type (stdlib or builtin), which handles JSON objects nicer than dicts do and has its own literal
So again, if you have personal code you find cleaner and more usable for this job, publish it on PyPI, publicise it and gain support for it, and then propose it for stdlib inclusion. This also has the advantage of being useful for Python 3.4/3.5 users (and you can even support Python 2.7 if you wish). I've no idea how we would ensure that a built in syntax for URLs was correctly designed without having got design experience with the feature as a library module. Paul
On 29.03.2016 11:09, Paul Moore wrote:
I don't think we (as the Python community) can allow ourselves to stand still. We need to have tools that are simple and available. If YAML is the new standard for everything, Python should move. If $ before variable names are the new standard (I hope they never will again), Python should move.
I understand that there are preservers who don't want anything to change. So, compromises will be necessary. I agree 100% that Python should not stand still. However, every addition to the core language involves a backward compatibility cost
On 29 March 2016 at 09:44, Sven R. Kunze <srkunze@mail.de> wrote: that must be very carefully weighed. If we built XML into the language back when that was the standard, then we'd now be looking at adding YAML, but we wouldn't be able to remove XML without significant pain.
So a conservative approach *to core language syntax* is entirely reasonable.
Having said that, Python has a very good means of adding new *functionality* in the form of modules, and an very good path for gradually standardising modules in the form of personal library -> PyPI package -> stdlib package.
It seems to me that this route is appropriate here. At the moment, pathlib has been included as a stdlib library, based on the experience gained from the various PyPI packages developed beforehand. There still seem to be some reservations in the community over the adoption of pathlib, and the first thing we need to do is to resolve those, before we even think about promoting the library to syntax (after all, regular expressions have been round *forever*, and still aren't built in syntax, in spite of the precedents in Perl, Javascript, and other languages - why are Path objects so much more deserving of a fast track?)
Age is no indicator for "deserving". Because you are asking specifically for regexes, I might be able to give an explanation. Regular expressions have, let's say, the tendency to grow. It's more or less code, so they grow naturally. Readability, however, diminishes extremely fast with regexes when compared to regular code. So, speaking from experience (and the moaning and groaning of the fellow team members), they might have been a good idea back then, nowadays we see they disadvantages more clearly. In short, we have far less usage of regular expressions than we see for paths (from what I can see). If this speaks _for_ paths, I cannot say. But waiting 20 years for this to happen might be a bit long. And hey, I wouldn't mind a builtin type for regular expressions either if this makes things considerably easier. :)
I'm much more in favour of discussions on how to address the perceived shortcomings of pathlib over alternatives like path.py and pylib's path object. If it's all about subclassing str, then let's by all means reopen that debate - making Path a str subclass is something that could be done as a stdlib change if the decision made in the original PEP was agreed to be flawed (I'm not sure it was, personally, but I'm open to a discussion on that, as the need to convert to and from strings certainly is a nuisance).
To be honest, I do think it feels like URL:s are becoming (or have become) just as important as paths, and that pathlib.Path should in the future work with URLs just like it now works with windows and posix paths. The difference between "http://domain.xyz/" and "C:\\" is not huge. I also think there should be a Python type (stdlib or builtin), which handles JSON objects nicer than dicts do and has its own literal So again, if you have personal code you find cleaner and more usable for this job, publish it on PyPI, publicise it and gain support for it, and then propose it for stdlib inclusion. This also has the advantage of being useful for Python 3.4/3.5 users (and you can even support Python 2.7 if you wish).
I've no idea how we would ensure that a built in syntax for URLs was correctly designed without having got design experience with the feature as a library module.
Are you addressing me? The way you quoted makes it seem as if I had written those lines. This is not true. Btw. I totally agree with the experience-proven way: module -> stdlib module -> syntax. Btw2. we already have design experience in this field: requests + URLObject. Not sure if you've ever used them, but they are amazing (mainly they work and don't get into your way). Best, Sven
On 29 March 2016 at 10:35, Sven R. Kunze <srkunze@mail.de> wrote:
Age is no indicator for "deserving".
Absolutely. But you do need experience over time to *know* if something is deserving. Many other languages have builtin regexes - Python proved (by not rushing to make them built in) that it wasn't needed.
Are you addressing me? The way you quoted makes it seem as if I had written those lines. This is not true.
Bah. My mail client (or a previous post) may have messed up the quoting - it seemed to me that what I was replying to was your original comment. My apologies if it was someone else's.
Btw. I totally agree with the experience-proven way: module -> stdlib module -> syntax. Btw2. we already have design experience in this field: requests + URLObject. Not sure if you've ever used them, but they are amazing (mainly they work and don't get into your way).
I've used requests - it is indeed excellent (although I don't believe it treats URLs as anything other than strings - if it does, I've never used that part of it). I've not used URLObject, I'll take a look at that (although I don't have much need for more than casual URL parsing in my code). Paul
On Tue, Mar 29, 2016, at 05:35, Sven R. Kunze wrote:
Because you are asking specifically for regexes, I might be able to give an explanation.
Regular expressions have, let's say, the tendency to grow. It's more or less code, so they grow naturally. Readability, however, diminishes extremely fast with regexes when compared to regular code. ... And hey, I wouldn't mind a builtin type for regular expressions either if this makes things considerably easier. :)
Is it time again to bring up my regex AST idea?
On 29.03.2016 15:33, Random832 wrote:
On Tue, Mar 29, 2016, at 05:35, Sven R. Kunze wrote:
Because you are asking specifically for regexes, I might be able to give an explanation.
Regular expressions have, let's say, the tendency to grow. It's more or less code, so they grow naturally. Readability, however, diminishes extremely fast with regexes when compared to regular code. ... And hey, I wouldn't mind a builtin type for regular expressions either if this makes things considerably easier. :) Is it time again to bring up my regex AST idea?
Own thread? ;-)
On 3/26/2016 7:13 PM, Greg Ewing wrote:
Path objects, on the other hand, come with a pile of semantics relating to a particular application area -- manipulation of the file system. Moreover, those semantics vary depending on what operating system you're running on. So it's hard so see them as being part of the language the way str and bytes are.
This feels like a slippery slope to me. If we include special syntax for pathnames, why shouldn't we have it for dates and times? Regular expressions? URLs? JSON data? SQL queries? XML data? Where do we draw the line?
PEP 501 tries to address these issues, with i-strings. There were other proposals on python-ideas relating to arbitrary string prefixes when PEPs 498 and 501 were being discussed. Eric.
On Fri, 25 Mar 2016 at 16:50 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 24.03.2016 22:06, Koos Zevenhoven wrote:
Or even better, that you could do p"filename.txt", which would give you a Path string object.
That would tie Path objects deeply into the parser and compiler, which I'm not sure is a good idea.
I'll go one step further than Greg and say that it's not a good idea. To make something like this work you have two options. One is you do what importlib does and very carefully construct it to only rely on built-in modules (and if you look at https://hg.python.org/cpython/file/default/Lib/pathlib.py you will notice that is no where near true ATM). Two, is Python gets a two-tier syntax system for before and after pathlib is imported and available. That would mean that all the dependencies of pathlib would need to not use this syntax until it's bootstrapped in. If you notice, both solutions are extremely painful to implement and radiate well past the code in pathilb.py and would potentially become hard to maintain.
Also, it would be stretching the string-prefix concept considerably. Currently, the prefixes just represent different ways of specifying a string -- the end result is still always an instance of str. In this proposal, it would be a different type of object with greatly different behaviour.
Yep, that too.
Not sure whether I feel positively or negatively about this, so +0j.
I'm sure how I feel :) -1. I also want to mention two things. One, pathlib.path is a thing now and something most people are probably not aware of as an alternative to doing `str(path)`: https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.path . And two, wrapping path inputs in `pathlib.Path(path)` is a small price to pay for a nicer API. As time passes it will be less and less of a thing as APIs will start to come up that explicitly accept either, and then eventually they will only accept pathlib. Plus if you do it at the edges of your own API then you can do the conversion early and only get out the string as necessary later on as needed (much like how you handle text/binary data boundaries). I get the desire of succinctness as Python spoils us for that, but much like __future__ statements it's really a small price to pay in the grand scheme of things in order to migrate to a better API.
I'm not going to say how I feel. But I noticed one thing in this thread. These seems to be a growing difference between people who use Python for "serious" programming and those who consider themselves testers, QA integrators and the like (more power to them!). Maybe both groups can occasionally try to walk in the others' shoes. --Guido (mobile)
On 26.03.2016 20:21, Guido van Rossum wrote:
I'm not going to say how I feel. But I noticed one thing in this thread. These seems to be a growing difference between people who use Python for "serious" programming and those who consider themselves testers, QA integrators and the like (more power to them!). Maybe both groups can occasionally try to walk in the others' shoes.
+1
On Sat, Mar 26, 2016 at 9:10 PM, Brett Cannon <brett@python.org> wrote:
On Fri, 25 Mar 2016 at 16:50 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 24.03.2016 22:06, Koos Zevenhoven wrote:
Or even better, that you could do p"filename.txt", which would give you a Path string object.
That would tie Path objects deeply into the parser and compiler, which I'm not sure is a good idea.
I'll go one step further than Greg and say that it's not a good idea. To make something like this work you have two options. One is you do what importlib does and very carefully construct it to only rely on built-in modules (and if you look at https://hg.python.org/cpython/file/default/Lib/pathlib.py you will notice that is no where near true ATM). Two, is Python gets a two-tier syntax system for before and after pathlib is imported and available. That would mean that all the dependencies of pathlib would need to not use this syntax until it's bootstrapped in. If you notice, both solutions are extremely painful to implement and radiate well past the code in pathilb.py and would potentially become hard to maintain.
As I understand it, this could be implemented by making the compiler essentially turn p"/path/to/whatever" into something like _make_path("/path/to/whatever"), where _make_path would be builtin and do something like this. def _make_path(str_path): import pathlib return pathlib.Path(str_path) Unless of course Path could be modified to import its dependencies on demand and put in builtins. Am I missing something crucial?
I also want to mention two things. One, pathlib.path is a thing now and something most people are probably not aware of as an alternative to doing `str(path)`: https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.path .
I assume you meant to type pathlib.Path.path, so that Path("...").path == str(Path("...")). That's a good start, and I'm looking forward to Serhiy's patch for making the stdlib accept Paths. But if Path will not subclass str, we also need new stdlib functions that *return* Paths. - Koos
On Sun, Mar 27, 2016 at 1:23 AM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
On Sat, Mar 26, 2016 at 9:10 PM, Brett Cannon <brett@python.org> wrote:
[...]
I also want to mention two things. One, pathlib.path is a thing now and
something most people are probably not aware of as an alternative to doing `str(path)`: https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.path .
I assume you meant to type pathlib.Path.path, so that Path("...").path == str(Path("...")). That's a good start, and I'm looking forward to Serhiy's patch for making the stdlib accept Paths. But if Path will not subclass str, we also need new stdlib functions that *return* Paths.
Well, just to reply to myself, here's a slightly crazy idea, which I'll mention before I realize that it's a bad idea: What if there was a another class, say StringPath, that inherits from both str and Path, which wraps another instance of Path, but is also a str. When you call its Path methods, it would delegate them to the wrapped Path object so that functions that now return paths as plain str could in future versions start returning that type? - Koos
On Wed, Mar 30, 2016 at 6:18 PM, Paul Moore <p.f.moore@gmail.com> wrote:
Of course this doesn't address functions that *return* paths (as strings). There the caller has to wrap the return value in Path(). Or the function changes to return Path objects, which won't be backward compatible (whether that matters depends on what the code is).
So, below was a though of how to deal with this: StringPath, which is both a str and a Path (and/or perhaps PurePath). On Sun, Mar 27, 2016 at 2:33 AM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
Well, just to reply to myself, here's a slightly crazy idea, which I'll mention before I realize that it's a bad idea:
What if there was a another class, say StringPath, that inherits from both str and Path, which wraps another instance of Path, but is also a str. When you call its Path methods, it would delegate them to the wrapped Path object so that functions that now return paths as plain str could in future versions start returning that type?
While I still think it's a crazy idea, I'm not 100% convinced that it is the wrong thing to do, because there are already many kinds of Path objects. So a wrote a super hacky implementation of StringPath. Instances of something like this could be returned from functions that now return paths in strings. Here's a little demo with the toy implementation: >>> p = StringPath("foo/bar/baz") >>> p StringPath(PosixPath('foo/bar/baz')) >>> isinstance(p, pathlib.Path) True >>> isinstance(p, str) True >>> str(p) 'foo/bar/baz' >>> p + 'hello' 'foo/bar/bazhello' >>> p / 'hello' PosixPath('foo/bar/baz/hello') >>> 'hello' / p PosixPath('hello/foo/bar/baz') >>> p.split('r') ['foo/ba', '/baz'] >>> pathlib.Path(p) PosixPath('foo/bar/baz') So it's a str, but as soon as you do something Path-like with it, it gives you a Path back. But for anyone who thinks they have a string, it's a string, except for its repr. -Koos P.S. The implementation I used here is actually completely ridiculous, but if you want to look at it, it's here: https://gist.github.com/k7hoven/defb7f2eb9ccd9dbd0be0063a475058e
On Sun, Mar 27, 2016 at 1:23 AM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
On Sat, Mar 26, 2016 at 9:10 PM, Brett Cannon <brett@python.org> wrote:
I also want to mention two things. One, pathlib.path is a thing now and something most people are probably not aware of as an alternative to doing `str(path)`: https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.path .
I assume you meant to type pathlib.Path.path, so that Path("...").path == str(Path("...")). That's a good start, and I'm looking forward to Serhiy's patch for making the stdlib accept Paths. But if Path will not subclass str, we also need new stdlib functions that *return* Paths.
Actually, now that .path is not out yet, would it make sense to call it Path.str or Path.strpath instead, and introduce the same thing on DirEntry and guarantee a str (instead of str or bytes as DirEntry.path now does)? Maybe that would lead to fewer broken implementations in third-party libraries too? - Koos
On Sat, 26 Mar 2016 at 16:23 Koos Zevenhoven <k7hoven@gmail.com> wrote:
On Sat, Mar 26, 2016 at 9:10 PM, Brett Cannon <brett@python.org> wrote:
On Fri, 25 Mar 2016 at 16:50 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 24.03.2016 22:06, Koos Zevenhoven wrote:
Or even better, that you could do p"filename.txt", which would give you a Path string object.
That would tie Path objects deeply into the parser and compiler, which I'm not sure is a good idea.
I'll go one step further than Greg and say that it's not a good idea. To make something like this work you have two options. One is you do what importlib does and very carefully construct it to only rely on built-in modules (and if you look at https://hg.python.org/cpython/file/default/Lib/pathlib.py you will notice that is no where near true ATM). Two, is Python gets a two-tier syntax system for before and after pathlib is imported and available. That would mean that all the dependencies of pathlib would need to not use this syntax until it's bootstrapped in. If you notice, both solutions are extremely painful to implement and radiate well past the code in pathilb.py and would potentially become hard to maintain.
As I understand it, this could be implemented by making the compiler essentially turn p"/path/to/whatever" into something like _make_path("/path/to/whatever"), where _make_path would be builtin and do something like this.
def _make_path(str_path): import pathlib return pathlib.Path(str_path)
Unless of course Path could be modified to import its dependencies on demand and put in builtins. Am I missing something crucial?
The trick is whether that import statement will work when you call it. Import is currently the only bit of syntax that relies on Python code in CPython to work and making that happen took a lot of work. If you add in this concept of p-strings then suddenly we have two pieces of syntax that require Python code to work and on top of it p-strings would depend on import but also that we would then have to make sure to not have import depend on p-strings. And on top of it we would have to freeze pathlib and all of its dependencies which may or may not be preferable. My point is that it's not straight-forward and this proposal will have to consider some technical difficulties involved with it if it happens to go forward. -Brett
I also want to mention two things. One, pathlib.path is a thing now and something most people are probably not aware of as an alternative to doing `str(path)`: https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.path .
I assume you meant to type pathlib.Path.path, so that Path("...").path == str(Path("...")). That's a good start, and I'm looking forward to Serhiy's patch for making the stdlib accept Paths. But if Path will not subclass str, we also need new stdlib functions that *return* Paths.
- Koos
On 27.03.2016 19:50, Brett Cannon wrote:
The trick is whether that import statement will work when you call it. Import is currently the only bit of syntax that relies on Python code in CPython to work and making that happen took a lot of work. If you add in this concept of p-strings then suddenly we have two pieces of syntax that require Python code to work and on top of it p-strings would depend on import but also that we would then have to make sure to not have import depend on p-strings. And on top of it we would have to freeze pathlib and all of its dependencies which may or may not be preferable.
My point is that it's not straight-forward and this proposal will have to consider some technical difficulties involved with it if it happens to go forward.
What solution can you propose to solve this? Can this procedure be generalized and smoothed in order to reduce that "lot of work" and to some extend enable a simpler language evolution? Or would it be better to leave the bootstrapping infrastructure alone? Best, Sven
On Fri, Mar 25, 2016, at 17:59, Greg Ewing wrote:
On 24.03.2016 22:06, Koos Zevenhoven wrote:
Or even better, that you could do p"filename.txt", which would give you a Path string object.
That would tie Path objects deeply into the parser and compiler, which I'm not sure is a good idea.
It's not obviously _not_ a good idea. Are float and complex more "worthy" of being tied into the parser and compiler than Path?
On 03/25/2016 03:03 PM, Greg Ewing wrote:
Koos Zevenhoven wrote:
Or even better, that you could do p"filename.txt", which would give you a Path string object. Has this been discussed?
Also, this would give a privileged place to one particular path library. How do we choose which one?
Um, the one in the stdlib? ;) -- ~Ethan~
Ethan Furman wrote:
Also, this would give a privileged place to one particular path library. How do we choose which one?
Um, the one in the stdlib? ;)
Not everyone likes that one though. As things stand, you can easily substitute one of your own that's more to your liking. That won't be true if path objects get baked into the language syntax. -- Greg
On 03/26/2016 03:55 PM, Greg Ewing wrote:
Ethan Furman wrote:
Also, this would give a privileged place to one particular path library. How do we choose which one?
Um, the one in the stdlib? ;)
Not everyone likes that one though. As things stand, you can easily substitute one of your own that's more to your liking. That won't be true if path objects get baked into the language syntax.
Yeah, I'm one of those (primarily because it's a pain to work with). My point was just that baked-in syntax would naturally go with stdlib objects (and it would still not preclude using your preferred library). -- ~Ethan~
On 26.03.2016 23:55, Greg Ewing wrote:
Ethan Furman wrote:
Also, this would give a privileged place to one particular path library. How do we choose which one?
Um, the one in the stdlib? ;)
Not everyone likes that one though. As things stand, you can easily substitute one of your own that's more to your liking. That won't be true if path objects get baked into the language syntax.
I'd rather submit bug reports and help improving a commonly used library than re-inventing the wheel. If the help is not wanted, that's another thing. But I consider working together important. Furthermore, file paths are an essential part of programming even of todays operating systems. As are strings, numbers btw. Best, Sven
On Mar 25, 2016, at 13:20, Koos Zevenhoven <k7hoven@gmail.com> wrote:
So, let's start a new thread about how to deal with pathlib.Path objects, involving how/whether to convert between str and Path.
On Fri, Mar 25, 2016 at 7:14 PM, Sven R. Kunze <srkunze@mail.de> wrote:
On 24.03.2016 22:06, Koos Zevenhoven wrote:
Or even better, that you could do p"filename.txt", which would give you a Path string object. Has this been discussed?
Interesting. I had this thought half a year ago. Quite recently I mentioned this to Andrew in a private conversation.
Yeah, I first thought about this at some point last summer when I had written the little np package for creating numpy arrays as np[3,4,5] (which I am planning to polish and maybe advertise), probably also inspired by the f-string discussions on this list. And now that you asked me about dealing with Paths...
Anyway, as you say, if different people have thought about this independently, maybe its worth discussing.
p'/etc/hosts' would make a perfect path which subclassed from str.
Having Path types subclass str would solve 90% of the problems with pathlib. Adding p-strings raises that to 91%; I don't think it's enough extra win to be worth it. The main reason not to use pathlib today is that almost nothing takes Path objects. A handful of functions (open, stat, access) have methods on the Path objects. But if you want to zlib.open a Path, or add one to a ZipFile, or pass one to json.load or etree.parse, or use it with any third-party library, you have to convert to str manually. And in the opposite direction, paths you get from sys.argv or WSGI or anywhere else are str, not Path, so you have to construct Path objects manually too. So, attempting to use Path all over the place means converting back and forth to str all over the place. And the benefit of having Path objects is so tiny that it's not worth the cost. Path seems to have been designed for a statically-typed language with implicit conversions. In fact, it's very similar to the Boost/C++ design. But that design works for C++ because every time you pass a path to a function that wants a string, or try to store a string in a path-typed variable, the compiler sees what you're doing, sees that the types are unrelated, and looks for a conversion operator or constructor so it can insert an implicit call. That doesn't, and can't, happen in Python. As mentioned in one of the other replies, pathlib's main PyPI predecessor does subclass str, and that makes it a lot more usable.
The p-string idea has not been discussed in the PEP I think. The subclassing thing however was and I think its resolution was a mistake. The explanation, at least from my point of view, is a bit weird.
I assume there are good arguments against making Path inherit from str. Unfortunately, those arguments don't seem to appear in the PEP. They really should. But, given that they don't, anyone proposing a change like this really should go back and look at the archived previous discussions, summarize and link the old arguments, and offer counter-arguments.
Anyway, I think the question of whether Path should subclass str is partly separate from the p-string concept. After all, b'byte strings' are not subclassed from str either.
Well, adding p-strings but _not_ subclassing str seems to add almost nothing. It makes it easier to construct literal Path objects that you still can't use anywhere, so you end up writing "spam(str(p'eggs.txt'))" instead of just "spam('eggs.txt')"? But I agree that they can be separated, for the opposite reason: a str-subclassing Path is useful with or without p-strings.
Le 26/03/2016 11:44, Andrew Barnert via Python-ideas a écrit :
On Mar 25, 2016, at 13:20, Koos Zevenhoven <k7hoven@gmail.com> wrote:
So, let's start a new thread about how to deal with pathlib.Path objects, involving how/whether to convert between str and Path.
On Fri, Mar 25, 2016 at 7:14 PM, Sven R. Kunze <srkunze@mail.de> wrote:
On 24.03.2016 22:06, Koos Zevenhoven wrote:
Or even better, that you could do p"filename.txt", which would give you a Path string object. Has this been discussed?
Interesting. I had this thought half a year ago. Quite recently I mentioned this to Andrew in a private conversation.
Yeah, I first thought about this at some point last summer when I had written the little np package for creating numpy arrays as np[3,4,5] (which I am planning to polish and maybe advertise), probably also inspired by the f-string discussions on this list. And now that you asked me about dealing with Paths...
Anyway, as you say, if different people have thought about this independently, maybe its worth discussing.
p'/etc/hosts' would make a perfect path which subclassed from str.
Having Path types subclass str would solve 90% of the problems with pathlib. Adding p-strings raises that to 91%; I don't think it's enough extra win to be worth it.
The main reason not to use pathlib today is that almost nothing takes Path objects. A handful of functions (open, stat, access) have methods on the Path objects. But if you want to zlib.open a Path, or add one to a ZipFile, or pass one to json.load or etree.parse, or use it with any third-party library, you have to convert to str manually. And in the opposite direction, paths you get from sys.argv or WSGI or anywhere else are str, not Path, so you have to construct Path objects manually too. So, attempting to use Path all over the place means converting back and forth to str all over the place. And the benefit of having Path objects is so tiny that it's not worth the cost.
Path seems to have been designed for a statically-typed language with implicit conversions. In fact, it's very similar to the Boost/C++ design. But that design works for C++ because every time you pass a path to a function that wants a string, or try to store a string in a path-typed variable, the compiler sees what you're doing, sees that the types are unrelated, and looks for a conversion operator or constructor so it can insert an implicit call. That doesn't, and can't, happen in Python.
As mentioned in one of the other replies, pathlib's main PyPI predecessor does subclass str, and that makes it a lot more usable.
The p-string idea has not been discussed in the PEP I think. The subclassing thing however was and I think its resolution was a mistake. The explanation, at least from my point of view, is a bit weird.
I assume there are good arguments against making Path inherit from str. Unfortunately, those arguments don't seem to appear in the PEP. They really should. But, given that they don't, anyone proposing a change like this really should go back and look at the archived previous discussions, summarize and link the old arguments, and offer counter-arguments.
Anyway, I think the question of whether Path should subclass str is partly separate from the p-string concept. After all, b'byte strings' are not subclassed from str either.
Well, adding p-strings but _not_ subclassing str seems to add almost nothing. It makes it easier to construct literal Path objects that you still can't use anywhere, so you end up writing "spam(str(p'eggs.txt'))" instead of just "spam('eggs.txt')"?
+1 to everything. Remember that because of this, all projects I work on still use path.py. We prefer to add an additional dependency than to deal with the casting, because it spreads everywhere. And we are not a conservative crew, some projects use Python 3.5 and asyncio. Although honestly, p'/foo/bar'.write_text(stuff) would be awesome at some many levels. Can you imagine how scripting becomes easier because of this ? Sys admin ? Shell sessions ?
But I agree that they can be separated, for the opposite reason: a str-subclassing Path is useful with or without p-strings.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Mar 25, 2016, at 13:20, Koos Zevenhoven <k7hoven@gmail.com> wrote:
So, let's start a new thread about how to deal with pathlib.Path objects, involving how/whether to convert between str and Path
As a different point: If we _don't_ either subclass str or find some way to make things magically work, is there any other way to start getting more uptake on pathlib? This may only be anecdotal, but from what I can tell, nobody is using it, because everyone who tries is put off by the need to convert back and forth everywhere. Last year, everyone agreed that it would be good if at least the stdlib accepted paths everywhere, which might prompt third party libs to start doing the same. But nobody's started writing patches to do that. And I'm not sure we'd want it to happen in an uncoordinated way anyway, since there are at least four different ways to do it, and if we pick among them arbitrarily for different parts of the stdlib, we'll have a huge mess, and possibly behavioral inconsistencies. The four ways I can think of are (in every function that currently takes a "path: str" argument, and should now take a "path: str | Path"): * path = str(path) * path = Path(path) * if isinstance(path, Path): ... else: ... * try: f = path.open('w') except AttributeError: open(path, 'w') It's also worth noting that all but the first require every module to depend on Path. Including C modules. And modules in the bootstrap. But the first version makes a bad guide for third-party code, because a lot of third-party code is dual-version/single-source libs, and you definitely don't want to unconditionally call str on a path argument that may be Unicode in 2.7 (or to unconditionally call a six-style Unicode function on a path argument that may be str in 2.7 on Linux). So, assuming we all want a future in which pathlib is actually used by people, and assuming str subclassing is out and there's no other magic bullet, how do we get there from here?
I have been watching this thread, and read the docs and the PEP, and... What is pathlib even for? Yes, its there to be an object oriented abstraction of a file system. Why? Why do I want this? Am I alone in wondering why this exists? Is it even worth improving? On 3/26/2016 06:59, Andrew Barnert via Python-ideas wrote:
On Mar 25, 2016, at 13:20, Koos Zevenhoven <k7hoven@gmail.com> wrote:
So, let's start a new thread about how to deal with pathlib.Path objects, involving how/whether to convert between str and Path As a different point:
If we _don't_ either subclass str or find some way to make things magically work, is there any other way to start getting more uptake on pathlib? This may only be anecdotal, but from what I can tell, nobody is using it, because everyone who tries is put off by the need to convert back and forth everywhere.
Last year, everyone agreed that it would be good if at least the stdlib accepted paths everywhere, which might prompt third party libs to start doing the same. But nobody's started writing patches to do that. And I'm not sure we'd want it to happen in an uncoordinated way anyway, since there are at least four different ways to do it, and if we pick among them arbitrarily for different parts of the stdlib, we'll have a huge mess, and possibly behavioral inconsistencies.
The four ways I can think of are (in every function that currently takes a "path: str" argument, and should now take a "path: str | Path"):
* path = str(path) * path = Path(path) * if isinstance(path, Path): ... else: ... * try: f = path.open('w') except AttributeError: open(path, 'w')
It's also worth noting that all but the first require every module to depend on Path. Including C modules. And modules in the bootstrap. But the first version makes a bad guide for third-party code, because a lot of third-party code is dual-version/single-source libs, and you definitely don't want to unconditionally call str on a path argument that may be Unicode in 2.7 (or to unconditionally call a six-style Unicode function on a path argument that may be str in 2.7 on Linux).
So, assuming we all want a future in which pathlib is actually used by people, and assuming str subclassing is out and there's no other magic bullet, how do we get there from here? _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Le 26/03/2016 13:43, Alexander Walters a écrit :
I have been watching this thread, and read the docs and the PEP, and...
What is pathlib even for?
Yes, its there to be an object oriented abstraction of a file system. Why? Why do I want this?
Because it makes a very common task (FS manipulation) much more natural. This is the same reason people like request over urllib or like @decorator over func = decorator(func) or unpacking vs manual item extract. In a directory of music files, you ave mp3 files you converted to other formats. Now you want to remove those conversion. Your task is to find all files having the same name as the ones with the mp3 ones, but with a different extension, remove them, then list the mp3 absolument path in a text file. This example has baked in a lot of the tasks you do when you use Python as a scripting language at the core of you job such as if it's your glue language, if you are a tester or for sysadmin tasks. But it helps also everybody that once in a file does a one off script. The traditional approach: import os import glob import sys root = os.path.abspath(sys.argv[1]) playlist = os.path.join(root, 'playlist.m3u8') with open(playlist, 'w') as f: for path in glob.glob(os.path.join(root, '*.mp3')): name, ext = os.path.splitext(os.path.basename(path)) for to_remove in glob.glob(os.path.join(root, name + '.*')): if not to_remove.endswith('mp3'): os.remove(to_remove) f.write(os.path.join(root, path) + "\n") Now with pathlib you don't have to wonder about whether the feature you are looking for is on "os", "os.path" or "glob" or "open". You don't have to deal the file opening for such a small script. You don't have to switch between functions and methods all the time and have to choose between nested function calls or intermediary variables. The pathlib version is way easier to figure out without knowing the stdlib by heart, it is one line shorter and one level of indent less. And you can discover it all from iPython with ".<tab>": import sys import pathlib root = pathlib.Path(sys.argv[1]) files = [] for path in root.glob('*.mp3'): name = str(path).replace(path.suffix, '.*') files.append(str(path.absolute())) for to_remove in root.glob(name): if to_remove.suffix != ".mp3": to_remove.unlink() (root / 'playlist.m3u8').write_text('\n'.join(files)) And this true while pathlib is a half backed written lib, since the competition (that existed before pathlib and that pathlib failed to inspire from), can do even shorter, easier and cleaner: import sys import path root = path.Path(sys.argv[1]).realpath() files = [] for p in root.glob('*.mp3'): name = p.namebase + '.*' files.append(p) for to_remove in root.glob(name): if to_remove.ext != ".mp3": to_remove.remove() (root / 'playlist.m3u8').write_lines(files) Because path.py: - inherit from str - has all the methods from os, not just a few cherry picked - has logical names for attributes and methods - have more utilities than pathlib or os So yes, if you do a lot of scripting, this is a must. It's also way easier for beginers to grasp. Am I alone in wondering why this exists? Is
it even worth improving?
On 3/26/2016 06:59, Andrew Barnert via Python-ideas wrote:
On Mar 25, 2016, at 13:20, Koos Zevenhoven <k7hoven@gmail.com> wrote:
So, let's start a new thread about how to deal with pathlib.Path objects, involving how/whether to convert between str and Path As a different point:
If we _don't_ either subclass str or find some way to make things magically work, is there any other way to start getting more uptake on pathlib? This may only be anecdotal, but from what I can tell, nobody is using it, because everyone who tries is put off by the need to convert back and forth everywhere.
Last year, everyone agreed that it would be good if at least the stdlib accepted paths everywhere, which might prompt third party libs to start doing the same. But nobody's started writing patches to do that. And I'm not sure we'd want it to happen in an uncoordinated way anyway, since there are at least four different ways to do it, and if we pick among them arbitrarily for different parts of the stdlib, we'll have a huge mess, and possibly behavioral inconsistencies.
The four ways I can think of are (in every function that currently takes a "path: str" argument, and should now take a "path: str | Path"):
* path = str(path) * path = Path(path) * if isinstance(path, Path): ... else: ... * try: f = path.open('w') except AttributeError: open(path, 'w')
It's also worth noting that all but the first require every module to depend on Path. Including C modules. And modules in the bootstrap. But the first version makes a bad guide for third-party code, because a lot of third-party code is dual-version/single-source libs, and you definitely don't want to unconditionally call str on a path argument that may be Unicode in 2.7 (or to unconditionally call a six-style Unicode function on a path argument that may be str in 2.7 on Linux).
So, assuming we all want a future in which pathlib is actually used by people, and assuming str subclassing is out and there's no other magic bullet, how do we get there from here? _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
I had to read the traditional way to figure out what you were even trying with the other two, so I don't know how 'natural' it is. As long as the traditional way continues to be supported, and this is what pathlib is intended to do, I guess I don't care. I know I'm not going to use it. On 3/26/2016 10:03, Michel Desmoulin wrote:
Because it makes a very common task (FS manipulation) much more natural.
This is the same reason people like request over urllib or like @decorator over func = decorator(func) or unpacking vs manual item extract.
In a directory of music files, you ave mp3 files you converted to other formats. Now you want to remove those conversion. Your task is to find all files having the same name as the ones with the mp3 ones, but with a different extension, remove them, then list the mp3 absolument path in a text file.
This example has baked in a lot of the tasks you do when you use Python as a scripting language at the core of you job such as if it's your glue language, if you are a tester or for sysadmin tasks. But it helps also everybody that once in a file does a one off script.
The traditional approach:
import os import glob import sys
root = os.path.abspath(sys.argv[1])
playlist = os.path.join(root, 'playlist.m3u8')
with open(playlist, 'w') as f:
for path in glob.glob(os.path.join(root, '*.mp3')):
name, ext = os.path.splitext(os.path.basename(path))
for to_remove in glob.glob(os.path.join(root, name + '.*')): if not to_remove.endswith('mp3'): os.remove(to_remove)
f.write(os.path.join(root, path) + "\n")
Now with pathlib you don't have to wonder about whether the feature you are looking for is on "os", "os.path" or "glob" or "open". You don't have to deal the file opening for such a small script. You don't have to switch between functions and methods all the time and have to choose between nested function calls or intermediary variables.
The pathlib version is way easier to figure out without knowing the stdlib by heart, it is one line shorter and one level of indent less. And you can discover it all from iPython with ".<tab>":
import sys import pathlib
root = pathlib.Path(sys.argv[1])
files = [] for path in root.glob('*.mp3'):
name = str(path).replace(path.suffix, '.*') files.append(str(path.absolute()))
for to_remove in root.glob(name): if to_remove.suffix != ".mp3": to_remove.unlink()
(root / 'playlist.m3u8').write_text('\n'.join(files))
And this true while pathlib is a half backed written lib, since the competition (that existed before pathlib and that pathlib failed to inspire from), can do even shorter, easier and cleaner:
import sys import path
root = path.Path(sys.argv[1]).realpath()
files = [] for p in root.glob('*.mp3'):
name = p.namebase + '.*' files.append(p)
for to_remove in root.glob(name): if to_remove.ext != ".mp3": to_remove.remove()
(root / 'playlist.m3u8').write_lines(files)
Because path.py:
- inherit from str - has all the methods from os, not just a few cherry picked - has logical names for attributes and methods - have more utilities than pathlib or os
So yes, if you do a lot of scripting, this is a must. It's also way easier for beginers to grasp.
On Sat, Mar 26, 2016 at 4:03 PM, Michel Desmoulin <desmoulinmichel@gmail.com
wrote:
[...] Now with pathlib you don't have to wonder about whether the feature you are looking for is on "os", "os.path" or "glob" or "open". You don't have to deal the file opening for such a small script. You don't have to switch between functions and methods all the time and have to choose between nested function calls or intermediary variables.
Here we completely agree. I used to do scripting of filesystem-intensive tasks with bash, because it felt more natural, partly because I was doing that before I started using Python for anything. Due to pathlib I now do this stuff in Python, which I of course like better for other reasons. Pretty much everything is there, and nicely auto-completed in editors. The fact that you can do / for joining paths is not necessarily better than a method (which exists too as .joinpath()), but it works and may be easy to remember.
The pathlib version is way easier to figure out without knowing the stdlib by heart, it is one line shorter and one level of indent less. And you can discover it all from iPython with ".<tab>":
import sys import pathlib
root = pathlib.Path(sys.argv[1])
files = [] for path in root.glob('*.mp3'):
name = str(path).replace(path.suffix, '.*') files.append(str(path.absolute()))
for to_remove in root.glob(name): if to_remove.suffix != ".mp3": to_remove.unlink()
(root / 'playlist.m3u8').write_text('\n'.join(files))
Or even cleaner, and hopefully easier to understand (not tested, sorry): import sys import pathlib root = pathlib.Path(sys.argv[1]) files = [f.absolute() for f in root.glob("*.mp3")] for mp3file in files: for f in root.glob(mp3file.stem + ".*"): if f.suffix != ".mp3": f.unlink() # remove non-mp3 version (root / "playlist.m3u8").write_text("\n".join(files)) Yes, some of the properties could have more logical names like .remove() instead of .unlink(), and there could be even more functionality. Subclassing from str would have obvious benefits, but I'm just not sure whether it's a good idea to inherit things like str.find, str.splitlines, str.title, ... So maybe the best thing is not to inherit from str but to make Path quack as much like a str-duck as possible (and vice versa?) without introducing confusing things? BTW, here's a link to issue #22570 (Better stdlib support for Path objects) http://www.psf.upfronthosting.co.za/issue22570 - Koos
On 26.03.2016 20:53, Koos Zevenhoven wrote:
Yes, some of the properties could have more logical names like .remove() instead of .unlink(), and there could be even more functionality. Subclassing from str would have obvious benefits, but I'm just not sure whether it's a good idea to inherit things like str.find, str.splitlines, str.title, ...
So maybe the best thing is not to inherit from str but to make Path quack as much like a str-duck as possible (and vice versa?) without introducing confusing things?
I would not like this limitation. Paths are strings (even for the os). It's just that Python devs are terribly lazy so, they want a string deluxe instead of a plain string in case a string represents a path. A lot of companies encode data right into the path of some files. So, when dealing with such situation, I would very like to have all capabilities of str available. Everything else would be frustrating IMHO. Best, Sven
On 29 March 2016 at 08:31, Sven R. Kunze <srkunze@mail.de> wrote:
Paths are strings (even for the os).
On Unix, paths are bytestrings, not strings (in Python 3, string = Unicode string), but on Windows they are Unicode strings. Having an abstraction over this OS difference is useful. Paul
On 29.03.2016 09:51, Paul Moore wrote:
On 29 March 2016 at 08:31, Sven R. Kunze <srkunze@mail.de> wrote:
Paths are strings (even for the os). On Unix, paths are bytestrings, not strings
Yep.
(in Python 3, string = Unicode string),
Do you mean str? When reading the docs, let's say: https://docs.python.org/3.5/library/stdtypes.html and search for "string" you find a lot of places where "string" is used and both "str" and "bytes" are working. ;-)
but on Windows they are Unicode strings. Having an abstraction over this OS difference is useful.
Indeed true. Best, Sven
I'm wondering how often literal pathames are really used in actual programs. In my experience, most pathnames are received as arguments, read from config files, etc. Most literals are just fragments that get concatenated with existing pathnames, and that's already covered by operations between path objects and strings. -- Greg
Le 27/03/2016 00:38, Greg Ewing a écrit :
I'm wondering how often literal pathames are really used in actual programs.
In my experience, most pathnames are received as arguments, read from config files, etc. Most literals are just fragments that get concatenated with existing pathnames, and that's already covered by operations between path objects and strings.
That's because you make big programs. I used to think that as well but spending now half of my time as a trainer, I met many other ways to use Python: - financial analists: load recursively all those csv files in pandas for me please. - biologists : damn where did I put my data again ? ipython => numpy => ok not this one. Not this one. Not this one... - geomaticians : hum, it's too slow, what if I reduce my layers to get only bob shape files ? Let me write that again. - sysadmins : ok let's move all those around, it's becoming a mess here. I'll make real script later to prevent this for happening again. - sunday scipters : oh, I'll reorganize my music today; - students : I don't know what I'm doing, but I know I want to do it in this dir. - testers : what's the f***** ? let me fire a shell to explore this. Python is not just used by programmers. It's used by 1000 people for whom reconfigurability is a nice to have in the end, but first they want to explore stuff manually and easily. I've been spending a lot of time in docker containers lately, and a switched a lot between python and bash because Python is still less productive to use for files. I can see myself in the shell using p'' to replace open, os and the like. Just because it's faster, easier to remember, doesn't require import and it one ".<tab>" away from giving me all I need to manipulate the file. ipython and ptpython already provide completion on file paths.
On 27.03.2016 00:38, Greg Ewing wrote:
I'm wondering how often literal pathames are really used in actual programs.
Don't underestimate what made Python great. Exploration is very important.
In my experience, most pathnames are received as arguments, read from config files, etc. Most literals are just fragments that get concatenated with existing pathnames, and that's already covered by operations between path objects and strings.
True, matured programs usually work this way. But as we all know, agile is better than waterfall, so we might start with literals and replacing them with config variables later when needed. Best, Sven
On 26.03.16 12:59, Andrew Barnert via Python-ideas wrote:
Last year, everyone agreed that it would be good if at least the stdlib accepted paths everywhere, which might prompt third party libs to start doing the same. But nobody's started writing patches to do that. And I'm not sure we'd want it to happen in an uncoordinated way anyway, since there are at least four different ways to do it, and if we pick among them arbitrarily for different parts of the stdlib, we'll have a huge mess, and possibly behavioral inconsistencies.
There is an open issue and I'm working on the patch. I'm going to make major path-related functions in the stdlib accepting the Path object in 3.6.
The four ways I can think of are (in every function that currently takes a "path: str" argument, and should now take a "path: str | Path"):
* path = str(path) * path = Path(path) * if isinstance(path, Path): ... else: ... * try: f = path.open('w') except AttributeError: open(path, 'w')
The official way: try: path = path.path except AttributeError: pass The advantage is that this works not with concrete Path implementation, but with any third-party Path class and other path-related objects like DirEntry that provide the "path" attribute.
There is an open issue and I'm working on the patch. I'm going to make major path-related functions in the stdlib accepting the Path object in 3.6.
This is great! Thanks Serhiy.
try: path = path.path except AttributeError: pass
The advantage is that this works not with concrete Path implementation, but with any third-party Path class and other path-related objects like DirEntry that provide the "path" attribute.
This sounds like the way to go, also. Could have saved ourselves some discussion :-) -CHB
Koos Zevenhoven writes:
One downside to p-strings like this is that they don't help you when you have the path in a plain str variable (i.e. not a literal),
IMO that kills this simple form of the idea, since hard-coding paths is a bad idea in many many programs, and paths are naturally derived from user input in a host of others. It becomes an attractive nuisance. How about generalizing f-strings, or perhaps giving p-strings (some of) the power of f-strings?
On Sat, Mar 26, 2016 at 7:32 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Koos Zevenhoven writes:
How about generalizing f-strings, or perhaps giving p-strings (some of) the power of f-strings?
Just their combination: pf"{ path_as_string }" ? That would at least be very consistent. - Koos
On 26.03.2016 19:17, Koos Zevenhoven wrote:
On Sat, Mar 26, 2016 at 7:32 PM, Stephen J. Turnbull <stephen@xemacs.org <mailto:stephen@xemacs.org>> wrote:
Koos Zevenhoven writes:
How about generalizing f-strings, or perhaps giving p-strings (some of) the power of f-strings?
That'll be a good idea.
Just their combination: pf"{ path_as_string }" ? That would at least be very consistent.
Could be make implicit: p'{my_variable}'
On 29.03.2016 09:11, Sven R. Kunze wrote:
On 26.03.2016 19:17, Koos Zevenhoven wrote:
On Sat, Mar 26, 2016 at 7:32 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Koos Zevenhoven writes:
How about generalizing f-strings, or perhaps giving p-strings (some of) the power of f-strings?
That'll be a good idea.
Just their combination: pf"{ path_as_string }" ? That would at least be very consistent.
Could be make implicit: p'{my_variable}'
Thinking more about it, that would even come in nicely with path separators: p'/{whereever}/{youwant}/{togo}' That'll finally an easy-to-write and readable path generation. Great idea, Stephen. Best, Sven
On Tue, Mar 29, 2016 at 6:22 PM, Sven R. Kunze <srkunze@mail.de> wrote:
Thinking more about it, that would even come in nicely with path separators:
p'/{whereever}/{youwant}/{togo}'
That'll finally an easy-to-write and readable path generation. Great idea, Stephen.
Agreed, although this introduces a new edge case: what if 'togo' contains a slash? Should this raise an exception, or should it be interpreted as a multi-part path component? Arguments can be put for both sides. But it's pretty cool. ChrisA
On Tue, Mar 29, 2016 at 06:27:09PM +1100, Chris Angelico wrote:
On Tue, Mar 29, 2016 at 6:22 PM, Sven R. Kunze <srkunze@mail.de> wrote:
Thinking more about it, that would even come in nicely with path separators:
p'/{whereever}/{youwant}/{togo}'
That'll finally an easy-to-write and readable path generation. Great idea, Stephen.
Agreed, although this introduces a new edge case: what if 'togo' contains a slash? Should this raise an exception, or should it be interpreted as a multi-part path component? Arguments can be put for both sides.
As we all know, togo is user input containing ../../../../../../../../../../../../../../../etc/passwd I agree that both sides can get arguments. However, if it does raise an exception, you'd have to go through hoops to make it work with components that might contain a slash. The other way around, however, is p'/{whereever}/{youwant}/{nosl(togo)}' where 'nosl' (or something similar) is a function that checks that the string 'togo' matches some sensible expectations: - Does not contain a '/'. - Does not equal '..' One of the codebases I worked at also had to work with relative paths, and we had a similar helper function which we used for os.path.join( whereever, youwant, nosl(togo), ) It hurts me to say this, but in this case it is probably better to slap on extra security later, than to make it too secure a default. An alternative might be to look at what frameworks like Django do w.r.t. XSS prevention, and do something like p'/{whereever}/{youwant}/{togo:|safe}' However I'm not really a fan of that.
But it's pretty cool.
ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Mar 29, 2016 at 10:49 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 29 March 2016 at 08:22, Sven R. Kunze <srkunze@mail.de> wrote:
Thinking more about it, that would even come in nicely with path separators:
p'/{whereever}/{youwant}/{togo}'
That'll finally an easy-to-write and readable path generation. Great idea, Stephen.
P(wherever)/youwant/togo
is as good, if not better - just "from pathlib import Path as P" (or use Path(wherever) if you don't like unnecessary 1-letter aliases...)
Yes. I assume you meant P('/') /wherever/youwant/togo. Or with p-strings, p'/' /wherever/youwant/togo I'm not necessarily against Sven's version either, but my first thoughts about the power of combined pf strings would be something like this (assuming Sven's implicit f): all_data = [p'data{i}.txt'.read_text() for i in range(1, 307)] which would read the contents of data1.txt, data2.txt, ..., and data306.txt into a list. And below, to the security issue raised by Sjoerd. On Tue, Mar 29, 2016 at 10:46 AM, Sjoerd Job Postmus <sjoerdjob@sjec.nl> wrote:
On Tue, Mar 29, 2016 at 06:27:09PM +1100, Chris Angelico wrote:
On Tue, Mar 29, 2016 at 6:22 PM, Sven R. Kunze <srkunze@mail.de> wrote:
Thinking more about it, that would even come in nicely with path separators:
p'/{whereever}/{youwant}/{togo}'
That'll finally an easy-to-write and readable path generation. Great idea, Stephen.
Agreed, although this introduces a new edge case: what if 'togo' contains a slash? Should this raise an exception, or should it be interpreted as a multi-part path component? Arguments can be put for both sides.
As we all know, togo is user input containing ../../../../../../../../../../../../../../../etc/passwd
How about p'/' /wherever/youwant//togo That is, the floordiv operator could be used to prevent 'togo' from going up the directory tree with "../../" or "/etc/passwd". The // would thus restrict the user (who provides `togo`) into /wherever/youwant/ and it its subdirectories. -Koos
On 29 March 2016 at 11:24, Koos Zevenhoven <k7hoven@gmail.com> wrote:
As we all know, togo is user input containing ../../../../../../../../../../../../../../../etc/passwd
How about
p'/' /wherever/youwant//togo
That is, the floordiv operator could be used to prevent 'togo' from going up the directory tree with "../../" or "/etc/passwd". The // would thus restrict the user (who provides `togo`) into /wherever/youwant/ and it its subdirectories.
There have been a number of previous threads about the security of path objects (and for that matter of os.path) and "sandboxing" paths to disallow traversing "up" out of a particular directory. I've no idea whether the floordiv operator would work for this purpose (I'm not a security expert, and most of my personal applications can afford to take a very lax view of such things, luckily) but it's an interesting idea. Potentially, it's a bit easy to miss, which may be a problem. Paul
On Tue, Mar 29, 2016 at 1:41 PM, Paul Moore <p.f.moore@gmail.com> wrote:
On 29 March 2016 at 11:24, Koos Zevenhoven <k7hoven@gmail.com> wrote:
As we all know, togo is user input containing ../../../../../../../../../../../../../../../etc/passwd
How about
p'/' /wherever/youwant//togo
That is, the floordiv operator could be used to prevent 'togo' from going up the directory tree with "../../" or "/etc/passwd". The // would thus restrict the user (who provides `togo`) into /wherever/youwant/ and it its subdirectories.
There have been a number of previous threads about the security of path objects (and for that matter of os.path) and "sandboxing" paths to disallow traversing "up" out of a particular directory.
I've no idea whether the floordiv operator would work for this purpose (I'm not a security expert, and most of my personal applications can afford to take a very lax view of such things, luckily) but it's an interesting idea. Potentially, it's a bit easy to miss, which may be a problem.
Actually, after having looked into RFC 3986s, I'd say using | instead of // for the path sandboxing operator may be better for at least two reasons: 1a. Because // is used in relative references for URIs to mean essentially 'go all the way up to scheme://', which is quite opposite to the concept of sandboxing. 1b. Because | kind of looks like a wall. 2. Because | has lower precedence than /, meaning that dontmesswiththis | relative / paths == dontmesswiththis | (relative / paths) while dontmesswiththis // relative / paths == (dontmesswiththis // relative) / paths So unless (dontmesswiththis // relative) somehow remembers the sandboxing barrier, the | would do the right thing and // the wrong thing. Then of course you might want it to remember the barrier anyway. I won't open the pandora's box in front of myself now regarding the old threads you mention. But if someone else wants to do that and/or there is something to discuss regarding this, I suggest to move the sandboxing discussion to a new thread. -Koos
On 29 March 2016 at 08:22, Sven R. Kunze <srkunze@mail.de> wrote:
Thinking more about it, that would even come in nicely with path separators:
p'/{whereever}/{youwant}/{togo}'
That'll finally an easy-to-write and readable path generation. Great idea, Stephen.
P(wherever)/youwant/togo is as good, if not better - just "from pathlib import Path as P" (or use Path(wherever) if you don't like unnecessary 1-letter aliases...) Paul
On Mar 30, 2016 00:39, "Ethan Furman" <ethan@stoneleaf.us> wrote:
On 03/29/2016 02:30 PM, Greg Ewing wrote:
Paul Moore wrote:
P(wherever)/youwant/togo
If P were a suitable seed object, this could be
P/wherever/youwant/togo
An interesting idea, but how would you differentiate between relative and
absolute paths?
Indeed interesting. It would currently work with P = Path(''), kinda like my earlier p'' / ... example. It works even with relative paths, although they would not look like relative ones because of the first /. How short can a name be to be acceptable in builtins?-) K (mobile)
On 03/29/2016 02:47 PM, Koos Zevenhoven wrote:
On Mar 30, 2016 00:39, "Ethan Furman" wrote:
On 03/29/2016 02:30 PM, Greg Ewing wrote:
If P were a suitable seed object, this could be
P/wherever/youwant/togo
An interesting idea, but how would you differentiate between relative and absolute paths?
Indeed interesting. It would currently work with P = Path(''), kinda like my earlier p'' / ... example. It works even with relative paths, although they would not look like relative ones because of the first /.
Please provide an example of both a relative and an absolute path construction using `P`. -- ~Ethan~
On Mar 30, 2016 01:07, "Ethan Furman" <ethan@stoneleaf.us> wrote:
On 03/29/2016 02:47 PM, Koos Zevenhoven wrote:
On Mar 30, 2016 00:39, "Ethan Furman" wrote:
On 03/29/2016 02:30 PM, Greg Ewing wrote:
If P were a suitable seed object, this could be
P/wherever/youwant/togo
An interesting idea, but how would you differentiate between relative and absolute paths?
Indeed interesting. It would currently work with P = Path(''), kinda like my earlier p'' / ... example. It works even with relative paths, although they would not look like relative ones because of the first /.
Please provide an example of both a relative and an absolute path
construction using `P`.
P/'relative/path' P/'/absolute/path' -K / mobile
On 03/29/2016 03:10 PM, Koos Zevenhoven wrote:
On Mar 30, 2016 01:07, "Ethan Furman" wrote:
Please provide an example of both a relative and an absolute path construction using `P`.
P/'relative/path'
P/'/absolute/path'
Thank you. So under this system a relative path appears with one leading slash, and an absolute path appears with two leading slashes. Can't say I'm a fan. :( -- ~Ethan~
On Mar 30, 2016 01:07, "Ethan Furman" wrote:
Please provide an example of both a relative and an absolute path construction using `P`.
P/'relative/path'
P/'/absolute/path'
Thank you.
So under this system a relative path appears with one leading slash, and an absolute path appears with two leading slashes.
Can't say I'm a fan. :(
I'll have to agree. But to be fair, Greg wrote something like 'a suitable seed object' so we could for ex. have P=PathSeed(), and use another operator. P@'rel/path' P@'/abs/path' Dunno. -K@mobile
On Tue, 29 Mar 2016 at 15:41 Koos Zevenhoven <k7hoven@gmail.com> wrote:
On Mar 30, 2016 01:07, "Ethan Furman" wrote:
Please provide an example of both a relative and an absolute path construction using `P`.
P/'relative/path'
P/'/absolute/path'
Thank you.
So under this system a relative path appears with one leading slash, and an absolute path appears with two leading slashes.
Can't say I'm a fan. :(
I'll have to agree. But to be fair, Greg wrote something like 'a suitable seed object' so we could for ex. have P=PathSeed(), and use another operator.
P@'rel/path'
P@'/abs/path'
Dunno.
If you are constructing a new object then you don't need to limit yourself to a single one. You could use A for absolute paths and R for relative paths: R/'relative'/'path' A/'absolute'/'path' Or even P.absolute and P.relative: P.relative/'relative'/'path' P.absolute/'absolute'/'path' And you could even go as far as make one the default and the other not to promote one over the other: P.relative/'relative'/'path' P/'absolute'/'path' -Brett
-K@mobile _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Mar 30, 2016 at 1:49 AM, Brett Cannon <brett@python.org> wrote:
On Tue, 29 Mar 2016 at 15:41 Koos Zevenhoven <k7hoven@gmail.com> wrote:
I'll have to agree. But to be fair, Greg wrote something like 'a suitable seed object' so we could for ex. have P=PathSeed(), and use another operator.
P@'rel/path'
P@'/abs/path'
Dunno.
[...]
Or even P.absolute and P.relative:
P.relative/'relative'/'path' P.absolute/'absolute'/'path'
And you could even go as far as make one the default and the other not to promote one over the other:
P.relative/'relative'/'path' P/'absolute'/'path'
The trick is, again, that the path is often in a variable coming from user input etc. and you don't (want to) deal with whether it is relative or absolute. This also still has a slash before relative paths too. Another thing is that, while / looks very nice, it makes you have to use parentheses when you want to directly call a method on the path: (somedir / relative_file_path).method() We have been making examples with /many/'path'/'objects'/joined_together. But how often do people really join more than two paths at once? Sometimes, in interactive sessions, I have wished for this to work: somedir(relative_file_path).method() ...which is kind of stupid ;) - Koos
On 30.03.2016 16:55, Koos Zevenhoven wrote:
The trick is, again, that the path is often in a variable coming from user input etc. and you don't (want to) deal with whether it is relative or absolute. This also still has a slash before relative paths too.
Another thing is that, while / looks very nice, it makes you have to use parentheses when you want to directly call a method on the path:
(somedir / relative_file_path).method()
We have been making examples with /many/'path'/'objects'/joined_together. But how often do people really join more than two paths at once?
Sometimes, in interactive sessions, I have wished for this to work:
somedir(relative_file_path).method()
...which is kind of stupid ;)
This seems all to speak for p-strings. :) Best, Sven
On 03/30/2016 07:55 AM, Koos Zevenhoven wrote:
On Wed, Mar 30, 2016 at 1:49 AM, Brett Cannon <brett@python.org> wrote:
Or even P.absolute and P.relative:
P.relative/'relative'/'path' P.absolute/'absolute'/'path'
Which involves even more typing and is less readable.
Sometimes, in interactive sessions, I have wished for this to work:
somedir(relative_file_path).method()
...which is kind of stupid ;)
Do you mean: base = Path('source/antipathy') base('test').listdir() I don't know about pathlib, but with antipathy's Path [1] you can do: base.listdir('test') -- ~Ethan~ [1] Yes, that one is mine. The step is to make it pathlib compatible, which will involve, at a minimum, renaming some attributes.
On Wed, Mar 30, 2016 at 7:56 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/30/2016 07:55 AM, Koos Zevenhoven wrote:
Sometimes, in interactive sessions, I have wished for this to work:
somedir(relative_file_path).method()
...which is kind of stupid ;)
Do you mean:
base = Path('source/antipathy') base('test').listdir()
Yes, I think. That is, equivalent to (base / 'test').listdir()
I don't know about pathlib, but with antipathy's Path [1] you can do:
base.listdir('test')
I have wished for base.mkdir("subdir") too, but the addition to pathlib would be backwards incompatible for people that give the mode argument as a positional argument.
[1] Yes, that one is mine. The step is to make it pathlib compatible, which will involve, at a minimum, renaming some attributes.
I wonder if it is still possible to rename pathlib attributes (and keep the old ones as deprecated aliases for a while), now that the adoption of pathlib is still limited. -Koos
(I have no interest in this subject of this thread, but wanted to point this out.) On Wed, Mar 30, 2016 at 1:51 PM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
I have wished for base.mkdir("subdir") too, but the addition to pathlib would be backwards incompatible for people that give the mode argument as a positional argument.
[snip]
I wonder if it is still possible to rename pathlib attributes (and keep the old ones as deprecated aliases for a while), now that the adoption of pathlib is still limited.
pathlib is still provisional (PEP 411), thus any changes can be made if deemed necessary, including backward incompatible changes. The bar for such changes is described by PEP 411 as "quite high", but they can occur.
On 03/30/2016 10:51 AM, Koos Zevenhoven wrote:
I wonder if it is still possible to rename pathlib attributes (and keep the old ones as deprecated aliases for a while), now that the adoption of pathlib is still limited.
The `.path` attribute, which seems a silly name, I look at as being necessary for compatibility with DirEntry. -- ~Ethan~
On Wed, 30 Mar 2016 at 11:10 Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/30/2016 10:51 AM, Koos Zevenhoven wrote:
I wonder if it is still possible to rename pathlib attributes (and keep the old ones as deprecated aliases for a while), now that the adoption of pathlib is still limited.
The `.path` attribute, which seems a silly name, I look at as being necessary for compatibility with DirEntry.
I don't remember the exact issue that tracked the discussion, but there was back-and-forth on that attribute name and in the end Guido chose `path`.
On 03/30/2016 11:26 AM, Brett Cannon wrote:
On Wed, 30 Mar 2016 at 11:10 Ethan Furman wrote:
The `.path` attribute, which seems a silly name, I look at as being necessary for compatibility with DirEntry.
I don't remember the exact issue that tracked the discussion, but there was back-and-forth on that attribute name and in the end Guido chose `path`.
Hmmm.... so - Path.path -> str - DirEntry.path -> str Perhaps DirEntry should grow a .Path attribute to a Path from. ;) -- ~Ethan~
On Wed, Mar 30, 2016 at 9:26 PM, Brett Cannon <brett@python.org> wrote:
The `.path` attribute, which seems a silly name, I look at as being necessary for compatibility with DirEntry.
I don't remember the exact issue that tracked the discussion, but there was back-and-forth on that attribute name and in the end Guido chose `path`.
This one: https://bugs.python.org/issue22570 See my comment below (from this thread, but there were no reactions). I'm not sure if that option was ever considered. Can we let DirEntry dictate the future of Python? On Sun, Mar 27, 2016 at 5:40 PM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
Actually, now that .path is not out yet, would it make sense to call it Path.str or Path.strpath instead, and introduce the same thing on DirEntry and guarantee a str (instead of str or bytes as DirEntry.path now does)? Maybe that would lead to fewer broken implementations in third-party libraries too?
-Koos
On Wed, Mar 30, 2016 at 9:45 PM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
See my comment below (from this thread, but there were no reactions). I'm not sure if that option was ever considered. Can we let DirEntry dictate the future of Python?
BTW, don't get me wrong, I'm sure os.scandir is great, even if I have not used it :). What I mean is that it's easier to change something that was added more recently, i.e. when there are no old codebases that depend on it. I just think we should get these things right, before it's too late.
On Sun, Mar 27, 2016 at 5:40 PM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
Actually, now that .path is not out yet, would it make sense to call it Path.str or Path.strpath instead, and introduce the same thing on DirEntry and guarantee a str (instead of str or bytes as DirEntry.path now does)? Maybe that would lead to fewer broken implementations in third-party libraries too?
-Koos
Koos Zevenhoven wrote:
See my comment below (from this thread, but there were no reactions). I'm not sure if that option was ever considered. Can we let DirEntry dictate the future of Python?
No. On the other hand, we'd need a /really/ good reason to change it now.
Actually, now that .path is not out yet, would it make sense to call it Path.str or Path.strpath instead, and introduce the same thing on DirEntry and guarantee a str (instead of str or bytes as DirEntry.path now does)? Maybe that would lead to fewer broken implementations in third-party libraries too?
DirEntry needs to return the same type it was given, and one can give it both bytes and strings. Your first point /may/ be valid. The big question is: is DirEntry provisional? I don't think it is. In which case, we're stuck with .path on DirEntry, and may as well use the same name for the same idea on pathlib.Path. -- ~Ethan~
On Wed, Mar 30, 2016, at 20:02, Ethan Furman wrote:
Koos Zevenhoven wrote:
See my comment below (from this thread, but there were no reactions). I'm not sure if that option was ever considered. Can we let DirEntry dictate the future of Python?
No. On the other hand, we'd need a /really/ good reason to change it now.
Just change it in Python 4000 ;)
On Tue, Mar 29, 2016, at 17:39, Ethan Furman wrote:
On 03/29/2016 02:30 PM, Greg Ewing wrote:
Paul Moore wrote:
P(wherever)/youwant/togo
If P were a suitable seed object, this could be
P/wherever/youwant/togo
An interesting idea, but how would you differentiate between relative and absolute paths?
Path.ROOT/absolute/path Path.DOT/relative/path
On 30.03.2016 05:25, Random832 wrote:
On Tue, Mar 29, 2016, at 17:39, Ethan Furman wrote:
An interesting idea, but how would you differentiate between relative and absolute paths?
Path.ROOT/absolute/path Path.DOT/relative/path
Very interesting. I like how it looks with variables. But I have to admit I don't like the look so much with regards to string literals. This is where the format string really shines. Best, Sven
participants (26)
-
Alexander Walters -
Andrew Barnert -
Brett Cannon -
Chris Angelico -
Chris Barker - NOAA Federal -
Donald Stufft -
Eric V. Smith -
Ethan Furman -
Greg Ewing -
Guido van Rossum -
Jonathan Goble -
Joseph Martinot-Lagarde -
Koos Zevenhoven -
Michael Selik -
Michel Desmoulin -
Nick Coghlan -
Paul Moore -
Pavol Lisy -
Random832 -
Serhiy Storchaka -
Sjoerd Job Postmus -
Stephen J. Turnbull -
Steven D'Aprano -
Sven R. Kunze -
Wes Turner -
Yann Kaiser