I see PEP 428 is both targeted at 3.4 and still in draft status. What remains to be done to ask for pronouncement? -- ~Ethan~
Le Sun, 15 Sep 2013 06:46:08 -0700, Ethan Furman <ethan@stoneleaf.us> a écrit :
I see PEP 428 is both targeted at 3.4 and still in draft status.
What remains to be done to ask for pronouncement?
I think I have a couple of items left to integrate in the PEP. Mostly it needs me to take a bit of time and finalize the PEP, and then have a PEP delegate (or Guido) pronounce on it. That's unless someone else wants to add it, of course. I don't mind suggestions, as long as they don't deter from the short-/middle-term goal of getting the PEP approved :) Note that the pathlib API has to be provisional. While "OO path objects" have been a common desire for a long time, it is a much less well-paved avenue than e.g. TransformDict :) Regards Antoine.
I hope there is a volunteer for delegate. --Guido van Rossum (sent from Android phone) On Sep 16, 2013 1:17 AM, "Antoine Pitrou" <solipsis@pitrou.net> wrote:
Le Sun, 15 Sep 2013 06:46:08 -0700, Ethan Furman <ethan@stoneleaf.us> a écrit :
I see PEP 428 is both targeted at 3.4 and still in draft status.
What remains to be done to ask for pronouncement?
I think I have a couple of items left to integrate in the PEP. Mostly it needs me to take a bit of time and finalize the PEP, and then have a PEP delegate (or Guido) pronounce on it.
That's unless someone else wants to add it, of course. I don't mind suggestions, as long as they don't deter from the short-/middle-term goal of getting the PEP approved :)
Note that the pathlib API has to be provisional. While "OO path objects" have been a common desire for a long time, it is a much less well-paved avenue than e.g. TransformDict :)
Regards
Antoine.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org
2013/9/16 Antoine Pitrou <solipsis@pitrou.net>:
Le Sun, 15 Sep 2013 06:46:08 -0700, Ethan Furman <ethan@stoneleaf.us> a écrit :
I see PEP 428 is both targeted at 3.4 and still in draft status.
What remains to be done to ask for pronouncement?
I think I have a couple of items left to integrate in the PEP. Mostly it needs me to take a bit of time and finalize the PEP, and then have a PEP delegate (or Guido) pronounce on it.
IIRC, during the last discussion round, we were still debating between implicit stat() result caching - which requires an explicit restat() method - vs a mapping between the stat() method and a stat() syscall. What was the conclusion?
On Mon, 16 Sep 2013 19:06:37 +0200 Charles-François Natali <cf.natali@gmail.com> wrote:
2013/9/16 Antoine Pitrou <solipsis@pitrou.net>:
Le Sun, 15 Sep 2013 06:46:08 -0700, Ethan Furman <ethan@stoneleaf.us> a écrit :
I see PEP 428 is both targeted at 3.4 and still in draft status.
What remains to be done to ask for pronouncement?
I think I have a couple of items left to integrate in the PEP. Mostly it needs me to take a bit of time and finalize the PEP, and then have a PEP delegate (or Guido) pronounce on it.
IIRC, during the last discussion round, we were still debating between implicit stat() result caching - which requires an explicit restat() method - vs a mapping between the stat() method and a stat() syscall.
What was the conclusion?
No definite conclusion. You and Nick liked the idea of a rich stat object (returned by os.stat()) with is_dir() methods and the like: https://mail.python.org/pipermail/python-dev/2013-May/125809.html However, nothing was done about that since then ;-) There was also the scandir() proposal to return rich objects with optional stat-like fields, but similarly it didn't get a conclusion: https://mail.python.org/pipermail/python-dev/2013-May/126119.html So I would like to propose the following API change: - Path.stat() (and stat-accessing methods such as get_mtime()...) returns an uncached stat object by default - Path.cache_stat() can be called to return the stat() *and* cache it for future use, such that any future call to stat(), cache_stat() or a stat-accessing function reuses that cached stat In other words, only if you use cache_stat() at least once is the stat() value cached and reused by the Path object. (also, it's a per-Path decision) Regards Antoine.
On Mon, Sep 16, 2013 at 3:45 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Mon, 16 Sep 2013 19:06:37 +0200 Charles-François Natali <cf.natali@gmail.com> wrote:
2013/9/16 Antoine Pitrou <solipsis@pitrou.net>:
Le Sun, 15 Sep 2013 06:46:08 -0700, Ethan Furman <ethan@stoneleaf.us> a écrit :
I see PEP 428 is both targeted at 3.4 and still in draft status.
What remains to be done to ask for pronouncement?
I think I have a couple of items left to integrate in the PEP. Mostly it needs me to take a bit of time and finalize the PEP, and then have a PEP delegate (or Guido) pronounce on it.
IIRC, during the last discussion round, we were still debating between implicit stat() result caching - which requires an explicit restat() method - vs a mapping between the stat() method and a stat() syscall.
What was the conclusion?
No definite conclusion. You and Nick liked the idea of a rich stat object (returned by os.stat()) with is_dir() methods and the like: https://mail.python.org/pipermail/python-dev/2013-May/125809.html
However, nothing was done about that since then ;-)
There was also the scandir() proposal to return rich objects with optional stat-like fields, but similarly it didn't get a conclusion: https://mail.python.org/pipermail/python-dev/2013-May/126119.html
So I would like to propose the following API change:
- Path.stat() (and stat-accessing methods such as get_mtime()...) returns an uncached stat object by default
- Path.cache_stat() can be called to return the stat() *and* cache it for future use, such that any future call to stat(), cache_stat() or a stat-accessing function reuses that cached stat
In other words, only if you use cache_stat() at least once is the stat() value cached and reused by the Path object. (also, it's a per-Path decision)
Any reason why stat() can't get a keyword-only cached=True argument instead? Or have stat() never cache() but stat_cache() always so that people can choose if they want fresh or cached based on API and not whether some library happened to make a decision for them?
2013/9/16 Brett Cannon <brett@python.org>:
Any reason why stat() can't get a keyword-only cached=True argument instead? Or have stat() never cache() but stat_cache() always so that people can choose if they want fresh or cached based on API and not whether some library happened to make a decision for them?
I also prefer a single function, but only if the default is cached=False. Caching by default can be surprising and unexpected. Victor
On Mon, 16 Sep 2013 15:48:54 -0400 Brett Cannon <brett@python.org> wrote:
So I would like to propose the following API change:
- Path.stat() (and stat-accessing methods such as get_mtime()...) returns an uncached stat object by default
- Path.cache_stat() can be called to return the stat() *and* cache it for future use, such that any future call to stat(), cache_stat() or a stat-accessing function reuses that cached stat
In other words, only if you use cache_stat() at least once is the stat() value cached and reused by the Path object. (also, it's a per-Path decision)
Any reason why stat() can't get a keyword-only cached=True argument instead? Or have stat() never cache() but stat_cache() always so that people can choose if they want fresh or cached based on API and not whether some library happened to make a decision for them?
1. Because you also want the helper functions (get_mtime(), etc.) to cache the value too. It's not only about stat(). 2. Because of the reverse use case where you want a library to reuse a cached value despite the library not using an explicit caching call. Basically, the rationale is: 1. Caching should be opt-in, which is what this new API achieves. 2. Once you have asked for caching, most always you also want the subsequent accesses to be cached. I realize there should be a third method clear_cache(), though ;-) Regards Antoine.
On Sep 16, 2013, at 1:05 PM, Antoine Pitrou wrote:
On Mon, 16 Sep 2013 15:48:54 -0400 Brett Cannon <brett@python.org> wrote:
So I would like to propose the following API change:
- Path.stat() (and stat-accessing methods such as get_mtime()...) returns an uncached stat object by default
- Path.cache_stat() can be called to return the stat() *and* cache it for future use, such that any future call to stat(), cache_stat() or a stat-accessing function reuses that cached stat
In other words, only if you use cache_stat() at least once is the stat() value cached and reused by the Path object. (also, it's a per-Path decision)
Any reason why stat() can't get a keyword-only cached=True argument instead? Or have stat() never cache() but stat_cache() always so that people can choose if they want fresh or cached based on API and not whether some library happened to make a decision for them?
1. Because you also want the helper functions (get_mtime(), etc.) to cache the value too. It's not only about stat().
With the proposed rich stat object the convenience methods living on Path wouldn't result in much added convenience: p.is_dir() vs p.stat().is_dir() Why not move these methods from Path to a rich stat obj and not cache stat results at all? It's easy enough for users to cache them themselves and much more explicit. -- Philip Jenvey
On 18 September 2013 11:10, Philip Jenvey <pjenvey@underboss.org> wrote:
On Sep 16, 2013, at 1:05 PM, Antoine Pitrou wrote:
On Mon, 16 Sep 2013 15:48:54 -0400 Brett Cannon <brett@python.org> wrote:
So I would like to propose the following API change:
- Path.stat() (and stat-accessing methods such as get_mtime()...) returns an uncached stat object by default
- Path.cache_stat() can be called to return the stat() *and* cache it for future use, such that any future call to stat(), cache_stat() or a stat-accessing function reuses that cached stat
In other words, only if you use cache_stat() at least once is the stat() value cached and reused by the Path object. (also, it's a per-Path decision)
Any reason why stat() can't get a keyword-only cached=True argument instead? Or have stat() never cache() but stat_cache() always so that people can choose if they want fresh or cached based on API and not whether some library happened to make a decision for them?
1. Because you also want the helper functions (get_mtime(), etc.) to cache the value too. It's not only about stat().
With the proposed rich stat object the convenience methods living on Path wouldn't result in much added convenience:
p.is_dir() vs p.stat().is_dir()
Why not move these methods from Path to a rich stat obj and not cache stat results at all? It's easy enough for users to cache them themselves and much more explicit.
Because that doesn't help iterator based os.walk inspired APIs like walkdir, which would benefit greatly from a path type with implicit caching, but would have to complicate their APIs significantly to pass around separate stat objects. Rewriting walkdir to depend on pathlib has been on my todo list for a while, as it solves a potentially serious walkdir performance problem where chained iterators have to make repeated stat calls to answer questions that were already asked by earlier iterators in the pipeline. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Le Tue, 17 Sep 2013 18:10:53 -0700, Philip Jenvey <pjenvey@underboss.org> a écrit :
On Sep 16, 2013, at 1:05 PM, Antoine Pitrou wrote:
On Mon, 16 Sep 2013 15:48:54 -0400 Brett Cannon <brett@python.org> wrote:
So I would like to propose the following API change:
- Path.stat() (and stat-accessing methods such as get_mtime()...) returns an uncached stat object by default
- Path.cache_stat() can be called to return the stat() *and* cache it for future use, such that any future call to stat(), cache_stat() or a stat-accessing function reuses that cached stat
In other words, only if you use cache_stat() at least once is the stat() value cached and reused by the Path object. (also, it's a per-Path decision)
Any reason why stat() can't get a keyword-only cached=True argument instead? Or have stat() never cache() but stat_cache() always so that people can choose if they want fresh or cached based on API and not whether some library happened to make a decision for them?
1. Because you also want the helper functions (get_mtime(), etc.) to cache the value too. It's not only about stat().
With the proposed rich stat object the convenience methods living on Path wouldn't result in much added convenience:
p.is_dir() vs p.stat().is_dir()
One reason is that the proposed rich stat object doesn't exist yet :-) Regards Antoine.
On Mon, 16 Sep 2013 15:48:54 -0400, Brett Cannon <brett@python.org> wrote:
On Mon, Sep 16, 2013 at 3:45 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
So I would like to propose the following API change:
- Path.stat() (and stat-accessing methods such as get_mtime()...) returns an uncached stat object by default
- Path.cache_stat() can be called to return the stat() *and* cache it for future use, such that any future call to stat(), cache_stat() or a stat-accessing function reuses that cached stat
In other words, only if you use cache_stat() at least once is the stat() value cached and reused by the Path object. (also, it's a per-Path decision)
Any reason why stat() can't get a keyword-only cached=True argument instead? Or have stat() never cache() but stat_cache() always so that people can choose if they want fresh or cached based on API and not whether some library happened to make a decision for them?
Well, we tend to avoid single boolean arguments in favor of differently named functions. But here is an alternate API: expose the state by having a 'cache_stat' attribute of the Path that is 'False' by default but can be set 'True'. It could also (or only?) be set via an optional constructor argument. --David
On Mon, 16 Sep 2013 16:14:43 -0400 "R. David Murray" <rdmurray@bitdance.com> wrote:
On Mon, 16 Sep 2013 15:48:54 -0400, Brett Cannon <brett@python.org> wrote:
On Mon, Sep 16, 2013 at 3:45 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
So I would like to propose the following API change:
- Path.stat() (and stat-accessing methods such as get_mtime()...) returns an uncached stat object by default
- Path.cache_stat() can be called to return the stat() *and* cache it for future use, such that any future call to stat(), cache_stat() or a stat-accessing function reuses that cached stat
In other words, only if you use cache_stat() at least once is the stat() value cached and reused by the Path object. (also, it's a per-Path decision)
Any reason why stat() can't get a keyword-only cached=True argument instead? Or have stat() never cache() but stat_cache() always so that people can choose if they want fresh or cached based on API and not whether some library happened to make a decision for them?
Well, we tend to avoid single boolean arguments in favor of differently named functions.
But here is an alternate API: expose the state by having a 'cache_stat' attribute of the Path that is 'False' by default but can be set 'True'.
Thanks for the suggestion, that's a possibility too.
It could also (or only?) be set via an optional constructor argument.
That's impractical if you get the Path object from a library call. Regards Antoine.
On 17 Sep 2013 06:45, "Antoine Pitrou" <solipsis@pitrou.net> wrote:
On Mon, 16 Sep 2013 16:14:43 -0400 "R. David Murray" <rdmurray@bitdance.com> wrote:
On Mon, 16 Sep 2013 15:48:54 -0400, Brett Cannon <brett@python.org>
wrote:
On Mon, Sep 16, 2013 at 3:45 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
So I would like to propose the following API change:
- Path.stat() (and stat-accessing methods such as get_mtime()...) returns an uncached stat object by default
- Path.cache_stat() can be called to return the stat() *and* cache it for future use, such that any future call to stat(), cache_stat() or a stat-accessing function reuses that cached stat
In other words, only if you use cache_stat() at least once is the stat() value cached and reused by the Path object. (also, it's a per-Path decision)
Any reason why stat() can't get a keyword-only cached=True argument instead? Or have stat() never cache() but stat_cache() always so that people can choose if they want fresh or cached based on API and not whether some library happened to make a decision for them?
Well, we tend to avoid single boolean arguments in favor of differently named functions.
But here is an alternate API: expose the state by having a 'cache_stat' attribute of the Path that is 'False' by default but can be set 'True'.
Thanks for the suggestion, that's a possibility too.
It could also (or only?) be set via an optional constructor argument.
That's impractical if you get the Path object from a library call.
Given that this is a behavioural state change, I think asking for a possibly *new* path with caching enabled in that case would be a good way to go. If we treat path objects as effectively immutable (aside from the optional internal stat cache), then checking in __new__ if a passed in path object already has the appropriate caching status and returning it directly if so, but otherwise creating a new path object with the cache setting changed would avoid having libraries potentially alter the behaviour of applications' path objects and vice-versa. In effect, the unique "identity" of a path would be a triple representing the type, the filesystem path and whether or not it cached stat results internally. If you wanted to change any of those, you would have to create a new object. Cheers, Nick.
Regards
Antoine.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
On 9/16/2013 4:14 PM, R. David Murray wrote:
Well, we tend to avoid single boolean arguments in favor of differently named functions.
The stdlib has lots of boolean arguments. My impression is that they are to be avoided when they would change the return type or otherwise do something disjointly different. I do not think this would apply here. -- Terry Jan Reedy
Terry Reedy writes:
On 9/16/2013 4:14 PM, R. David Murray wrote:
Well, we tend to avoid single boolean arguments in favor of differently named functions.
The stdlib has lots of boolean arguments. My impression is that they are to be avoided when they would change the return type or otherwise do something disjointly different. I do not think this would apply here.
I remember reading that the criterion is whether the argument is most often given a literal value. Then "stat_cache()" is preferable to "stat(cache=True)". OTOH, "stat(cache=want_cache)" is better than if want_cache: result = stat_cache() else: result = stat() or "result = stat_cache() if want_cache else stat()".
participants (11)
-
Antoine Pitrou -
Brett Cannon -
Charles-François Natali -
Ethan Furman -
Guido van Rossum -
Nick Coghlan -
Philip Jenvey -
R. David Murray -
Stephen J. Turnbull -
Terry Reedy -
Victor Stinner