[Twisted-Python] symlink support in filepath.FilePath
I've been trying to put a few patches into twisted.python.filepath lately to support symlinks. This is because I'm trying to implement a conch SFTP server using FilePath. I know FilePath is supposed to be cross-platform, so I am wondering how to go about it. For instance, it would be really easy to add a flag to restat that specifies whether to use os.stat o os.lstat. But how should I handle the getAccessTime/etc. functions? Should I add a FilePath.symlink() function (that creates a symlink to said FilePath)? Should I just implement my own subclass of FilePath that handles symlinks and not try to change FilePath? Thanks for the help, Ying
On Mon, 24 Mar 2008 09:34:00 -0500, Ying Li <cyli@alum.mit.edu> wrote:
I've been trying to put a few patches into twisted.python.filepath lately to support symlinks. This is because I'm trying to implement a conch SFTP server using FilePath.
Hooray! Thanks for working on FilePath. :)
I know FilePath is supposed to be cross-platform, so I am wondering how to go about it.
Hmm. I guess FilePath _is_ supposed to be cross-platform. Maybe it would be useful to list the other things it is supposed to be: * easier to use than os.path.* * more efficient in naive usage (hence `restat´) * sufficiently self-contained so that things like ZipPath are possible * safe against common security-related mistakes (hence `child´) Whether or not it succeeds at any of these goals is probably an open question, and work could probably be done to improve it in all these areas, _and_ the approach taken to accomplish some of these goals may be questionable.
For instance, it would be really easy to add a flag to restat that specifies whether to use os.stat o os.lstat.
It sounds like this change would increase the (already slightly too high) level of weirdness in FilePath stat caching. On the other hand, I'm not really sure how link handling should sensibly be done in FilePath at all.
But how should I handle the getAccessTime/etc. functions? Should I add a FilePath.symlink() function (that creates a symlink to said FilePath)?
`FilePath.symlink´ (or maybe better, something like `FilePath.linkTo´) seems like a good addition to the API. `get*Time´ is a bit trickier. An option might be to expose an `lstat´ method and let applications worry about getting the necessary information out of that when they know they care about the link, not its target.
Should I just implement my own subclass of FilePath that handles symlinks and not try to change FilePath?
It'd be great to have anything that's generally useful incorporated into Twisted. It might make sense to have a FilePath subclass for a while to see what is an isn't useful for your SFTP server - or you might be happy enough to briefly have a local copy of Twisted with your modifications and only submit them once you're happy with them. Jean-Paul
Should I just implement my own subclass of FilePath that handles symlinks and not try to change FilePath?
It'd be great to have anything that's generally useful incorporated into Twisted. It might make sense to have a FilePath subclass for a while to see what is an isn't useful for your SFTP server - or you might be happy enough to briefly have a local copy of Twisted with your modifications and only submit them once you're happy with them.
Well, I did write one, and that's when I was told to start submitting my changes into FilePath. :) Basically I need to be able to handle symlinks as links - hence the restat. There are other things I need, such as an openWithFlags method (basically, the ISFPTServer interface specifies that I need to be able to open a file with a bunch of flags (read, write, append, create, truncate, exclusive), and I'd like to simply pass that through to a method. I've tried to come up with a fairly elaborate mapping of said flags onto (r, w, r+, w+, etc.), but there are some things (like the exclusive flag) I don't know what to do with. I'm not sure if this would be useful in FilePath. Another thing I need is to be able to change permissions. My applications ignores requests to change ownership and such, but being able to change permissions I think would be good. And I know that FilePath.copyTo and FilePath.moveTo right now do not preserve permissions or ownership - I'm not sure if this is desired behavior, but if it isn't, then FilePath needs some mechanism for changing the attributes of a file or directory. But then we get into platform problems, and FilePath may get overly complicated.
On Mon, 24 Mar 2008 13:15:46 -0500, Ying Li <cyli@alum.mit.edu> wrote:
Should I just implement my own subclass of FilePath that handles symlinks and not try to change FilePath?
It'd be great to have anything that's generally useful incorporated into Twisted. It might make sense to have a FilePath subclass for a while to see what is an isn't useful for your SFTP server - or you might be happy enough to briefly have a local copy of Twisted with your modifications and only submit them once you're happy with them.
Well, I did write one, and that's when I was told to start submitting my changes into FilePath. :)
Aha. :)
Basically I need to be able to handle symlinks as links - hence the restat.
So are you looking for something like this? fp = FilePath(...) fp.restat(lstat=True) fp.getModificationTime() # link's modification time I'm not sure about that. Aside from the freaky action at a distance, it probably is prone to failures when certain FilePath methods internally decide to restat, thus replacing the link data with data about the target. The cop-out API would be one like this: fp = FilePath(...) fp.lstat().st_mtime # link's modification time I can't think of anything strictly better at the moment, though. What does usage in your application suggest is a good approach?
There are other things I need, such as an openWithFlags method (basically, the ISFPTServer interface specifies that I need to be able to open a file with a bunch of flags (read, write, append, create, truncate, exclusive), and I'd like to simply pass that through to a method. I've tried to come up with a fairly elaborate mapping of said flags onto (r, w, r+, w+, etc.), but there are some things (like the exclusive flag) I don't know what to do with.
It seems like FilePath.open has some issues currently. It would be a good thing if it had an API which allowed explicit, precise control over how the open happened.
I'm not sure if this would be useful in FilePath. Another thing I need is to be able to change permissions. My applications ignores requests to change ownership and such, but being able to change permissions I think would be good. And I know that FilePath.copyTo and FilePath.moveTo right now do not preserve permissions or ownership - I'm not sure if this is desired behavior, but if it isn't, then FilePath needs some mechanism for changing the attributes of a file or directory. But then we get into platform problems, and FilePath may get overly complicated.
FilePath.chmod would be a great addition. `copyTo´ and `moveTo´ should try to preserve permission (at least optionally). I'm not sure if this would have scary consequences (handling failures, for example, complicates the matter), but at least the idea is nice. Jean-Paul
So are you looking for something like this?
fp = FilePath(...) fp.restat(lstat=True) fp.getModificationTime() # link's modification time
I'm not sure about that. Aside from the freaky action at a distance, it probably is prone to failures when certain FilePath methods internally decide to restat, thus replacing the link data with data about the target.
The cop-out API would be one like this:
fp = FilePath(...) fp.lstat().st_mtime # link's modification time
I can't think of anything strictly better at the moment, though. What does usage in your application suggest is a good approach?
Well I had fp.restat(lstat=True) and fp.getModificationTime(lstat=True), and I'd just pass the lstat variable through. Mostly I did this to make it compatible with the existing FilePath. I actually only need the second approach - I sort of need to get all the attributes at once, so convenience attribute-getting methods are kind of useless for me. So the "cop-out" API you suggest actually works better for me.
On 06:30 pm, exarkun@divmod.com wrote:
On Mon, 24 Mar 2008 13:15:46 -0500, Ying Li <cyli@alum.mit.edu> wrote:
So are you looking for something like this?
fp = FilePath(...) fp.restat(lstat=True) fp.getModificationTime() # link's modification time
I'm not sure about that. Aside from the freaky action at a distance, it probably is prone to failures when certain FilePath methods internally decide to restat, thus replacing the link data with data about the target.
The cop-out API would be one like this:
fp = FilePath(...) fp.lstat().st_mtime # link's modification time
I can't think of anything strictly better at the moment, though. What does usage in your application suggest is a good approach?
I am not working on this application, but I have some ideas of my own... It occurs to me that isfile and isdir both use cached stat results, but islink re-lstat()s each time. This could of course give a potentially inconsistent view of the filesystem without restat()'ing. Without regard to backwards compatibility, I would probably suggest that we do something like this: fp = FilePath(...) fp.getModificationTime() # link time fp.followLink().getModificationTime() # destination time (even if dest is a link!) This would allow finer-grained control of what exactly you were asking for. (It would also work more nicely by default with broken symlinks.) However, keeping compatibility in mind (given that this is easily API- compatible, but behavior would change very slightly and ostensibly be less buggy, I'm not sure whether to do this or not): fp = FilePath(...) fp.getModificationTime() # destination time fp.asLink().getModificationTime() # link time fp.asLink().followLink() #... etc This is basically the same as your cop-out example, except we don't introduce yet another interface for other filepath implementations to emulate.
participants (3)
-
glyph＠divmod.com -
Jean-Paul Calderone -
Ying Li