[Twisted-Python] SFTP Server sample
Hi, I am a newbie in twisted and python. I would like to create a python SFTP server with twisted, but I cannot find any sample. Can someone give me some advise? Thanks, Osborn
On Fri, Aug 19, 2011 at 2:43 PM, Osborn Chan <ychan@quova.com> wrote:
I don't believe there are any examples of building an SFTP server aimed at newcomers to twisted and python but there are existing implementations in the wild--which might honestly be difficult to digest if you don't know python well enough. I think zope at one time has SFTP server based on twisted (which I can't seem to summon with google) and there's also SFTP server implementation in tahoe-lafs (https://github.com/warner/tahoe-lafs/blob/master/src/allmydata/frontends/sft...). -Drew
Yes. It implements SFTP pretty much the way documents SFTP. Unfortunately. Let's just say that SFTP is not the easiest of protocols when you just want something like os/shutil but remote-ier. cheers lvh
Yes. It is SFTP, and I found that filetransfer.FileTransferServer is a subsystem under UnixConchUser. self.subsystemLookup.update({"sftp": filetransfer.FileTransferServer}) However, I was able to start a ssh shell by using SSHFactory and UnixSSHReam, but I was not able to start SFTP as sub system. Do I need to do anything explicitly to start sftp subsystem? Thanks, -Osborn newbie in Python and Twisted. On 8/22/11 12:50 PM, "Jasper St. Pierre" <jstpierre@mecheye.net> wrote:
Not sure what you mean, but here is a basic example: from zope.interface import implements from twisted.application import service, internet from twisted.conch.ssh.keys import Key from twisted.conch.ssh.factory import SSHFactory from twisted.conch.unix import UnixSSHRealm from twisted.cred.checkers import ICredentialsChecker from twisted.cred.credentials import IUsernamePassword from twisted.cred.portal import Portal def get_key(path): return Key.fromString(data=open(path).read()) class DummyChecker(object): credentialInterfaces = (IUsernamePassword,) implements(ICredentialsChecker) def requestAvatarId(self, credentials): return credentials.username def makeService(): public_key = get_key('id_rsa.pub') private_key = get_key('id_rsa') factory = SSHFactory() factory.privateKeys = {'ssh-rsa': private_key} factory.publicKeys = {'ssh-rsa': public_key} factory.portal = Portal(UnixSSHRealm()) factory.portal.registerChecker(DummyChecker()) return internet.TCPServer(2200, factory) application = service.Application("sftp server") sftp_server = makeService() sftp_server.setServiceParent(application) Put this content into a file like sftp.tac The keys can be generated by ckeygen utility which is part of Twisted (e.g. ckeygen -b 2048 -t rsa -f id_rsa) and run with twistd -ny sftp.tac If you want more background on Twisted conch i suggest the "Twisted Conch in 60 Seconds" series by Jean-Paul Calderone on his blog http://as.ynchrono.us/ Michal Mach On Mon, Aug 22, 2011 at 10:31 PM, Osborn Chan <ychan@quova.com> wrote:
Hi Michal, Thanks a lot! It is a very nice working example, and it is good for newbie as well. Regards, Osborn From: Michal Mach <michal.mach@gmail.com<mailto:michal.mach@gmail.com>> Reply-To: Twisted general discussion <twisted-python@twistedmatrix.com<mailto:twisted-python@twistedmatrix.com>> Date: Tue, 23 Aug 2011 00:31:00 +0200 To: Twisted general discussion <twisted-python@twistedmatrix.com<mailto:twisted-python@twistedmatrix.com>> Subject: Re: [Twisted-Python] SFTP Server sample Not sure what you mean, but here is a basic example: from zope.interface import implements from twisted.application import service, internet from twisted.conch.ssh.keys import Key from twisted.conch.ssh.factory import SSHFactory from twisted.conch.unix import UnixSSHRealm from twisted.cred.checkers import ICredentialsChecker from twisted.cred.credentials import IUsernamePassword from twisted.cred.portal import Portal def get_key(path): return Key.fromString(data=open(path).read()) class DummyChecker(object): credentialInterfaces = (IUsernamePassword,) implements(ICredentialsChecker) def requestAvatarId(self, credentials): return credentials.username def makeService(): public_key = get_key('id_rsa.pub') private_key = get_key('id_rsa') factory = SSHFactory() factory.privateKeys = {'ssh-rsa': private_key} factory.publicKeys = {'ssh-rsa': public_key} factory.portal = Portal(UnixSSHRealm()) factory.portal.registerChecker(DummyChecker()) return internet.TCPServer(2200, factory) application = service.Application("sftp server") sftp_server = makeService() sftp_server.setServiceParent(application) Put this content into a file like sftp.tac The keys can be generated by ckeygen utility which is part of Twisted (e.g. ckeygen -b 2048 -t rsa -f id_rsa) and run with twistd -ny sftp.tac If you want more background on Twisted conch i suggest the "Twisted Conch in 60 Seconds" series by Jean-Paul Calderone on his blog http://as.ynchrono.us/ Michal Mach On Mon, Aug 22, 2011 at 10:31 PM, Osborn Chan <ychan@quova.com<mailto:ychan@quova.com>> wrote: Yes. It is SFTP, and I found that filetransfer.FileTransferServer is a subsystem under UnixConchUser. self.subsystemLookup.update({"sftp": filetransfer.FileTransferServer}) However, I was able to start a ssh shell by using SSHFactory and UnixSSHReam, but I was not able to start SFTP as sub system. Do I need to do anything explicitly to start sftp subsystem? Thanks, -Osborn newbie in Python and Twisted. On 8/22/11 12:50 PM, "Jasper St. Pierre" <jstpierre@mecheye.net<mailto:jstpierre@mecheye.net>> wrote:
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com<mailto:Twisted-Python@twistedmatrix.com> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com<mailto:Twisted-Python@twistedmatrix.com> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 08/23/2011 01:10 AM, Osborn Chan wrote:
I attached it to http://twistedmatrix.com/trac/ticket/5237 so it could be included in the documentation. Cheers, Thijs -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk5TqU0ACgkQBRbMGagbo3lrAgCfR7lAPz5f+4J/R5bZmkFJA/iB cW4An2Iqi+CGLZftFV+JZlh/R+UmsAkC =WET7 -----END PGP SIGNATURE-----
FYI: The sample code is working on my side, but I had a problem to list the files with permission issue. I guessed it may be a issue related to my server setting. However, it would be good if we have another person to test the code as well. Thanks On 8/23/11 6:21 AM, "Thijs Triemstra" <lists@collab.nl> wrote:
Are you getting the error "Operation not permitted" when you try to do directory listing? This can be caused by running the SFTP server under a user which can't change his effective group and uid to 0 (uid of root user). This is because when you login as user "foo" the server needs to execute all subsequent commands as this user (e.g. when you upload a file this file should be owned by the user "foo") and therefore temporary changing to user "foo". This is exactly what the default implementation of UnixConchUser wants to do, but fails and therefore you got that error. Regards, Michal Mach On Tue, Aug 23, 2011 at 7:54 PM, Osborn Chan <ychan@quova.com> wrote:
Hi Michal, Actually, I was running the example under root. (sudo) I was able to list/put/get file for my non-root user after login. However, listing (ls) doesn't work on top level directory (e.g. /home/ychan), but it works fine anywhere else. The following shows the issue. ychan@ubuntu:~/sandbox$ sftp -P 2200 localhost ychan@localhost's password: Connected to localhost. sftp> ls Couldn't read directory: Permission denied I guess there may be some issue related to my machine, and I may check it later. Thanks for you help, Osborn From: Michal Mach <michal.mach@gmail.com<mailto:michal.mach@gmail.com>> Reply-To: Twisted general discussion <twisted-python@twistedmatrix.com<mailto:twisted-python@twistedmatrix.com>> Date: Wed, 24 Aug 2011 03:14:25 +0200 To: Twisted general discussion <twisted-python@twistedmatrix.com<mailto:twisted-python@twistedmatrix.com>> Subject: Re: [Twisted-Python] SFTP Server sample e server needs to execute all subsequent commands as this user (e.g. when you upload a file this file should be owned by the user "foo") and therefore temporary changing to user "foo"
On Wed, 2011-08-24 at 02:14 +0000, Osborn Chan wrote:
Your home folder most probably contains some special files/pipes for which Twisted can not read the attributes. When listing a folder, twisted FTP and SFTP servers are querying for file attributes and they will return permission denied if one of the file attributes can not be retrieved. I am not sure if this is a bug or a feature. Cheers, Adi
On Fri, Aug 19, 2011 at 2:43 PM, Osborn Chan <ychan@quova.com> wrote:
I don't believe there are any examples of building an SFTP server aimed at newcomers to twisted and python but there are existing implementations in the wild--which might honestly be difficult to digest if you don't know python well enough. I think zope at one time has SFTP server based on twisted (which I can't seem to summon with google) and there's also SFTP server implementation in tahoe-lafs (https://github.com/warner/tahoe-lafs/blob/master/src/allmydata/frontends/sft...). -Drew
Yes. It implements SFTP pretty much the way documents SFTP. Unfortunately. Let's just say that SFTP is not the easiest of protocols when you just want something like os/shutil but remote-ier. cheers lvh
Yes. It is SFTP, and I found that filetransfer.FileTransferServer is a subsystem under UnixConchUser. self.subsystemLookup.update({"sftp": filetransfer.FileTransferServer}) However, I was able to start a ssh shell by using SSHFactory and UnixSSHReam, but I was not able to start SFTP as sub system. Do I need to do anything explicitly to start sftp subsystem? Thanks, -Osborn newbie in Python and Twisted. On 8/22/11 12:50 PM, "Jasper St. Pierre" <jstpierre@mecheye.net> wrote:
Not sure what you mean, but here is a basic example: from zope.interface import implements from twisted.application import service, internet from twisted.conch.ssh.keys import Key from twisted.conch.ssh.factory import SSHFactory from twisted.conch.unix import UnixSSHRealm from twisted.cred.checkers import ICredentialsChecker from twisted.cred.credentials import IUsernamePassword from twisted.cred.portal import Portal def get_key(path): return Key.fromString(data=open(path).read()) class DummyChecker(object): credentialInterfaces = (IUsernamePassword,) implements(ICredentialsChecker) def requestAvatarId(self, credentials): return credentials.username def makeService(): public_key = get_key('id_rsa.pub') private_key = get_key('id_rsa') factory = SSHFactory() factory.privateKeys = {'ssh-rsa': private_key} factory.publicKeys = {'ssh-rsa': public_key} factory.portal = Portal(UnixSSHRealm()) factory.portal.registerChecker(DummyChecker()) return internet.TCPServer(2200, factory) application = service.Application("sftp server") sftp_server = makeService() sftp_server.setServiceParent(application) Put this content into a file like sftp.tac The keys can be generated by ckeygen utility which is part of Twisted (e.g. ckeygen -b 2048 -t rsa -f id_rsa) and run with twistd -ny sftp.tac If you want more background on Twisted conch i suggest the "Twisted Conch in 60 Seconds" series by Jean-Paul Calderone on his blog http://as.ynchrono.us/ Michal Mach On Mon, Aug 22, 2011 at 10:31 PM, Osborn Chan <ychan@quova.com> wrote:
Hi Michal, Thanks a lot! It is a very nice working example, and it is good for newbie as well. Regards, Osborn From: Michal Mach <michal.mach@gmail.com<mailto:michal.mach@gmail.com>> Reply-To: Twisted general discussion <twisted-python@twistedmatrix.com<mailto:twisted-python@twistedmatrix.com>> Date: Tue, 23 Aug 2011 00:31:00 +0200 To: Twisted general discussion <twisted-python@twistedmatrix.com<mailto:twisted-python@twistedmatrix.com>> Subject: Re: [Twisted-Python] SFTP Server sample Not sure what you mean, but here is a basic example: from zope.interface import implements from twisted.application import service, internet from twisted.conch.ssh.keys import Key from twisted.conch.ssh.factory import SSHFactory from twisted.conch.unix import UnixSSHRealm from twisted.cred.checkers import ICredentialsChecker from twisted.cred.credentials import IUsernamePassword from twisted.cred.portal import Portal def get_key(path): return Key.fromString(data=open(path).read()) class DummyChecker(object): credentialInterfaces = (IUsernamePassword,) implements(ICredentialsChecker) def requestAvatarId(self, credentials): return credentials.username def makeService(): public_key = get_key('id_rsa.pub') private_key = get_key('id_rsa') factory = SSHFactory() factory.privateKeys = {'ssh-rsa': private_key} factory.publicKeys = {'ssh-rsa': public_key} factory.portal = Portal(UnixSSHRealm()) factory.portal.registerChecker(DummyChecker()) return internet.TCPServer(2200, factory) application = service.Application("sftp server") sftp_server = makeService() sftp_server.setServiceParent(application) Put this content into a file like sftp.tac The keys can be generated by ckeygen utility which is part of Twisted (e.g. ckeygen -b 2048 -t rsa -f id_rsa) and run with twistd -ny sftp.tac If you want more background on Twisted conch i suggest the "Twisted Conch in 60 Seconds" series by Jean-Paul Calderone on his blog http://as.ynchrono.us/ Michal Mach On Mon, Aug 22, 2011 at 10:31 PM, Osborn Chan <ychan@quova.com<mailto:ychan@quova.com>> wrote: Yes. It is SFTP, and I found that filetransfer.FileTransferServer is a subsystem under UnixConchUser. self.subsystemLookup.update({"sftp": filetransfer.FileTransferServer}) However, I was able to start a ssh shell by using SSHFactory and UnixSSHReam, but I was not able to start SFTP as sub system. Do I need to do anything explicitly to start sftp subsystem? Thanks, -Osborn newbie in Python and Twisted. On 8/22/11 12:50 PM, "Jasper St. Pierre" <jstpierre@mecheye.net<mailto:jstpierre@mecheye.net>> wrote:
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com<mailto:Twisted-Python@twistedmatrix.com> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com<mailto:Twisted-Python@twistedmatrix.com> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 08/23/2011 01:10 AM, Osborn Chan wrote:
I attached it to http://twistedmatrix.com/trac/ticket/5237 so it could be included in the documentation. Cheers, Thijs -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk5TqU0ACgkQBRbMGagbo3lrAgCfR7lAPz5f+4J/R5bZmkFJA/iB cW4An2Iqi+CGLZftFV+JZlh/R+UmsAkC =WET7 -----END PGP SIGNATURE-----
FYI: The sample code is working on my side, but I had a problem to list the files with permission issue. I guessed it may be a issue related to my server setting. However, it would be good if we have another person to test the code as well. Thanks On 8/23/11 6:21 AM, "Thijs Triemstra" <lists@collab.nl> wrote:
Are you getting the error "Operation not permitted" when you try to do directory listing? This can be caused by running the SFTP server under a user which can't change his effective group and uid to 0 (uid of root user). This is because when you login as user "foo" the server needs to execute all subsequent commands as this user (e.g. when you upload a file this file should be owned by the user "foo") and therefore temporary changing to user "foo". This is exactly what the default implementation of UnixConchUser wants to do, but fails and therefore you got that error. Regards, Michal Mach On Tue, Aug 23, 2011 at 7:54 PM, Osborn Chan <ychan@quova.com> wrote:
Hi Michal, Actually, I was running the example under root. (sudo) I was able to list/put/get file for my non-root user after login. However, listing (ls) doesn't work on top level directory (e.g. /home/ychan), but it works fine anywhere else. The following shows the issue. ychan@ubuntu:~/sandbox$ sftp -P 2200 localhost ychan@localhost's password: Connected to localhost. sftp> ls Couldn't read directory: Permission denied I guess there may be some issue related to my machine, and I may check it later. Thanks for you help, Osborn From: Michal Mach <michal.mach@gmail.com<mailto:michal.mach@gmail.com>> Reply-To: Twisted general discussion <twisted-python@twistedmatrix.com<mailto:twisted-python@twistedmatrix.com>> Date: Wed, 24 Aug 2011 03:14:25 +0200 To: Twisted general discussion <twisted-python@twistedmatrix.com<mailto:twisted-python@twistedmatrix.com>> Subject: Re: [Twisted-Python] SFTP Server sample e server needs to execute all subsequent commands as this user (e.g. when you upload a file this file should be owned by the user "foo") and therefore temporary changing to user "foo"
On Wed, 2011-08-24 at 02:14 +0000, Osborn Chan wrote:
Your home folder most probably contains some special files/pipes for which Twisted can not read the attributes. When listing a folder, twisted FTP and SFTP servers are querying for file attributes and they will return permission denied if one of the file attributes can not be retrieved. I am not sure if this is a bug or a feature. Cheers, Adi
participants (7)
-
Adi Roiban -
Drew Smathers -
Jasper St. Pierre -
Laurens Van Houtven -
Michal Mach -
Osborn Chan -
Thijs Triemstra