[ python-Bugs-1469557 ] FTP modue functions are not re-entrant, give odd exceptions
SourceForge.net
noreply at sourceforge.net
Fri Jul 28 19:02:42 CEST 2006
Bugs item #1469557, was opened at 2006-04-12 17:13
Message generated for change (Comment added) made by brucepeterson
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1469557&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Closed
Resolution: Invalid
Priority: 5
Submitted By: brucepeterson (brucepeterson)
Assigned to: Nobody/Anonymous (nobody)
Summary: FTP modue functions are not re-entrant,give odd exceptions
Initial Comment:
If I define a class using the Thread and FTP moudles,
start a process which gathers FTP responses,
additional calls to the class may have the responses
of the thread instead of the main loop (or vice versa)
This causes weird and unexpected exceptions from the
ftplib.
For instance I get the following error when the
thread process does a pwd() function
error_reply: 213 34603008
The "213" reply is a response from the main process
size() function
---------
Code
---------
from time import sleep
from threading import Thread
class ftpMachine(Thread, ftplib.FTP):
def __init__(self, svr, user, passwd):
Thread.__init__(self)
ftplib.FTP.__init__(self, svr, user, passwd)
def run(self):
for x in xrange(20):
output="Thread -"+str(self.nlst())[:30]
print output
sleep (0.0100)
def main():
aCon = ftpMachine("LocalFTP", "user", "")
aCon.start()
for x in xrange(20):
output = "Main -- " + str(aCon.size("File"))
print output
sleep (0.010)
Workround:
Rewrite code to create a third worker thread for
response isolation? Don't know that this would solve
the problem.
---------------
Exception example
---------------
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Python24\lib\threading.py", line 442, in
__bootstrap
self.run()
File "dualFTPIssue.py", line 17, in run
output = "Thread output " + str(self.nlst())[:30]
+" ..."
File "C:\Python24\lib\ftplib.py", line 448, in nlst
self.retrlines(cmd, files.append)
File "C:\Python24\lib\ftplib.py", line 396, in
retrlines
conn = self.transfercmd(cmd)
File "C:\Python24\lib\ftplib.py", line 345, in
transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "C:\Python24\lib\ftplib.py", line 321, in
ntransfercmd
host, port = self.makepasv()
File "C:\Python24\lib\ftplib.py", line 299, in
makepasv
host, port = parse227(self.sendcmd('PASV'))
File "C:\Python24\lib\ftplib.py", line 566, in
parse227
raise error_reply, resp
error_reply: 213 34603008
----------------------------------------------------------------------
>Comment By: brucepeterson (brucepeterson)
Date: 2006-07-28 10:02
Message:
Logged In: YES
user_id=1500983
I am running on a unit that limits the FTP connections to 4
high bandwidth connections and normally uses a passive FTP
transfer.
The response from the passive FTP transfer could come at
any time. Also, by inspection, the FTP module throws an
exception when an unexpected reply happens. So when I'm
querrying the size of the clip on the destination and
instead get a transfer complete message from the passive
FTP session, that transfer complete message causes the
exception.
To "fix" this issue would requre that when the FTP
message queue is read, that pending requests (passive
transfers) would be considered before returning immediate
command responses (cwd, ls, size, etc.). If I get a couple
of days, I may rewrite the functions that read the FTP
responses and submit the changes for consideration.
For my current project, I have decided not to querry the
transfer to see if it completes, but instead to do a simple
wait before sending the next transfer.
----------------------------------------------------------------------
Comment By: Ronald Oussoren (ronaldoussoren)
Date: 2006-07-27 23:26
Message:
Logged In: YES
user_id=580910
I don't understand why you cannot use two connection resources. Are you
running in a severely resource constrained environment?
Anyway, what you're doing right now is undefined behaviour. Unless explicitly
stated otherwise classes in the stdlib aren't fully reentrant. You will therefore
have to arrange some kind of exclusion mechanism. One way of doing that is
to introduce a lock (probably wrapping the FTP client instead of using
multiple inheritance in the progress). Another way it to introduce a 3th thread
that performs the actual FTP operations and communicate with that using a
queue. Please ask around on python-list/comp.lang.python if you need help
with this.
BTW. I didn't close this bug, although I do understand why it was closed: the
behaviour you describe isn't a bug but expected behaviour.
----------------------------------------------------------------------
Comment By: brucepeterson (brucepeterson)
Date: 2006-07-27 14:39
Message:
Logged In: YES
user_id=1500983
Unfortunately I can't start two instances of the FTP client
as the separate instances use two conneciton resources. I
was hoping for a non-exception when a unexpsected response
happens. That would require re-working the module.
For my workaround I set a time to determine when the
transfer should complete instead of a querry.
Thanks for your attention in this matter.
Closing this bug as my case is an unusual one.
----------------------------------------------------------------------
Comment By: Ronald Oussoren (ronaldoussoren)
Date: 2006-07-13 04:54
Message:
Logged In: YES
user_id=580910
IMHO this isn't a bug, your accessing a shared resource (the FTP connection)
from two threads without locking. Most of python's libraries aren't safe for this
kind of use.
BTW. The fact that you subclass form thread and ftplib.FTP is immaterial, you
will get the same effect if you create an ftplib.FTP and then use it in two threads.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1469557&group_id=5470
More information about the Python-bugs-list
mailing list