[Tutor] sockets, files, threads

[Marilyn Davis]

> 
> > What about when I do an explicit call to a close inside a __del__.  Is
> > that a bad idea?
> 
> I usually prefer to add a close() method to my objects that releases
> resources, rather than __del__(), because it's more visible.
> 

OK Danny!  I found it!  When I was almost asleep last night, I knew
what to do and this morning I whittled it down to this situation.
Maybe it can be whittled further but I'm holding up our project and I
can't take any more time right now.

Note that the two variables TO_WHOM and MAILER, I think, can be safely
changed to suit your environment, indeed MAILER='mail' should work for
others.

It happens on 2.4 but not on 2.3.  Either 2.3 masks my error, or 2.4
has an error?

I wouldn't be surprised if this is responsible for my thread problem
but we're not going to find out.

----
#!/usr/bin/env python2.4
'''On my python2.4, this produces:
./test_exim.py
close failed: [Errno 9] Bad file descriptor
close failed: [Errno 9] Bad file descriptor
close failed: [Errno 9] Bad file descriptor

but it runs just fine.

but if I specify python2.3, it runs cleanly.

'''

TO_WHOM = 'marilyn'
MAILER = 'exim'

import popen2
import select
import os

class Exim:
     def __init__(self):
         self.fdin = None
         self.err_flush = []
         self.stdout, self.stdin, self.stderr  = popen2.popen3('%s -t' % MAILER)
         self.fdin = self.stdin.fileno()
         self.fdout = self.stdout.fileno()
         self.fderr = self.stderr.fileno()
         self.outlist = [self.fdout, self.fderr]
         self.inlist = [self.fdin]

     def __del__(self):
         self.close_up()

     def close_up(self):
         if not self.fdin:
             return
         count = 0
         in_status = os.close(self.fdin)
         while 1:
             sel = select.select(self.outlist, [], [], 1)
             if sel[0] == []:
                 break
             else:
                 for fd in sel[0]:
                     r = os.read(fd, 16384)
                     if r:
                         self.err_flush += [r]
                         count = 0
                     count += 1
                     if count >= 5:
                         break
                 else:
                     continue
             break
         self.err_flush = ''.join(self.err_flush)
         out_status = os.close(self.fdout)
         err_status = os.close(self.fderr)
         self.fdin = None
         if not in_status:
             return
         raise RuntimeError, self.err_flush

     def write(self, stuff):
         while 1:
             sel = select.select(self.outlist, self.inlist, [], 1)
             if sel[1] != []:
                 os.write(sel[1][0], stuff)
                 return

################################################################
if __name__ == '__main__':
     msg = '''To: %s

xx''' % TO_WHOM

     p = Exim()
     p.write(msg)
     del p

----

On Tue, 18 Jan 2005, Danny Yoo wrote:
> 
> [About using the Standard Library]
> 
> > And since then, please don't shoot me, but I don't immediately trust the
> > modules.  I read them and see how many times they loop through the data,
> > and how many copies of the data they put into memory -- and usually
> > decide to write the simple things I need myself, looping zero times and
> > keeping only one block in memory.
> 
> Hmm.. . Do you remember which Standard Library modules you were looking at
> earlier?  Perhaps there was some funky stuff happening, in which case we
> should try to fix it, so that no one else runs into the same problems.


Yes.  poplib and socket.  But, it's not fair to assume that something
funky is happening in there.  A poplib should provide a set of tools
for making a pop client and I'm not making a pop client, just pumping
the message from a pop server through my python/mysql magic and into
exim.  Similarly with socket.  It does buffering and prepares things
that I don't need, but probably lots of people do.  So when I say I
don't "trust" stdlib modules, I mean I don't trust them to be simple
enough for my situation.

However, all that said, I do dimly remember that poplib perhaps had
some extra processing that maybe is not needed -- but I could be
wrong.

> In a similar vein, in Spawn.run(), it might be a good idea to explicitely
> call write_and_close() on our FileSocket instance.  For example, we can
> add a try/finally in the body of the revised start()  method:
> 
> ###
>     def start(self):
>         '''Given the command, provides the function to call.'''
>         global TESTING
>         function =  { 'acl_rcpt.py' : calls.acl_rcpt,
>                       'route_mail.py' : calls.route_mail,
>                       'route_errors.py' : calls.route_errors,
>                       'doorman.py' : calls.doorman,
>                       'doorman_errors.py' : calls.doorman_errors,
>                       'is_known_to.py' : is_known_to
>                       }
>         if log.level & log.calls:
>             log.it('fd%d: %s: %s', self.descriptor, self.exim_io.call,
>                    self.exim_io.collector_name)
>         try:
>             function[self.exim_io.call].main(self.exim_io)
>         finally:
>             self.exim_io.write_and_close()
> ###
> 
> By using try/finally, we can guarantee that finalizers like
> 'write_and_close()' can be called at the end.
> 
> I couldn't tell when write_and_close() was getting called in the original
> code; I suspected that each of the 'calls' dispatch functions was
> individually responsible for calling write_and_close(), but I wasn't sure.
> 

Yes, the function[..] is responsible to write_and_close at the proper
time in its processing, which varies from function to function.  At
this point we are very intimate with exim.  Right now a failure to
write_and_close causes mail to be deferred, which is good, but it does
it ungracefully.  So maybe I will fiddle there to make it better.

> 
> Best of wishes to you!

Thank you.  Things have improved.

Are you well yet?  A lot of sick people around these days!

M.


> 
> 

--