Mailman 3 January 2003 - Python-Dev

New version of PEP 304
by Skip Montanaro Feb. 1, 2003

Feb. 1, 2003

I just checked in a new version of PEP 304. It should be available at http://www.python.org/peps/pep-0304.html within a few hours (look for version >= 1.10). There is also a patch against CVS which implements most of the PEP for Unix-ish systems. That's referenced in the PEP but also available directly at http://www.python.org/sf/677103 Known shortcomings: * Not all regression tests pass yet, mostly (I think) because some tests expect to find auxiliary files in the same directory as mod.__file__. * There is no support yet for Windows paths, but this shouldn't be hard to add. I just can't build on Windows. * You can't delete a source file after generating a .pyc file because the .pyc file won't be in the same directory. I think I will have to modify the search for .pyc files to include the bytecode base directory as well. Feedback welcome. Windows C programmers even more welcome. ;-) Skip

2 1

Re: [Python-Dev] StringIO not an iterator?
by Andrew McNamara Jan. 31, 2003

Jan. 31, 2003

>> I've just noticed that the StringIO in CVS hasn't been updated to be an >> iterator, like real file objects. This probably should be fixed before >> 2.3 is released. > >I think you mean cStringIO. Actually, I meant StringIO, but I see that I had my details wrong - I was just accessing it incorrectly (it's not a self-iterator, and has no next() method). I'll get back in my box now. Thanks. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

3 2

RE: [Python-Dev] the new 2.3a1 settimeout() with httplib and SSL
by Geoffrey Talvola Jan. 31, 2003

Jan. 31, 2003

Ben Laurie [mailto:ben@algroup.co.uk] wrote: > > I've asked Ben Laurie to review the patch for me. Once he > > says it looks OK > > I'll assign both the bug and the patch to Guido to deal > > with. Hopefully > > it's finally right. > > Doesn't seem quite right to me yet - the problem is that if > data arrives > 1 byte at a time with just less than the timeout between each > byte, then > you can get n*timeout as the actual timeout (where n is > potentially very > large). You need to reduce the timeout after each select, surely? Hmmm. The question is, how should the timeout be interpreted? The patch I submitted interprets it as the timeout for activity on the underlying socket itself, since that's the object on which you set the timeout by calling sock.settimeout(). You're suggesting that it should be interpreted as the timeout for the call to read() or write() on the ssl object, which involves potentially many separate socket operations. Anyone else have any opinion on the correct interpretation? I lean toward the one I've already implemented, since it requires no new work :-) > Apart from that, it looks good. Apologies for delay, I managed to > overlook this. No problem. - Geoff

4 5

RE: [Python-Dev] the new 2.3a1 settimeout() with httplib and SSL
by Geoffrey Talvola Jan. 31, 2003

Jan. 31, 2003

Ben Laurie [mailto:ben@algroup.co.uk] wrote: > The point is that in the standard case, a byte on the network is a byte > in the application, so you either get a byte or you time out in the > specified time. > > In the SSL case, you could neither get a byte nor time out, at the > application layer, until much, much later than you thought you specified. > > This seems broken to me, and POLA would suggest I'm right (i.e. if I say > time out in 1 second, I'll be pretty astonished when that turns into a day). > I would agree if we were talking about a timeout that was explicitly set on the SSL layer. But if I set a timeout on a socket, I expect it to apply to the individual low-level socket operations, not to the higher level SSL operations. - Geoff

1 0

RE: [Python-Dev] the new 2.3a1 settimeout() with httplib and SSL
by Geoffrey Talvola Jan. 31, 2003

Jan. 31, 2003

An update: I've uploaded a new patch to set BIO to be non-blocking when necessary and to retry properly when the SSL error code indicates it: http://sourceforge.net/tracker/index.php?func=detail&aid=676472&group_id=547 0&atid=305470 And, as a result of properly retrying on SSL_connect, it happens to also fix this bug when socket.setdefaulttimeout() is used in conjunction with ssl: http://sourceforge.net/tracker/index.php?func=detail&aid=673797&group_id=547 0&atid=105470 I've asked Ben Laurie to review the patch for me. Once he says it looks OK I'll assign both the bug and the patch to Guido to deal with. Hopefully it's finally right. - Geoff > -----Original Message----- > From: Ben Laurie [mailto:ben@algroup.co.uk] > Sent: Tuesday, January 28, 2003 11:53 AM > To: Geoffrey Talvola > Cc: 'python-dev(a)python.org' > Subject: Re: [Python-Dev] the new 2.3a1 settimeout() with httplib and > SSL > > > Geoffrey Talvola wrote: > > Ben Laurie [mailto:ben@algroup.co.uk] wrote: > > > >>Guido van Rossum wrote: > >> > >>>Hm, from that page it looks like the internal implementation may > >>>actually repeatedly read from the socket, until it has processed a > >>>full 16K block. But I may be mistaken, since it also refers to a > >>>non-blocking underlying "BIO", whatever that is. :-( > >> > >>BIO is OpenSSL's I/O abstraction - if you have a nonblocking > >>one, then > >>SSL_read() will return when a read returns nothing, and if you want > >>SSL_read() to not block, then you pretty much have to use a > >>non-blocking > >>BIO (because even if select() says there's data, there may > >>not be enough > >>to actually return any via SSL_read()). > > > > > > That's OK, I think, because what we care about with > timeouts is detecting > > when there is _no_ activity on the socket for more than N > seconds, and > > select() does detect that situation properly. > > > > > >>I can help out here if there's still a problem. > > > > > > If you'd like, you could quickly review the latest checkin > here -- I have no > > prior experience with OpenSSL so that might be prudent: > > > > http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Module > s/_ssl.c > > But it seems to work fine. Yeah, but there are corner cases where it won't. If the other end dies partway through sending an SSL record, then your select will succeed, but the SSL_read will block forever (or at least until the socket closes). You do need to put the socket and BIO into a non-blocking mode for this to work properly. I can't remember whether you get an error or a 0 back (I think its an error) when the socket would block, but in any case, that would need to be handled (presumably by going back around for the remaining time). If you need more info, I can find it :-) Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." - Robert Woodruff

3 4

PEP 305 - CSV File API
by Skip Montanaro Jan. 31, 2003

Jan. 31, 2003

A new PEP (305), "CSV File API", is available for reader feedback. This PEP describes an API and implementation for reading and writing CSV files. There is a sample implementation available as well which you can take out for a spin. The PEP is available at http://www.python.org/peps/pep-0305.html (The latest version as of this note is 1.9. Please wait until that is available to grab a copy on which to comment.) The sample implementation, which is heavily based on Object Craft's existing csv module, is available at http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/nondist/sandbo… To those people who are already using the Object Craft module, make sure you rename your csv.so file before trying this one out. Please send feedback to csv(a)mail.mojam.com. You can subscribe to that list at http://manatee.mojam.com/mailman/listinfo/csv That page contains a pointer to the list archives. (Many thanks BTW to Barry Warsaw and the Mailman crew for Mailman 2.1. It looks awesome.) -- Skip Montanaro skip(a)pobox.com http://www.musi-cal.com/

1 0

Re: [Python-Dev] Property syntax
by Gerald S. Williams Jan. 31, 2003

Jan. 31, 2003

Guido van Rossum wrote: > > def C as class: > > suite > > Um, the whole point of this syntax is that property, synchronized > etc. do *not* have to be keywords -- they are just callables. How many different types of parsing are needed? Could some small number of keywords that determine how the thunk is parsed be intermixed in the same context as the thunk-aware objects? def C(B) as class: def spam() as staticmethod function: ... def spamspam(klass) as classmethod generator: ... def spamspamspam as class property: ... -Jerry

1 0

RE: [Python-Dev] the new 2.3a1 settimeout() with httplib and SSL
by Geoffrey Talvola Jan. 31, 2003

Jan. 31, 2003

Guido van Rossum [mailto:guido@python.org] wrote: > > On Fri, Jan 31, 2003, Geoffrey Talvola wrote: > > > Ben Laurie [mailto:ben@algroup.co.uk] wrote: > > >> > > >> Doesn't seem quite right to me yet - the problem is that if data > > >> arrives 1 byte at a time with just less than the timeout > > >> between each > > >> byte, then you can get n*timeout as the actual timeout > > >> (where n is > > >> potentially very large). You need to reduce the timeout > > >> after each > > >> select, surely? > > > > > > Hmmm. The question is, how should the timeout be interpreted? > > > > > > The patch I submitted interprets it as the timeout for > > > activity on the > > > underlying socket itself, since that's the object on which you set > > > the timeout by calling sock.settimeout(). You're > > > suggesting that it > > > should be interpreted as the timeout for the call to > > > read() or write() > > > on the ssl object, which involves potentially many separate socket > > > operations. > > > > > > Anyone else have any opinion on the correct > > > interpretation? I lean > > > toward the one I've already implemented, since it > > > requires no new work > > > :-) > > > > I'm in favor of it referring to network activity, since that's the > > interpretation used by non-SSL timeouts. > > -- > > Aahz (aahz(a)pythoncraft.com) <*> > http://www.pythoncraft.com/ > > I don't understand what Aahz meant, but if this is the only issue, I'm > with Geoff. In general, the timeout gets reset whenever a byte is > received. I think that's exactly what Aahz meant. In that case, my patch on SF should be good. I've assigned it to you to check in. While you're at it, you might also take a look at patch 678257 which fixes a case where socket.sendall() would return before its timeout period expired. Basically, it was only waiting for timeout on the first pass through the loop instead of on every iteration. - Geoff

1 0

Re: [Python-Dev] PEP 42: sizeof(obj) builtin
by Zooko Jan. 31, 2003

Jan. 31, 2003

GvR wrote: > > But the question is incredibly ill-defined, so it's not clear that > Raymond will be able to calculate a useful answer. (See Tim's post.) Even a simple measure which double-counted everything might be useful to me. Or perhaps a simple measure which counted only memory that would be freed by destroying this object. I am after all looking for a needle (excessive memory usage) which is two orders of magnitude bigger than the hay (memory usage that I am sure I want). I don't know what the very *best* kind of answer would be. Perhaps a recursively defined "weight", where an object which holds the *only* reference to another object carries the weight of that other object, and an object which holds one of K references to another object carries 1/Kth of its weight. I'm not sure that would be useful, but it might be. Regards, Zooko http://zooko.com/ http://mnet.sf.net/

2 1

Re: [Python-Dev] PEP 42: sizeof(obj) builtin
by Zooko Jan. 31, 2003

Jan. 31, 2003

Raymond Hettinger <raymond.hettinger(a)verizon.net> wrote: > > I'm about to start working on this one and wanted > to check here first to make sure there is still a > demand for it [...] Yes, please! I have wished for this many times as I struggled to figure out which part of my program was using up too much RAM. Regards, Zooko http://zooko.com/ ^-- my web page http://mnet.sf.net/ ^-- my big, RAM-over-using, Python program

3 2