Python-Dev June 2007

python-dev@python.org

66 participants
76 discussions

by A.M. Kuchling

What was the outcome of the discussion of Georg Brandl's reworked documentation ("The docs, reloaded")? Was any decision made on whether to go with reST, or on what changes need to made before that's possible? Did Fred Drake say what he thought? Georg, do you want access to python.org to host a version of the docs there? --amk

13 years, 10 months

Requesting commit access to python sandbox. Cleanup urllib2 - Summer of Code 2007 Project

by Senthil Kumaran

Hi, I am a student participant of Google Summer of Code 2007 and I am working on the cleanup task of urllib2, with Skip as my mentor. I would like to request for a commit access to the Python Sandbox for implementing the changes as part of the project. I have attached by SSH Public keys. preferred name : senthil.kumaran I am following up and adding comments to the urllib related bugs at sf.net page. I would also like to request addition of my sourceforge id : orsenthil to the python project, so I can close the defects raised against urllib modules. Summer of Code Project: http://code.google.com/soc/psf/appinfo.html?csaid=E73A6612F80229B6 The project actually commenced on May 28th itself. But, there was a delay from my side to get started. Ivan Sutherland's essay on Technology and Courage [1] did some good thing to me. :-) Thanks, Senthil [1] http://research.sun.com/techrep/Perspectives/smli_ps-1.pdf#search=%22suther… -- O.R.Senthil Kumaran http://phoe6.livejournal.com

13 years, 10 months

[RFC] urlparse - parse query facility

by Senthil Kumaran

Hi all, This mail is a request for comments on changes to urlparse module. We understand that urlparse returns the 'complete query' value as the query component and does not provide the facilities to separate the query components. User will have to use the cgi module (cgi.parse_qs) to get the query parsed. There has been a discussion in the past, on having a method of parse query string available from urlparse module itself. [1] To implement the query parse feature in urlparse module, we can: a) import cgi and call cgi module's query_ps. This approach will have problems as it i) imports cgi for urlparse module. ii) cgi module in turn imports urllib and urlparse. b) Implement a stand alone query parsing facility in urlparse *AS IN* cgi module. Below method implements the urlparse_qs(url, keep_blank_values,strict_parsing) that will help in parsing the query component of the url. It behaves same as the cgi.parse_qs. Please let me know your comments on the below code. ---------------------------------------------------------------------- def unquote(s): """unquote('abc%20def') -> 'abc def'.""" res = s.split('%') for i in xrange(1, len(res)): item = res[i] try: res[i] = _hextochr[item[:2]] + item[2:] except KeyError: res[i] = '%' + item except UnicodeDecodeError: res[i] = unichr(int(item[:2], 16)) + item[2:] return "".join(res) def urlparse_qs(url, keep_blank_values=0, strict_parsing=0): """Parse a URL query string and return the components as a dictionary. Based on the cgi.parse_qs method.This is a utility function provided with urlparse so that users need not use cgi module for parsing the url query string. Arguments: url: URL with query string to be parsed keep_blank_values: flag indicating whether blank values in URL encoded queries should be treated as blank strings. A true value indicates that blanks should be retained as blank strings. The default false value indicates that blank values are to be ignored and treated as if they were not included. strict_parsing: flag indicating what to do with parsing errors. If false (the default), errors are silently ignored. If true, errors raise a ValueError exception. """ scheme, netloc, url, params, querystring, fragment = urlparse(url) pairs = [s2 for s1 in querystring.split('&') for s2 in s1.split(';')] query = [] for name_value in pairs: if not name_value and not strict_parsing: continue nv = name_value.split('=', 1) if len(nv) != 2: if strict_parsing: raise ValueError, "bad query field: %r" % (name_value,) # Handle case of a control-name with no equal sign if keep_blank_values: nv.append('') else: continue if len(nv[1]) or keep_blank_values: name = unquote(nv[0].replace('+', ' ')) value = unquote(nv[1].replace('+', ' ')) query.append((name, value)) dict = {} for name, value in query: if name in dict: dict[name].append(value) else: dict[name] = [value] return dict ---------------------------------------------------------------------- Testing: $ python Python 2.6a0 (trunk, Jun 10 2007, 12:04:03) [GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import urlparse >>> dir(urlparse) ['BaseResult', 'MAX_CACHE_SIZE', 'ParseResult', 'SplitResult', '__all__', '__builtins__', '__doc__', '__file__', '__name__', '_parse_cache', '_splitnetloc', '_splitparams', 'clear_cache', 'non_hierarchical', 'scheme_chars', 'test', 'test_input', 'unquote', 'urldefrag', 'urljoin', 'urlparse', 'urlparse_qs', 'urlsplit', 'urlunparse', 'urlunsplit', 'uses_fragment', 'uses_netloc', 'uses_params', 'uses_query', 'uses_relative'] >>> URL = >>> 'http://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=utf-8&q=south+africa+tra…' >>> print urlparse.urlparse_qs(URL) {'q': ['south africa travel cape town'], 'oe': ['utf-8'], 'ie': ['UTF-8'], 'hl': ['en']} >>> print urlparse.urlparse_qs(URL,keep_blank_values=1) {'q': ['south africa travel cape town'], 'ie': ['UTF-8'], 'oe': ['utf-8'], 'lr': [''], 'hl': ['en']} >>> Thanks, Senthil [1] http://mail.python.org/pipermail/tutor/2002-August/016823.html -- O.R.Senthil Kumaran http://phoe6.livejournal.com

13 years, 10 months

sys.setdefaultencoding() vs. csv module + unicode

by Kumar McMillan

I'm seeing conflicting opinions on whether to put sys.setdefaultencoding('utf-8') in sitecustomize.py or not ([1] vs. [2]) and frankly I'm confused. The csv module says it's not unicode safe but the 2.5 docs [3] have a workaround for this. While the workaround says nothing about sys.setdefaultencoding() it simply does not work with the default encoding, "ascii." Is this _the_ problem with the csv module? Should I give up and use XML? Below is code that works vs. code that doesn't. Am I interpretting the workaround from the docs wrong? If so, can someone please give me a hint ;) I should also point out that I've tried this with the StringIO queued approach (from the workaround) but that doesn't solve anything. 1) with the default encoding : kumar$ python2.5 Python 2.5 (r25:51918, Sep 19 2006, 08:49:13) [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin >>> import sys, csv, codecs >>> f = codecs.open('unicsv.csv','wb','utf-8') >>> w = csv.writer(f) >>> w.writerow([u'lang', u'espa\xa4ol']) Traceback (most recent call last): File "<stdin>", line 1, in <module> UnicodeEncodeError: 'ascii' codec can't encode character u'\xa4' in position 4: ordinal not in range(128) >>> 2) with custom encoding : kumar$ python2.5 -S Python 2.5 (r25:51918, Sep 19 2006, 08:49:13) [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin >>> import sys, csv, codecs >>> sys.setdefaultencoding('utf-8') >>> f = codecs.open('unicsv.csv','wb','utf-8') >>> w = csv.writer(f) >>> w.writerow([u'lang', u'espa\xa4ol']) >>> f.close() thanks, Kumar [1] http://mail.python.org/pipermail/python-dev/2007-June/073593.html [2] http://diveintopython.org/xml_processing/unicode.html [3] http://docs.python.org/lib/csv-examples.html#csv-examples

13 years, 10 months

[RFC] urlparse - parse query facility

by Jim Jewett

> a) import cgi and call cgi module's query_ps. [circular imports] or > b) Implement a stand alone query parsing facility in urlparse *AS IN* > cgi module. Assuming (b), please remove the (code for the) parsing from the cgi module, and just import it back from urlparse (or urllib). Since cgi already imports urllib (which imports urlparse), this isn't adding any dependencies -- but it keeps the code in a single location. -jJ

13 years, 10 months

2.5 slower than 2.4 for some things?

by Greg Ewing

I've had a report from a user that Plex runs about half as fast in 2.5 as it did in 2.4. In particular, the NFA-to-DFA conversion phase, which does a lot of messing about with dicts representing mappings between sets of states. Does anyone in the Ministry for Making Python Blazingly fast happen to know of some change that might have pessimised things in this area? -- Greg

13 years, 10 months

TLSAbruptCloseError

by Todd Hopfinger

I am using TLS Lite and J2ME SecureConnection for the purposes of encrypting traffic to/from a Java Midlet client and a multithreaded Python server. However, I encounter a TLSAbruptCloseError. I have tried to determine the cause of the exception to no avail. I understand that it has to do with close_notify alerts. My abbreviated code follows. // Server def sslSockRecv(conn, num): data = '' while len(data) < num: data = conn.recv(num - len(data)) # TLSAbruptCloseError thrown here if len(data) == 0: raise NotEnoughBytes ('Too few bytes from client. Expected ' + str(num) + '; got ' + str(len(data)), num, len(data)) return data sslSockRecv() throws NotEnoughBytes exception to indicate that the client has closed the connection. The NotEnoughBytes exception handler subsequently closes the SSL connection and then the underlying socket. // Client import javax.microedition.io.SecureConnection; sc = (SecureConnection)Connector.open("ssl://host:port"); inStream = sc.openInputStream(); outStream = sc.openOutputStream(); // read/write some data using streams if (inStream != null) inStream.close(); if (outStream != null) outStream.close(); if (sc != null) sc.close(); When using the Java phone emulator, SSLDump indicates after the application data portions. 3 13 0.3227 (0.0479) C>SV3.0(22) Alert level warning value close_notify 3 0.3228 (0.0000) C>S TCP FIN 3 14 0.3233 (0.0005) S>CV3.0(22) Alert level warning value close_notify However, the server doesn't throw a TLSAbruptCloseError when using the emulator. Using the actual phone does cause a TLSAbruptCloseError on the server but SSLDump reports no errors, just. 4 1.6258 (0.7012) C>S TCP FIN 4 1.6266 (0.0008) S>C TCP FIN Any thoughts? Todd Hopfinger

13 years, 10 months

Re: [Python-Dev] Instance variable access and descriptors

by Phillip J. Eby

At 12:23 AM 6/10/2007 +0300, Eyal Lotem wrote: >A. It will break code that uses instance.__dict__['var'] directly, >when 'var' exists as a property with a __set__ in the class. I believe >this is not significant. >B. It will simplify getattr's semantics. Python should _always_ give >precedence to instance attributes over class ones, rather than have >very weird special-cases (such as a property with a __set__). Actually, these are features that are both used and desirable; I've been using them both since Python 2.2 (i.e., for many years now). I'm -1 on removing these features from any version of Python, even 3.0. >C. It will greatly speed up instance variable access, especially when >the class has a large mro. ...at the cost of slowing down access to properties and __slots__, by adding an *extra* dictionary lookup there. Note, by the way, that if you want to change attribute lookup semantics, you can always override __getattribute__ and make it work whatever way you like, without forcing everybody else to change *their* code.

13 years, 10 months

Question about dictobject.c:lookdict_string

by Eyal Lotem

My question is specifically regarding the transition back from lookdict_string (the initial value) to the general lookdict. Currently, when a string-only dict is trying to look up any non-string, it reverts back to a general lookdict. Wouldn't it be better (especially in the more important case of a string-key-only dict), to revert to the generic lookdict when a non-string is inserted to the dict, rather than when one is being searched? This seems to me as it would shift this (admittedly very slight) performance cost of a type ptr comparison from the read-access, to write-access on all dicts (which means insertions of new keys in non-string-only dicts may pay for another check, or that the lookdict funcptr will be replaced by two funcptrs so that a different insertion func on string-only dicts is used too [was tempted to say vtable here, but that would add another dereference to lookups]). It would also have the slight benefit of speeding up non-string lookups in string-only dicts. This does not seem like a significant issue, but as I know a lot of effort went into optimizing dicts, I was wondering if I am missing something here.

13 years, 10 months

Santa Fe Python Day report

by Facundo Batista

It was very succesful, around +300 people assisted, and there were a lot of interesting talks (two introductory talks, Turbogears, PyWeek, Zope 3, security, creating 3D games, Plone, automatic security testings, concurrency, and programming the OLPC). I want to thanks the PSF for the received support. Python is developing interestingly in Argentina, and this Python Days are both a prove of that, and a way to get more Python developers. Some links: Santa Fe Python Day: http://www.python-santafe.com.ar/ Python Argentina: http://www.python.com.ar/moin Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

13 years, 10 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev June 2007