What was the outcome of the discussion of Georg Brandl's reworked
documentation ("The docs, reloaded")? Was any decision made on
whether to go with reST, or on what changes need to made before that's
possible? Did Fred Drake say what he thought?
Georg, do you want access to python.org to host a version of the docs
there?
--amk
Hi,
I am a student participant of Google Summer of Code 2007 and I am
working on the cleanup task of urllib2, with Skip as my mentor.
I would like to request for a commit access to the Python Sandbox for
implementing the changes as part of the project. I have attached by
SSH Public keys.
preferred name : senthil.kumaran
I am following up and adding comments to the urllib related bugs at
sf.net page. I would also like to request addition of my sourceforge
id : orsenthil to the python project, so I can close the defects
raised against urllib modules.
Summer of Code Project:
http://code.google.com/soc/psf/appinfo.html?csaid=E73A6612F80229B6
The project actually commenced on May 28th itself. But, there was a
delay from my side to get started. Ivan Sutherland's essay on
Technology and Courage [1] did some good thing to me. :-)
Thanks,
Senthil
[1] http://research.sun.com/techrep/Perspectives/smli_ps-1.pdf#search=%22suther…
--
O.R.Senthil Kumaran
http://phoe6.livejournal.com
Hi all,
This mail is a request for comments on changes to urlparse module. We understand
that urlparse returns the 'complete query' value as the query
component and does not
provide the facilities to separate the query components. User will have to use
the cgi module (cgi.parse_qs) to get the query parsed.
There has been a discussion in the past, on having a method of parse query
string available from urlparse module itself. [1]
To implement the query parse feature in urlparse module, we can:
a) import cgi and call cgi module's query_ps.
This approach will have problems as it
i) imports cgi for urlparse module.
ii) cgi module in turn imports urllib and urlparse.
b) Implement a stand alone query parsing facility in urlparse *AS IN*
cgi module.
Below method implements the urlparse_qs(url, keep_blank_values,strict_parsing)
that will help in parsing the query component of the url. It behaves same as the
cgi.parse_qs.
Please let me know your comments on the below code.
----------------------------------------------------------------------
def unquote(s):
"""unquote('abc%20def') -> 'abc def'."""
res = s.split('%')
for i in xrange(1, len(res)):
item = res[i]
try:
res[i] = _hextochr[item[:2]] + item[2:]
except KeyError:
res[i] = '%' + item
except UnicodeDecodeError:
res[i] = unichr(int(item[:2], 16)) + item[2:]
return "".join(res)
def urlparse_qs(url, keep_blank_values=0, strict_parsing=0):
"""Parse a URL query string and return the components as a dictionary.
Based on the cgi.parse_qs method.This is a utility function provided
with urlparse so that users need not use cgi module for
parsing the url query string.
Arguments:
url: URL with query string to be parsed
keep_blank_values: flag indicating whether blank values in
URL encoded queries should be treated as blank strings.
A true value indicates that blanks should be retained as
blank strings. The default false value indicates that
blank values are to be ignored and treated as if they were
not included.
strict_parsing: flag indicating what to do with parsing errors.
If false (the default), errors are silently ignored.
If true, errors raise a ValueError exception.
"""
scheme, netloc, url, params, querystring, fragment = urlparse(url)
pairs = [s2 for s1 in querystring.split('&') for s2 in s1.split(';')]
query = []
for name_value in pairs:
if not name_value and not strict_parsing:
continue
nv = name_value.split('=', 1)
if len(nv) != 2:
if strict_parsing:
raise ValueError, "bad query field: %r" % (name_value,)
# Handle case of a control-name with no equal sign
if keep_blank_values:
nv.append('')
else:
continue
if len(nv[1]) or keep_blank_values:
name = unquote(nv[0].replace('+', ' '))
value = unquote(nv[1].replace('+', ' '))
query.append((name, value))
dict = {}
for name, value in query:
if name in dict:
dict[name].append(value)
else:
dict[name] = [value]
return dict
----------------------------------------------------------------------
Testing:
$ python
Python 2.6a0 (trunk, Jun 10 2007, 12:04:03)
[GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urlparse
>>> dir(urlparse)
['BaseResult', 'MAX_CACHE_SIZE', 'ParseResult', 'SplitResult', '__all__',
'__builtins__', '__doc__', '__file__', '__name__', '_parse_cache',
'_splitnetloc', '_splitparams', 'clear_cache', 'non_hierarchical',
'scheme_chars', 'test', 'test_input', 'unquote', 'urldefrag', 'urljoin',
'urlparse', 'urlparse_qs', 'urlsplit', 'urlunparse', 'urlunsplit',
'uses_fragment', 'uses_netloc', 'uses_params', 'uses_query', 'uses_relative']
>>> URL =
>>> 'http://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=utf-8&q=south+africa+tra…'
>>> print urlparse.urlparse_qs(URL)
{'q': ['south africa travel cape town'], 'oe': ['utf-8'], 'ie': ['UTF-8'],
'hl': ['en']}
>>> print urlparse.urlparse_qs(URL,keep_blank_values=1)
{'q': ['south africa travel cape town'], 'ie': ['UTF-8'], 'oe': ['utf-8'],
'lr': [''], 'hl': ['en']}
>>>
Thanks,
Senthil
[1] http://mail.python.org/pipermail/tutor/2002-August/016823.html
--
O.R.Senthil Kumaran
http://phoe6.livejournal.com
I'm seeing conflicting opinions on whether to put
sys.setdefaultencoding('utf-8') in sitecustomize.py or not ([1] vs.
[2]) and frankly I'm confused.
The csv module says it's not unicode safe but the 2.5 docs [3] have a
workaround for this. While the workaround says nothing about
sys.setdefaultencoding() it simply does not work with the default
encoding, "ascii." Is this _the_ problem with the csv module? Should
I give up and use XML? Below is code that works vs. code that
doesn't. Am I interpretting the workaround from the docs wrong? If
so, can someone please give me a hint ;) I should also point out that
I've tried this with the StringIO queued approach (from the
workaround) but that doesn't solve anything.
1) with the default encoding :
kumar$ python2.5
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
>>> import sys, csv, codecs
>>> f = codecs.open('unicsv.csv','wb','utf-8')
>>> w = csv.writer(f)
>>> w.writerow([u'lang', u'espa\xa4ol'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa4' in
position 4: ordinal not in range(128)
>>>
2) with custom encoding :
kumar$ python2.5 -S
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
>>> import sys, csv, codecs
>>> sys.setdefaultencoding('utf-8')
>>> f = codecs.open('unicsv.csv','wb','utf-8')
>>> w = csv.writer(f)
>>> w.writerow([u'lang', u'espa\xa4ol'])
>>> f.close()
thanks, Kumar
[1] http://mail.python.org/pipermail/python-dev/2007-June/073593.html
[2] http://diveintopython.org/xml_processing/unicode.html
[3] http://docs.python.org/lib/csv-examples.html#csv-examples
> a) import cgi and call cgi module's query_ps. [circular imports]
or
> b) Implement a stand alone query parsing facility in urlparse *AS IN*
> cgi module.
Assuming (b), please remove the (code for the) parsing from the cgi
module, and just import it back from urlparse (or urllib). Since cgi
already imports urllib (which imports urlparse), this isn't adding any
dependencies -- but it keeps the code in a single location.
-jJ
I've had a report from a user that Plex runs about half
as fast in 2.5 as it did in 2.4. In particular, the
NFA-to-DFA conversion phase, which does a lot of
messing about with dicts representing mappings between
sets of states.
Does anyone in the Ministry for Making Python Blazingly
fast happen to know of some change that might have
pessimised things in this area?
--
Greg
I am using TLS Lite and J2ME SecureConnection for the purposes of encrypting
traffic to/from a Java Midlet client and a multithreaded Python server.
However, I encounter a TLSAbruptCloseError. I have tried to determine the
cause of the exception to no avail. I understand that it has to do with
close_notify alerts. My abbreviated code follows.
// Server
def sslSockRecv(conn, num):
data = ''
while len(data) < num:
data = conn.recv(num - len(data)) #
TLSAbruptCloseError thrown here
if len(data) == 0:
raise NotEnoughBytes ('Too
few bytes from client. Expected ' + str(num) + '; got ' + str(len(data)),
num, len(data))
return data
sslSockRecv() throws NotEnoughBytes exception to indicate that the client
has closed the connection. The NotEnoughBytes exception handler subsequently
closes the SSL connection and then the underlying socket.
// Client
import javax.microedition.io.SecureConnection;
sc = (SecureConnection)Connector.open("ssl://host:port");
inStream = sc.openInputStream();
outStream = sc.openOutputStream();
// read/write some data using streams
if (inStream != null)
inStream.close();
if (outStream != null)
outStream.close();
if (sc != null)
sc.close();
When using the Java phone emulator, SSLDump indicates after the application
data portions.
3 13 0.3227 (0.0479) C>SV3.0(22) Alert
level warning
value close_notify
3 0.3228 (0.0000) C>S TCP FIN
3 14 0.3233 (0.0005) S>CV3.0(22) Alert
level warning
value close_notify
However, the server doesn't throw a TLSAbruptCloseError when using the
emulator. Using the actual phone does cause a TLSAbruptCloseError on the
server but SSLDump reports no errors, just.
4 1.6258 (0.7012) C>S TCP FIN
4 1.6266 (0.0008) S>C TCP FIN
Any thoughts?
Todd Hopfinger
At 12:23 AM 6/10/2007 +0300, Eyal Lotem wrote:
>A. It will break code that uses instance.__dict__['var'] directly,
>when 'var' exists as a property with a __set__ in the class. I believe
>this is not significant.
>B. It will simplify getattr's semantics. Python should _always_ give
>precedence to instance attributes over class ones, rather than have
>very weird special-cases (such as a property with a __set__).
Actually, these are features that are both used and desirable; I've
been using them both since Python 2.2 (i.e., for many years
now). I'm -1 on removing these features from any version of Python, even 3.0.
>C. It will greatly speed up instance variable access, especially when
>the class has a large mro.
...at the cost of slowing down access to properties and __slots__, by
adding an *extra* dictionary lookup there.
Note, by the way, that if you want to change attribute lookup
semantics, you can always override __getattribute__ and make it work
whatever way you like, without forcing everybody else to change *their* code.
My question is specifically regarding the transition back from
lookdict_string (the initial value) to the general lookdict.
Currently, when a string-only dict is trying to look up any
non-string, it reverts back to a general lookdict.
Wouldn't it be better (especially in the more important case of a
string-key-only dict), to revert to the generic lookdict when a
non-string is inserted to the dict, rather than when one is being
searched?
This seems to me as it would shift this (admittedly very slight)
performance cost of a type ptr comparison from the read-access, to
write-access on all dicts (which means insertions of new keys in
non-string-only dicts may pay for another check, or that the lookdict
funcptr will be replaced by two funcptrs so that a different insertion
func on string-only dicts is used too [was tempted to say vtable
here, but that would add another dereference to lookups]).
It would also have the slight benefit of speeding up non-string
lookups in string-only dicts.
This does not seem like a significant issue, but as I know a lot of
effort went into optimizing dicts, I was wondering if I am missing
something here.
It was very succesful, around +300 people assisted, and there were a lot of interesting talks (two introductory talks, Turbogears, PyWeek, Zope 3, security, creating 3D games, Plone, automatic security testings, concurrency, and programming the OLPC).
I want to thanks the PSF for the received support. Python is developing interestingly in Argentina, and this Python Days are both a prove of that, and a way to get more Python developers.
Some links:
Santa Fe Python Day: http://www.python-santafe.com.ar/
Python Argentina: http://www.python.com.ar/moin
Regards,
--
. Facundo
.
Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/