Mailman 3 June 2009 - Python-Dev

Binary Operator for New-Style String Formatting
by Jerry Chen June 22, 2009

June 22, 2009

Hello all, For better or for worse, I have created a patch against the py3k trunk which introduces a binary operator '@' as an alternative syntax for the new string formatting system introduced by PEP 3101 ("Advanced String Formatting"). [1] For common cases, this syntax should be as simple and as elegant as its deprecated [2] predecessor ('%'), while also ensuring that more complex use cases do not suffer needlessly. I would just like to know whether this idea will float before submitting … [View More]the patch on Roundup and going through the formal PEP process. This is my first foray into the internals of the Python core, and with any luck, I did not overlook any BDFL proclamations banning all new binary operators for string formatting. :-) QUICK EXAMPLES >>> "{} {} {}" @ (1, 2, 3) '1 2 3' >>> "foo {qux} baz" @ {"qux": "bar"} 'foo bar baz' One of the main complaints of a binary operator in PEP 3101 was the inability to mix named and unnamed arguments: The current practice is to use either a dictionary or a tuple as the second argument, but as many people have commented ... this lacks flexibility. To address this, a convention of having the last element of a tuple as the named arguments dictionary is introduced. >>> "{} {qux} {}" @ (1, 3, {"qux": "bar"}) '1 bar 3' Lastly, to print the repr() of a dictionary as an unnamed argument, one would have to append an additional dictionary so there is no ambiguity: >>> "{}" @ {"foo": "bar"} Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: tuple index out of range >>> "{}" @ ({"foo": "bar"}, {}) "{'foo': 'bar'}" Admittedly, these workarounds are less than clean, but the understanding is the '@' syntax would indeed be an alternative, so one could easily fall back to the str.format() method or the format() function. IMPLEMENTATION Code-wise, the grammar was edited per PEP 306 [3], and a function was introduced in unicodeobject.c as PyUnicode_FormatPrime (in the mathematical sense of A and A' -- I didn't fully understand or want to intrude upon the *_FormatAdvanced namespace). The PyUnicode_FormatPrime function transforms the incoming arguments, i.e. the operands of the binary '@', and makes the appropriate do_string_format() call. Thus, I have reused as much code as possible. I have done my development with git by using two branches: 'master' and 'subversion', the latter of which can be used to run 'svn update' and merge back into master. This way my code changes and the official ones going into the Subversion repository can stay separate, meanwhile allowing 'svn diff' to produce an accurate patch at any given time. The code is available at: http://github.com/jcsalterego/py3k-atsign/ The SVN patch [4] or related commit [5] are good starting points. References: [1] http://www.python.org/dev/peps/pep-3101 [2] http://docs.python.org/3.0/whatsnew/3.0.html [3] http://www.python.org/dev/peps/pep-0306/ [4] http://github.com/jcsalterego/py3k-atsign/blob/master/py3k-atsign.diff [5] http://github.com/jcsalterego/py3k-atsign/commit/5c8bdf72d9252cea78af2b7809… Thanks, -- Jerry Chen [View Less]

8 8

run time error anlysis of python source code
by omega_force2003＠yahoo.com June 22, 2009

June 22, 2009

To All, It appears that one possibility of investigation into the development of a safety-critical variant of the python language would be to conduct run time error analysis of the source code that is responsible for producing the python language. Therefore, I will now conduct these run time error analysis of the python source code as if the python environment itself was to be utilized as a FADEC controller within an aircraft engine. I have already conducted … [View More]

6 5

xmlrpc improvements
by Kristján Valur Jónsson June 21, 2009

June 21, 2009

Hello there. I've been trying to, in the last weeks, to pass on to the trunk the improvements I've made to XMLRPC. I've created several issues in order to do these changes incrementally but have got no comments. Perhaps it is best to show the whole thing in context, then. I´ve gathered all the changes in a single patch, here: http://codereview.appspot.com/73041/show I've also created a corresponding issue in the issue tracker: http://bugs.python.org/issue6267 I'd really like to get this … [View More]

4 6

io.BufferedReader.peek() Behaviour in python3.1
by Frederick Reeve June 21, 2009

June 21, 2009

Greetings, As I'm sure you all know there are currently two implementations of the io module one in python and one much faster implementation in C. As I recall the python version was used in python3 and the C version is now used by default in python3.1x. The behavior of the two is different in some ways especially regarding io.BufferedReader.peek(). I wrote an email to the authors of the new C code last Friday. I also sent a copy of it to the python list for comments. I was directed by … [View More]Antoine Pitrou that I should possibly bring up what I had asked there here or as a bug report. I elected to write here because I am not sure it constitutes a bug. In my former email I stated I was willing to submit patches if the old behavior was desired back and the code author was fine with the changes but didn't want to implement them. Antoine said this, "If people need more sophisticated semantics, I'm open to changing peek() to accommodate it." Antoine: If I do wrong quoting you are free to chastise me. So my basic question is: The behavior of io.BufferedReader.peek() has changed; is that change something that should: stay as is, revert or be different entirely? Here are the two behaviors: The python version of io.BufferedReader.peek() behaves as: If the buffer holds less than requested (upto buffersize) read from the raw stream the difference or up to EOF into the buffer. Return requested number of bytes from the start of the buffer. This may advance the raw stream but not the local stream. This version can guarantee a peek of one chunk (4096 bytes here). The C version behaves as: If the buffer holds 0 bytes fill from the raw stream or up to EOF. Return what is in the buffer. This may advance the raw stream but not the local stream. This version cannot guarantee a peek of over 1 byte if random length reads are being used at all and not tracked. Neither case limits what is possible, though, in my opinion, one makes it easier to accomplish certain things and is more efficient in those cases. Take the following two parser examples: s = io.BufferedReader wrapped stream with no negative seek in most cases. f = output file handler or such. python version work flow: are = re.compile(b'(\r\n|\r|\n)') while True: d = s.peek(4096) # chunk size or so. found = are.search(d) if found: w = d[:found.start()] s.seek(f.write(w)) p = s.peek(74) if p.startswith(multipart_boundary): s.seek(len(multipart_boundary)) # other code containing more possible splits # across boundaries continue w = d[found.start():found.end()] s.seek(f.write(w)) continue f.write(d) #more code continue C version work flow: old = b'' are = re.compile(b'(\r\n|\r|\n)') while True: d = old if old != b'' else s.read1(4096) found = are.search(d) if found: w = d[:found.start()] f.write(w) w = d[found.start():] p = w if len(w) >= 74 else w + s.read(73) if p.startswith(multipart_boundary): # Other code containing more possible splits # across boundaries and joins to p. old = ??? continue f.write(d[found.start():found.end()]) old = dd[found.end():] + p continue old = b'' f.write(d) #more code continue These two examples are not real code but get the point across and are based off code I put into a multipart parser. The former written for python3. I later tried running that parser on 3.1 after the new io layer and found it broken. Then rewrote it to the new interface. That rewrite is represented in the latter some what. This is only one example. Others may vary, of course. Peek seems to me to have little use outside of parsers. Thus I used parsers as an example. My opinion is that it would be better to have a peek function similar to the the python implementation in C like as follows: peek(n): If n is less than 0, None, or not set; return buffer contents with out advancing stream position. If the buffer is empty read a full chunk and return the buffer. Otherwise return exactly n bytes up to _chunk size_(not contents) with out advancing the stream position. If the buffer contents is less than n, buffer an additional chunk from the "raw" stream before hand. If EOF is encountered during any raw read then return as much as we can up to n. (maybe I should write that in code form??) This allows us to obtain the behavior of the current C io implementation easily and would give us the old python implementation's behavior when n is given. The basis for this is: 1. Code reduction and Simplicity Looking at the examples, the code reduction should be obvious. The logic needed to maintain a bytestring of the variously required lengths, so that it may be checked, would not be necessary. The need to hold a bytestring to the next iteration would be done away with as well. Other pieces of data handling would also be simpler. 2. Speed It would require less handling in the "slower" interpreter if we would use the buffer in the buffered reader. Also, all that logic mentioned in 1 is moved to the faster C code or done away with. There is very little necessity for peek outside of parsers, so speed in read-through and random reads would not have to be affected. I have other reasons and arguments, but I want to know what every one else thinks. This will most likely show me what I have missed or am not seeing, if anything. Please I have babbled enough. Thanks so much for the consideration. Frederick Reeve [View Less]

10 33

Summary of Python tracker Issues
by Python tracker June 19, 2009

June 19, 2009

ACTIVITY SUMMARY (06/12/09 - 06/19/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2239 open (+26) / 15895 closed (+14) / 18134 total (+40) Open issues with patches: 886 Average duration of open issues: 653 days. Median duration of open issues: 404 days. Open Issues Breakdown open 2210 (+25) pending 29 ( +1) Issues Created Or Reopened (40) … [View More]

1 0

Popen asynchronous input for Windows
by Eric Pruitt June 18, 2009

June 18, 2009

Hello, I am looking for alternatives to Josiah Carlson's asynchronous I/O patch for subprocess.Popen. While his patch seems to work well, it relies on pywin32 which is not part of the standard Python library. If I cannot find an alternative, I will be using cTypes with the parts of Mark Hammond's code that I need, license permitting. Any suggestions are greatly appreciated. Eric

2 2

Avoiding file descriptors leakage in subprocess.Popen()
by Facundo Batista June 18, 2009

June 18, 2009

In a long lived process at work, we started leaking file descriptors. The problem were that subprocess.Popen._execute_child() method creates two files descriptors through a call to os.pipe(), and after some work it closes them. But an os.read() on the descriptor was interrupted (EINTR), so an exception was raised, and the descriptors were not closed... leakage! This problem is easy to fix (I have a patch that fixes it, all tests pass ok, see http://bugs.python.org/issue6274). So, why this … [View More]

7 10

ElementTree issues in python 3.1 rc2
by Neil Muller June 18, 2009

June 18, 2009

The following issues in ElementTree still exist in the latest release candidate & all have small patches to fix the issue. Any chance of getting these in before the final release? http://bugs.python.org/issue6231 http://bugs.python.org/issue6233 http://bugs.python.org/issue2746 -- Neil Muller drnlmuller(a)gmail.com

2 1

SSL Certificate Validation
by Devin Cook June 17, 2009

June 17, 2009

Hi all, I have a few questions about validating SSL certificates. From what I gather, this validation occurs in the OpenSSL code called from _ssl.c. Is this correct? Also, I have looked through the docs and code, but haven't been able to figure out exactly what is included in certificate "validation". Is it just validating the chain? Does it check the NotBefore and NotAfter dates? Does it check that the host the socket is connected to is the same as what's given in the CN field in the … [View More]

4 11

Python script language or not
by abhishek goswami June 17, 2009

June 17, 2009

Hi, I have very basic question about Python that do we consider pyhton as script language. I searched in google but it becomes more confusion for me. After some analysis I came to know that Python support oops . Can anyone clarify me. Please let me know also it is right forum or not. I was looking for Python forum but i did not find any forum. Do we have forum like linux forum so it will be very easy for me to ask this question in forum and i will not ask repeated question becuase i am sure … [View More]

3 2