Mailman 3 April 2008 - Python-Dev

python hangs when parsing a bad-formed email
by Alberto Casado Martín April 23, 2008

April 23, 2008

Hi all, First of all, sorry if this isn't the list where I have to post this. And sorry for my english. As the subject says, I'm having problems with the attached email, when I try to get a email object reading the attached file, the python process gets hang and gets all cpu. I have debuged my code to find where it happens, and I found that is _parsegen method of the FeedParser class. I know that the email format is wrong but I don't know why python hangs. following paste the code showing … [View More]where hangs. def _parsegen(self): # Create a new message and start by parsing headers. self._new_message() headers = [] # Collect the headers, searching for a line that doesn't match the RFC # 2822 header or continuation pattern (including an empty line). for line in self._input: if line is NeedMoreData: yield NeedMoreData continue if not headerRE.match(line): # If we saw the RFC defined header/body separator # (i.e. newline), just throw it away. Otherwise the line is # part of the body so push it back. if not NLCRE.match(line): self._input.unreadline(line) break headers.append(line) # Done with the headers, so parse them and figure out what we're # supposed to see in the body of the message. self._parse_headers(headers) # Headers-only parsing is a backwards compatibility hack, which was # necessary in the older parser, which could throw errors. All # remaining lines in the input are thrown into the message body. if self._headersonly: lines = [] while True: line = self._input.readline() if line is NeedMoreData: yield NeedMoreData continue if line == '': break lines.append(line) self._cur.set_payload(EMPTYSTRING.join(lines)) return if self._cur.get_content_type() == 'message/delivery-status': !!!!!! AT THIS POINT HANGS, AND STRAT TO GET ALL CPU FOR THE PROCESS # message/delivery-status contains blocks of headers separated by # a blank line. We'll represent each header block as a separate # nested message object, but the processing is a bit different # than standard message/* types because there is no body for the # nested messages. A blank line separates the subparts. ... ... ... I have workaround the problem adding this line in _parse_headers method def _parse_headers(self, lines): # Passed a list of lines that make up the headers for the current msg lastheader = '' lastvalue = [] for lineno, line in enumerate(lines): # Check for continuation if line[0] in ' \t': if not lastheader: # The first line of the headers was a continuation. This # is illegal, so let's note the defect, store the illegal # line, and ignore it for purposes of headers. defect = errors.FirstHeaderLineIsContinuationDefect(line) self._cur.defects.append(defect) continue if line.strip()!='': !!!!!!! IF THE CONTINUATION LINE IS NOT EMPTY ADD THE LINE TO THE HEADER. lastvalue.append(line) continue if lastheader: ... ... ... I don't know why it hangs and I'm not sure why with this line works...... I have tried to parse this email in python 2.3.3 SunOs, python 2.3.3 gcc python 2.5.1 SunOs,gcc, Windows Xp, and linux SUSE 10. And I have alway the same result. bash-3.00$ python Python 2.5.1 (r251:54863, Feb 28 2008, 07:48:25) [GCC 3.4.6] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import email >>> fp = open('raro.txt') >>> mail = email.message_from_file(fp) never return............ I don't know if someone can tell me what is happening.... Best Regards. Alberto Casado. [View Less]

3 2

Known doctest bug with unicode?
by Jeroen Ruigrok van der Werven April 23, 2008

April 23, 2008

Is it a known doctest bug that when you have a dict with Unicode key values that doctest dies with a KeyError? When I excute my code from the regular python interpreter it works as expected. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai イェルーンラウフロックヴァンデルウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ Whenever you meet difficult situations dash forward bravely and joyfully...

4 5

Re: [Python-Dev] Encoding detection in the standard library?
by M.-A. Lemburg April 23, 2008

April 23, 2008

[CCing python-dev again] On 2008-04-22 12:38, Greg Wilson wrote: >>> I don't think that should be part of the standard library. People >>> will mistake what it tells them for certain. >>> [etc] > > These are all good arguments, but the fact remains that we can't control > our inputs (e.g., we're archiving mail messages sent to lists managed by > DrProject), and some of those inputs *don't* tell us how they're encoded. > Under those circumstances, … [View More]

1 0

3k checkin mails to python-checkins
by Georg Brandl April 23, 2008

April 23, 2008

Since a few days, checkin notifications for the 3k branch seem to be sent to both the python-checkins and the python-3000-checkins lists. Was that a deliberate decision or has some bug crept into the SVN hook? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. … [View More]

2 2

configure error: "rm: conftest.dSYM: is a directory"
by skip＠pobox.com April 22, 2008

April 22, 2008

I just noticed this error message during configure: checking whether gcc accepts -Olimit 1500... no checking whether gcc supports ParseTuple __format__... no checking whether pthreads are available without options... yes checking whether g++ also accepts flags for thread support... no checking for ANSI C header files... rm: conftest.dSYM: is a directory rm: conftest.dSYM: is a directory yes checking for sys/types.h... yes checking for sys/stat.h... yes … [View More]

4 4

Re: [Python-Dev] r62342 - python/branches/py3k/Objects/bytesobject.c
by Christian Heimes April 22, 2008

April 22, 2008

alexandre.vassalotti schrieb: > Author: alexandre.vassalotti > Date: Mon Apr 14 22:51:05 2008 > New Revision: 62342 > > Log: > Improved bytes_extend() to avoid making a full copy of the temporary > buffer. This also makes the code slightly cleaner. Changes to Objects/bytesobject.c should be applied to the trunk and merged into the py3k branch via svnmerge.py. We need to agree on a policy how we are going to sync the trunk and py3k for new code like bytesobject.c and io.… [View More]

5 8

very bad network performance
by Ralf Schmitt April 22, 2008

April 22, 2008

Hi all, I'm using mercurial with the release25-maint branch. I noticed that checking out a local repository now takes more than 5 minutes (it should be around 30s). I've tracked it down to this change: http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd this is svn revision 61009. Here is the diff inline: --- a/Lib/socket.py Fri Mar 23 14:27:29 2007 +0100 +++ b/Lib/socket.py Sat Feb 23 20:30:59 2008 +0100 @@ -305,7 +305,7 @@ self._rbuf = "" while True: … [View More]

7 17

Encoding detection in the standard library?
by Jim Jewett April 22, 2008

April 22, 2008

David Wolever wrote: > IMO, encoding estimation is something that > many web programs will have to deal with, > so it might as well be built in; I would prefer > the option to run `text=input.encode('guess')` > (or something similar) than relying on an external > dependency or worse yet using a hand-rolled > algorithm The (still draft) html5 spec is trying to get error-correction standardized, so it includes all sort of "if this fails, do X". Encoding detection will be … [View More]

1 0

thoughts on having EOFError inherit from EnvironmentError?
by Gregory P. Smith April 22, 2008

April 22, 2008

http://bugs.python.org/issue1481036 Basically as things are now EOFError is on its own but often wants to be handled the same as other I/O errors that EnvironmentError currently covers. Many uses of EOFError in our code base do not provide it any arguments so it doesn't really fit the (errno, message [, filename]) tuple style that EnvironmentError promises. But we could fudge that with a reasonable default (whats reasonable?) if we rerooted this under EnvironmentError. Alternatively the bug … [View More]

5 14

unscriptable?
by Jim Jewett April 22, 2008

April 22, 2008

> I dispute this. Indices aren't necessarily numeric > (think of an A-Z file), Python has recently added an __index__ slot which means "as an integer, and I really am an integer, I'm not just rounding like int(3.4) would do" So in the context of python, an index is numeric, whereas "subscript" has already been used for hashtables with arbitrary keys. -jJ

3 2