Mailman 3 June 2011 - Python-Dev

fatal error callback issue
by Tom Whittock 08 Jun '11

08 Jun '11

I'm writing in regards to http://bugs.python.org/issue1195571 I'm embedding Python in my application and ran into a need for this functionality. I wrote a similar patch myself, and was about to submit it. When I searched for similar issues I found that this one has been available since 2005. I'd really like to help get this patch approved and integrated into the python sources. I'm sure many other python embedders have run into this in the past - it's a legitimate concern for any process which is not 100% hosted in the Python world. Thanks. Tom.

2 1

Re: [Python-Dev] [Python-checkins] cpython (2.7): Merge
by Victor Stinner 08 Jun '11

08 Jun '11

Le jeudi 09 juin 2011 à 02:30 +0200, brian.curtin a écrit : > http://hg.python.org/cpython/rev/f1509fc75435 > changeset: 70715:f1509fc75435 > branch: 2.7 > parent: 70661:6e7a98cfcfab > user: Brian Curtin <brian(a)python.org> > date: Wed Jun 08 19:29:53 2011 -0500 > summary: > Merge FYI this commit has only one parent, it's not a merge. It should have a changelog describing the patch. ... whereas the following commit has two parents, it's a merge: > http://hg.python.org/cpython/rev/567f30527913 > changeset: 70714:567f30527913 > parent: 70712:964d0d65a2a9 > parent: 70713:88e318166eaf > user: Brian Curtin <brian(a)python.org> > date: Wed Jun 08 18:43:57 2011 -0500 > summary: > Fix #11583. Changed os.path.isdir to use GetFileAttributes (...) You can use "merge", "Merge 3.2", or other similir changelog. I prefer to use "(Merge 3.2)Fix #11583. Changed os.path._isdir ..." : copy/paste the changelog of the original commit with "(Merge 3.2) " prefix. Victor

1 0

Byte filenames in the posix module on Windows
by Victor Stinner 07 Jun '11

07 Jun '11

Hi, Last november, we "decided" (right?) to deprecate bytes filenames in the posix module on Windows in Python 3.2 and drop the support in 3.3: see "Removal of Win32 ANSI API" thread on python-dev. Python 3.2 has been released, so we should shift the versions numbers. I would like to take care of this. I propose three steps: 1) Remove the bytes implementation of each function (when the code is not shared with other OSes), decode bytes from MBCS and reuse the Unicode code. 2) Deprecate the bytes code in Python 3.3 3) Drop bytes support in Python 3.4 (I'm only talking about the posix module on Windows) The first step should not change anything for the user, but it will remove a lot of duplicated code. I expect something like removing the half of the code specific to Windows in the posix module. If we decide to keep bytes filenames on Windows, we can stop before the second step. -- One important point is the choice of the error handler: I would like to mimic the ANSI API and so I will use MultiByteToWideChar() with flags=0 (e.g. MBCS codec with ignore error handler, but see also the issue #12281 !). The MBCS codec uses the ANSI code page which can be a multibyte encoding, like ShiftJIS (cp932 with a japanese locale). os.fsdecode(), PyUnicode_DecodeFSDefault() and PyUnicode_FSDecoder() use the strict error handler to decode filenames on Windows. We may also use strict in the posix module. I'm +0 for this because it warns the developer (and user?) that he/she is doing something really bad. -- I would like to simplify posixmodule.c because I saw that it is difficult to patch it to fix bugs (you have to patch two functions, or more, for each fix). A recent example is the os.stat() symlink issue on Windows: http://bugs.python.org/issue12084 Since Windows 2000, filenames are stored as Unicode internally (e.g. VFAT and NTFS use UTF-16), the ANSI API was kept for backward compatibility (for lazy developers!). If you use bytes for filenames on Windows, you may get encode errors because the ANSI code page is a small subset of Unicode. For your information, I have a last -huge- pending patch to only use Unicode in the import machinery, issue #11619 ;-) Using this patch, you can use characters not encodable to the ANSI code page in your module name/path, yeah! But I am not completly convinced that we need this patch... Victor

1 0

AIX 5.3 - Build of Python 2.7.1
by Anurag Chourasia 07 Jun '11

07 Jun '11

Hi All, Python2.7.1 build on AIX 5.3 ML 12 is failing at the make step with the error indicated below de -I./Include -I /opt/freeware/include -I /opt/freeware/include/readline -I /opt/freeware/include/ncurses -DPy_BUILD_CORE -o Parser/tokenizer_pgen.o Parser/tokenizer_pgen.c gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I /opt/freeware/include -I /opt/freeware/include/readline -I /opt/freeware/include/ncurses -DPy_BUILD_CORE -o Parser/printgrammar.o Parser/printgrammar.c gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I /opt/freeware/include -I /opt/freeware/include/readline -I /opt/freeware/include/ncurses -DPy_BUILD_CORE -o Parser/pgenmain.o Parser/pgenmain.c gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Objects/obmalloc.o Python/mysnprintf.o Python/pyctype.o Parser/tokenizer_pgen.o Parser/printgrammar.o Parser/pgenmain.o -ldl -o Parser/pgen ld: 0711-593 SEVERE ERROR: Symbol C_BSTAT (entry 559) in object Parser/grammar1.o: The symbol refers to a csect with symbol number 0, which was not found. The new symbol cannot be associated with a csect and is being ignored. ld: 0711-593 SEVERE ERROR: Symbol C_BSTAT (entry 579) in object Parser/firstsets.o: The symbol refers to a csect with symbol number 0, which was not found. The new symbol cannot be associated with a csect and is being ignored. collect2: ld returned 12 exit status make: 1254-004 The error code from the last command is 1. Is there anything I could try to get past this? Here is the configuration and make step that i utilized ./configure –enable-shared --disable-ipv6 --with-gcc=gcc CPPFLAGS="-I /opt/freeware/include -I /opt/freeware/include/readline -I /opt/freeware/include/ncurses" make Regards, Anurag

2 1

Re: [Python-Dev] [Python-checkins] cpython: Issue #12040: Expose a new attribute `sentinel` on instances of
by Nick Coghlan 07 Jun '11

07 Jun '11

On Tue, Jun 7, 2011 at 6:22 PM, Antoine Pitrou <solipsis(a)pitrou.net> wrote: > On Tue, 7 Jun 2011 12:47:37 +1000 > Nick Coghlan <ncoghlan(a)gmail.com> wrote: >> >> We should probably follow this model for threading.Thread.ident as >> well (i.e. throwing an exception rather than returning None if the >> thread hasn't been started yet). > > Well, Process.ident returns None as well ;) Hmm, I guess in that case my preference for keeping the Thread and Process APIs reasonably consistent just edges out my dislike of returning None when a "you shouldn't do that" exception would be more appropriate. >> Also, for runtime state errors, we tend to use RuntimeError rather >> than ValueError (e.g. see the errors thrown by >> contextlib._GeneratorContextManager) > > Well, it depends. For example, closed files raise a ValueError when you > try to do operations on them. I tend to think of RuntimeError as > something that happens without any clear responsibility from the user, > although that's debatable (recursion errors can occur because of a > programming bug, but also simply because a structure is nested much too > deeply). Yeah, I only thought of the closed file counterexample after I had already posted. I guess I just have a slight personal preference for RuntimeError to flag state problems as they're less likely to be caught up in an overly broad try statement in user code (as catching RuntimeError seems to be less common than catching ValueError). > Of course, current coding practice in other multiprocessing.Process > methods and properties doesn't help, since it uses asserts to guard > against misuse! I have a vague recollection of an existing tracker issue complaining about that... or am I just remembering the one that addressed similar problems in threading? Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

1 0

Re: [Python-Dev] [Python-checkins] cpython: Issue #12040: Expose a new attribute `sentinel` on instances of
by Nick Coghlan 06 Jun '11

06 Jun '11

On Tue, Jun 7, 2011 at 3:36 AM, antoine.pitrou <python-checkins(a)python.org> wrote: > + @property > + def sentinel(self): > + ''' > + Return a file descriptor (Unix) or handle (Windows) suitable for > + waiting for process termination. > + ''' > + try: > + return self._sentinel > + except AttributeError: > + raise ValueError("process not started") > + We should probably follow this model for threading.Thread.ident as well (i.e. throwing an exception rather than returning None if the thread hasn't been started yet). Also, for runtime state errors, we tend to use RuntimeError rather than ValueError (e.g. see the errors thrown by contextlib._GeneratorContextManager) Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

1 0

Help requested for Python ISO Standard
by kevin coyne 06 Jun '11

06 Jun '11

To whom it may concern: I am working on an ISO Annex of Vulnerabilities for the Python language and am asking for help getting a list of language features that exhibit: unspecified behavior, undefined behavior, or implementation defined behavior. I am also searching for a list of deprecated features. Guido (copied) had suggested I contact you folks for help as we are coming up on an ISO deadline and I have not been successful finding this information. Thanks very much in advance for your help! Regards, Kevin Coyne Cell: 703.901.6774

3 3

FWD: gpg keys have problems
by Aahz 06 Jun '11

06 Jun '11

I'm not currently reading python-dev, dunno if this has come up before: ----- Forwarded message from "Michael J. Dinneen" <mjd(a)cs.auckland.ac.nz> ----- > Date: Sun, 05 Jun 2011 19:33:04 +1200 > From: "Michael J. Dinneen" <mjd(a)cs.auckland.ac.nz> > To: webmaster(a)python.org > Subject: gpg keys have problems > Organization: University of Auckland > > >From your python download page you need to update the public keys to not use the faulty MD5 digest algorithm. > (see the link listed below) > > $ gpg --import pubkeys.txt > gpg: key 6A45C816: public key "Anthony Baxter <anthony(a)interlink.com.au>" imported > gpg: WARNING: digest algorithm MD5 is deprecated > gpg: please see http://www.gnupg.org/faq/weak-digest-algos.html for more information > gpg: key ED9D77D5: public key "Barry A. Warsaw <barry(a)warsaw.us>" imported > gpg: Total number processed: 2 > gpg: imported: 2 (RSA: 1) > gpg: no ultimately trusted keys found > ----- End forwarded message ----- -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ Looking back over the years, after I learned Python I realized that I never really had enjoyed programming before.

2 1

Buildbots and regrtest timeout
by Victor Stinner 06 Jun '11

06 Jun '11

Hi, Stephan Krah asked me to change how the default timeout is defined for regrtest (issue #12250): "The implicit timeout in regrtest.py makes it harder to write automated test scripts for 3rd party modules. First, you have to remember to set --timeout=0 for long running tests. Then, you have to remember not to use the --timeout option when compiling --without-threads. I'd much prefer that there's no timeout unless explicitly specified. For the buildbots, I think this could be done in the Makefile." First I replaced the hardcoded constant in regrtest.py by a command line argument (--timeout=3600) in the TESTOPTS variable of the Makefile, so if you call regrtest directly (without make), there is no more default timeout. But today I saw a a buildbot timeout without any traceback: a possible hang in test_io on "x86 FreeBSD 7.2 3.x" buildbot, "command timed out: 3900 seconds without output". I realized that some buildbots (all buildbots?) override the TESTOPTS variable ("make buildbottest TESTOPTS= TESTPYTHONOPTS=" for "x86 FreeBSD 7.2 3.x"). I moved the timeout option from TESTOPTS to a new variable: TESTTIMEOUT, with a default value of 1 hour (3600 seconds). The timeout is now only used for "make buildbottest". Use TESTTIMEOUT=0 to disable completly regrtest timeout. The timeout argument if ignored (with a warning) if faulthandler.dump_tracebacks_later() function is missing (e.g. if Python is compiled without threads). Please, don't disable the TESTTIMEOUT option for your buildbot. You may want to use a shorter or longer timeout, it just have to be smaller than the buildbot timeout (3900 seconds by default, 1 hour 6 minutes, just a little bit more than regrtest timeout: 1 hour). If you want a timeout longer than 1 hour, you can to change TESTTIMEOUT and the buildbot timeout (I don't know how to configure the buildbot timeout). I didn't touch test, testall, testuniversal, quicktest make rules, which don't use the regrtest timeout anymore. I would prefer to use the same timeout options for all test rules in TESTOPTS, but it doesn't work because some buildbots do override TESTOPTS variable. Victor

3 2

packaging landed in stdlib
by Tarek Ziadé 06 Jun '11

06 Jun '11

Hey I've pushed packaging in stdlib. There are a few buildbots errors we're fixing right now. We will continue our work in their directly for now on. The next "big" commit will be for the documentation, Cheers Tarek -- Tarek Ziadé | http://ziade.org

3 4