The long anticipated "emergency" 2.7.5 release has now been tagged. It
will be publicly announced as binaries arrive.
Originally, I was just going to cherrypick regression fixes onto the
2.7.4 release and release those as 2.7.5. I started to this but ran
into some conflicts. Since we don't have buildbot testing of release
branches, I decided it would be best to just cut from the maintenance
I have a reproducable crash on Windows XP with Python 2.7 which I would
like to investigate. I have Visual Studio 2008 installed and I
downloaded the pdb files. However I could not find any instructions on
how to use them and was unsuccessful at getting anything out of it.
I checked the developer guide but could not find anything on debugging
crashes. On internet, this seems to be also an underdocumented topic.
So, a few questions :
- is there some documentation to help debugging crashes ?
- are the pdb files released along python usable with Visual Studio and
stock Python ? Or do you need a hand-compiled version ?
On 05/13/2013 11:32 AM, Guido van Rossum wrote:
> But now you enter a different phase of your project, or one of your collaborators does, or perhaps you've released your code on PyPI and one of your users does. So someone tries to pickle some class instance that happens to contain an unpicklable enum. That's not a great experience. Pickling and unpickling errors are often remarkably hard to debug. (Especially the latter, so I have privately admonished Ethan to ensure that if the getframe hack doesn't work, the pickle failure should happen at pickling time, not at unpickle time.)
I can get pickle failure on members created using the functional syntax with no module set; I cannot get pickle failure
on those same classes; I cannot get pickle failure on class syntax enums that inherit complex types (such as the NEI
class in the tests).
If anybody has any insight on how to make that work, I'm all ears.
Hi guys! This is my first post on this list.
I'd like have your opinion on how to safely implement WSGI on a production server.
My benchmarks show no performance differences between our PHP and Python environments. I'm using mod_wsgi v3.4 with Apache 2.4.
Is that ok or can it get faster?
Thanks in advance.
A few of us were having a discussion at
http://bugs.python.org/issue11406 about adding os.scandir(): a
generator version of os.listdir() to make iterating over very large
directories more memory efficient. This also reflects how the OS gives
things to you -- it doesn't give you a big list, but you call a
function to iterate and fetch the next entry.
While I think that's a good idea, I'm not sure just that much is
enough of an improvement to make adding the generator version worth
But what would make this a killer feature is making os.scandir()
generate tuples of (name, stat_like_info). The Windows directory
iteration functions (FindFirstFile/FindNextFile) give you the full
stat information for free, and the Linux and OS X functions
(opendir/readdir) give you partial file information (d_type in the
dirent struct, which is basically the st_mode part of a stat, whether
it's a file, directory, link, etc).
Having this available at the Python level would mean we can vastly
speed up functions like os.walk() that otherwise need to make an
os.stat() call for every file returned. In my benchmarks of such a
generator on Windows, it speeds up os.walk() by 9-10x. On Linux/OS X,
it's more like 1.5-3x. In my opinion, that kind of gain is huge,
especially on Windows, but also on Linux/OS X.
So the idea is to add this relatively low-level function that exposes
the extra information the OS gives us for free, but which os.listdir()
currently throws away. Then higher-level, platform-independent
functions like os.walk() could use os.scandir() to get much better
performance. People over at Issue 11406 think this is a good idea.
HOWEVER, there's debate over what kind of object the second element in
the tuple, "stat_like_info", should be. My strong vote is for it to be
a stat_result-like object, but where the fields are None if they're
unknown. There would be basically three scenarios:
1) stat_result with all fields set: this would happen on Windows,
where you get as much info from FindFirst/FindNext as from an
2) stat_result with just st_mode set, and all other fields None: this
would be the usual case on Linux/OS X
3) stat_result with all fields None: this would happen on systems
whose readdir()/dirent doesn't have d_type, or on Linux/OS X when
d_type was DT_UNKNOWN
Higher-level functions like os.walk() would then check the fields they
needed are not None, and only call os.stat() if needed, for example:
# Build lists of files and directories in path
files = 
dirs = 
for name, st in os.scandir(path):
if st.st_mode is None:
st = os.stat(os.path.join(path, name))
Not bad for a 2-10x performance boost, right? What do folks think?
P.S. A few non-essential further notes:
1) As a Windows guy, a nice-to-have addition to os.scandir() would be
a keyword arg like win_wildcard which defaulted to '*.*', but power
users can pass in to utilize the wildcard feature of
FindFirst/FindNext on Windows. We have plenty of other low-level
functions that expose OS-specific features in the OS module, so this
would be no different. But then again, it's not nearly as important as
exposing the stat info.
2) I've been dabbling with this concept for a while in my BetterWalk
Note that the benchmarks there are old, and I've made further
improvements in my local copy. The ctypes version gives speed gains
for os.walk() of 2-3x on Windows, but I've also got a C version, which
is giving 9-10x speed gains. I haven't yet got a Linux/OS X version
written in C.
3) See also the previous python-dev thread on BetterWalk:
While reviewing the behaviour of Vinay's "distil" installer tool (see
distutils-sig for details, but it's not relevant here) I have found what I
think is a flaw in the behaviour of the py.exe launcher for Windows.
To recap for people unfamiliar with the launcher, it emulates #! line
interpretation on Windows, interpreting commonly used forms from Unix and
launching the appropriate installed Python interpreter (finding its
location from the registry, as python.exe may not be on PATH).
The problem is with the interpretation of #!/usr/bin/env python. The
launcher treats this the same as #!/usr/bin/python, launching the "default"
Python. But that is *not* what the equivalent line does on Unix, where it
launches the *currently active* Python (a crucial difference when there is
an active virtualenv). The result is that a script written to run with the
active Python works on Unix as expected, but can use an unexpected version
of Python on Windows. This is particularly unpleasant when the program in
question is an (un)installer like distil!
I would propose that the behaviour of the launcher on Windows should be
changed when it encounters specifically the hashbang line #!/usr/bin/env
python. In that case, it should search PATH for a copy of python.exe, and
if it finds one, use that. If there is no python.exe on PATH, it should
fall back to the same version of Python as would have been used if the line
This will mean that scripts written with #!/usr/bin/env python will behave
the same on Unix and Windows in the presence of activated virtualenvs.
Would people be happy with this change? If so I will open an issue on
bugs.python.org. I can look at producing a patch, as well.
I'm working on a patch to remove reference cycles from heap-allocated classes: http://bugs.python.org/issue17950
Part of the patch involves making sure that descriptors in the class dictionary don't contain strong references to the class itself.
This is item 2) in the defect description.
I have implemented this via weak references and hit no issues at all when running the test suite.
But I'd like to ask the oracle if there is anything I may be overlooking with this approach? Any hidden problems we might encounter?
An exchange in one of the enum threads prompted me to write down
something I've occasionally thought about regarding locals(): it is
currently severely underspecified, and I'd like to make the current
CPython behaviour part of the language/library specification. (We
recently found a bug in the interaction between the __prepare__ method
and lexical closures that was indirectly related to this
Specifically, rather than the current vague "post-modification of
locals may not work", I would like to explicitly document the expected
behaviour at module, class and function scope (as well as clearly
documenting the connection between modules, classes and the single-
and dual-namespace variants of exec() and eval()):
* at module scope, as well as when using exec() or eval() with a
single namespace, locals() must return the same thing as globals(),
which must be the actual execution namespace. Subsequent execution may
change the contents of the returned mapping, and changes to the
returned mapping must change the execution environment.
* at class scope, as well as when using exec() or eval() with separate
global and local namespaces, locals() must return the specified local
namespace (which may be supplied by the metaclass __prepare__ method
in the case of classes). Subsequent execution may change the contents
of the returned mapping, and changes to the returned mapping must
change the execution environment. For classes, this mapping will not
be used as the actual class namespace underlying the defined class
(the class creation process will copy the contents to a fresh
dictionary that is only accessible by going through the class
* at function scope, locals() must return a *snapshot* of the current
locals and free variables. Subsequent execution must not change the
contents of the returned mapping and changes to the returned mapping
must not change the execution environment.
Rather than adding this low level detail to the library reference
docs, I would suggest adding it to the data model section of the
language reference, with a link to the appropriate section from the
docs for the locals() builtin. The warning in the locals() docs would
be softened to indicate that modifications won't work at function
scope, but are supported at module and class scope.
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
To Python Windows Release Managers:
My name is Jianfeng Mao and I am a software developer at the U2 group in Rocket Software (http://u2.rocketsoftware.com/). I am currently working on a project to embed a slightly customized Python interpreter in our product. For easy installation and setup, we hope to be able to do the standard Python installation during the installation of our software. Basically I want to create a .msi file that can be called to install the full Python if the user needs this new feature. Brian Curtin (brian(a)python.org<mailto:email@example.com>) pointed me to Tools/msi/msi.py for the Windows MSI builder. I tried to follow the instructions in the README but couldn't make it to work after a few twists and turns. Brian mentioned that few people needs to do this and only release managers handle the packaging of Python. I have listed the steps I have done in my attempt to create the .msi file. Please let me know if I have missed anything or done anything wrong.
1. hg clone http://hg.python.org/cpython
2. cd cpython
3. hg update 3.3
4. cd tools\buildbot, edit build.bat to change the configuration from Debug to Releaes; edit external.bat, change DEBUG=1 to DEBUG=0
5. go back to cpython\ and run tools\buildbot\build.bat
6. cd PC, then do 'nmake -f icons.mak'
7. cd ..\tools\msi
8. c:\python27\python msi.py
WARNING: nm did not run successfully - libpythonXX.a not built
cl /O2 /D WIN32 /D NDEBUG /D _WINDOWS /MT /W3 /c msisupport.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.40219.01 for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.
link.exe /OUT:msisupport.dll /INCREMENTAL:NO /NOLOGO /DLL /SUBSYSTEM:WIN
DOWS /OPT:REF /OPT:ICF msisupport.obj msi.lib kernel32.lib
Creating library msisupport.lib and object msisupport.exp
Traceback (most recent call last):
File "msi.py", line 1336, in <module>
File "msi.py", line 961, in add_files
File "msi.py", line 914, in generate_license
raise ValueError, "Could not find "+srcdir+"/../"+pat
ValueError: Could not find C:\temp\cpython/../tcl8*
I'd like to mark a few PEPs that are not currently being actively
considered for 3.4 as Deferred:
S 286 Enhanced Argument Tuples von Löwis
S 337 Logging Usage in the Standard Library Dubner
S 368 Standard image protocol and class Mastrodomenico
I 396 Module Version Numbers Warsaw
S 400 Deprecate codecs.StreamReader and codecs.StreamWriter Stinner
S 419 Protecting cleanup statements from interruptions Colomiets
I 423 Naming conventions and recipes related to packaging Bryon
I 444 Python Web3 Interface
S 3124 Overloading, Generic Functions, Interfaces, and ... Eby
S 3142 Add a "while" clause to generator expressions Britton
S 3143 Standard daemon process library Finney
S 3145 Asynchronous I/O For subprocess.Popen
Pruitt, McCreary, Carlson
S 3152 Cofunctions Ewing
Obviously, they can be reactivated at any time, but I think it would
be beneficial to have the "Open" list more accurately reflect
proposals that are currently being championed.
I'd also like to mark this one as rejected by Guido at PyCon US 2013
in favour of an updated PEP 436:
S 437 A DSL for specifying signatures, annotations and ... Krah
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia