On 15-04-15, Akira Li <4kir4.1i(a)gmail.com> wrote:
> Isaac Schwabacher <ischwabacher(a)wisc.edu> writes:
> > On 15-04-15, Akira Li <4kir4.1i(a)gmail.com> wrote:
> >> Isaac Schwabacher <ischwabacher(a)wisc.edu> writes:
> >> > ...
> >> >
> >> > I know that you can do datetime.now(tz), and you can do datetime(2013,
> >> > 11, 3, 1, 30, tzinfo=zoneinfo('America/Chicago')), but not being able
> >> > to add a time zone to an existing naive datetime is painful (and
> >> > strptime doesn't even let you pass in a time zone).
> >> `.now(tz)` is correct. `datetime(..., tzinfo=tz)`) is wrong: if tz is a
> >> pytz timezone then you may get a wrong tzinfo (LMT), you should use
> >> `tz.localize(naive_dt, is_dst=False|True|None)` instead.
> > The whole point of this thread is to finalize PEP 431, which fixes the
> > problem for which `localize()` and `normalize()` are workarounds. When
> > this is done, `datetime(..., tzinfo=tz)` will be correct.
> > ijs
> The input time is ambiguous. Even if we assume PEP 431 is implemented in
> some form, your code is still missing isdst parameter (or the
> analog). PEP 431 won't fix it; it can't resolve the ambiguity by
> itself. Notice is_dst paramter in the `tz.localize()` call (current
...yeah, I forgot to throw that in there. It was supposed to be there all along. Nothing to see here, move along.
> .now(tz) works even during end-of-DST transitions (current API) when the
> local time is ambiguous.
I know that. That's what I was complaining about-- I was trying to talk about how astimezone() was going to be inadequate even after the PEP was implemented because it couldn't turn naive datetimes into aware ones, and people were giving examples that started with aware datetimes generated by now(tz), which completely went around the point I was trying to make. But it looks like astimezone() is going to grow an is_dst parameter, and everything will be OK.
This is Vamsi from Server Scripting Languages Optimization team at Intel Corporation.
Would like to submit a request to enable the computed goto based dispatch in Python 2.x (which happens to be enabled by default in Python 3 given its performance benefits on a wide range of workloads). We talked about this patch with Guido and he encouraged us to submit a request on Python-dev (email conversation with Guido shown at the bottom of this email).
Attached is the computed goto patch (along with instructions to run) for Python 2.7.10 (based on the patch submitted by Jeffrey Yasskin at http://bugs.python.org/issue4753). We built and tested this patch for Python 2.7.10 on a Linux machine (Ubuntu 14.04 LTS server, Intel Xeon - Haswell EP CPU with 18 cores, hyper-threading off, turbo off).
Below is a summary of the performance we saw on the "grand unified python benchmarks" suite (available at https://hg.python.org/benchmarks/). We made 3 rigorous runs of the following benchmarks. In each rigorous run, a benchmark is run 100 times with and without the computed goto patch. Below we show the average performance boost for the 3 rigorous runs.
Python 2.7.10 (original) vs Computed Goto performance
Delta (rigorous run #1) %
Delta (rigorous run #2) %
Delta (rigorous run #3) %
Avg. Delta %
From: gvanrossum(a)gmail.com<mailto:email@example.com> [mailto:firstname.lastname@example.org] On Behalf Of Guido van Rossum
Sent: Tuesday, May 19, 2015 1:59 PM
To: Cohn, Robert S
Cc: R. David Murray (r.david.murray(a)murrayandwalker.com<mailto:email@example.com>)
Subject: Re: meeting at PyCon
Hi Robert and David,
I just skimmed that thread. There were a lot of noises about backporting it to 2.7 but the final message on the topic, by Antoine, claimed it was too late for 2.7. However, that was before we had announced the EOL extension of 2.7 till 2020, and perhaps we were also in denial about 3.x uptake vs. 2.x. So I think it's definitively worth bringing this up. I would start with a post on python-dev linking to the source code for your patch, and adding a message to the original tracker issue too (without reopening it though -- just so the people who were on the bug will be pinged about it).
Because of backwards compatibility with previous 2.7.x releases, it's very important that the patch not break anything -- in particular this means you can't add opcodes or change their specification. You will also undoubtedly be asked to test this on a variety of platforms 32-bit and 64-bit that people care about. But I'm sure you're expecting all that. :-)
You might also check with Benjamin Peterson, who is the 2.7 release manager. I think he just announced 2.7.10, so it's too late for that, but I assume we'll keep doing 2.7.x releases until 2020.
PS. I am assuming you are contributing this under a PSF-accepted license, e.g. Apache 2.0, otherwise it's an automatic nogo.
On Tue, May 19, 2015 at 9:33 AM, Cohn, Robert S <robert.s.cohn(a)intel.com<mailto:firstname.lastname@example.org>> wrote:
When we met for lunch at pycon, I asked if performance related patches would be ok for python 2.x. My understanding is that you thought it was possible if it did not create a maintainability problem. We have an example now, a 2.7 patch for computed goto based on the implementation in python 3 http://bugs.python.org/issue4753 It increases performance by up to 10% across a wide range of workloads.
As I mentioned at lunch, we hired David Murray's company, and he is guiding intel through the development process for cpython. David and I thought it would be good to run this by you before raising the issue on python-dev. Do you have a specific concern about this patch or a more general concern about performance patches to 2.7? Thanks.
Apologies in advance; I'm not a regular, and this may have been
handled already (but I couldn't find it when searching).
I've been using the new async/await functionality (congrats again to
Yury on getting that through!), and I'd like to get a stack trace
between the place at which blocking occurs and the outer co-routine.
For example, consider this code:
async def a():
async def b():
coro_a = a()
At this point I'd really like to be able to somehow get a stack trace
Using the gi_frame attribute of coro_a, I can get the line number of
the outer frame (e.g.: line 2), but from there there is no way to
descend the stack to reach the actual yield point.
I thought that perhaps the switch() co-routine could yield the frame
object returned from inspect.currentframe(), however once that
function yields that frame object has f_back changed to None.
A hypothetical approach would be to work the way down form the
outer-frame, but that requires getting access to the co-routine object
that the outer-frame is currently await-ing. Some hypothetical code
if dis.opname[coro.gi_code.co_code[coro.gi_frame.f_lasti + 1]] ==
This relies on the fact that an await-ing co-routine will be executing
a YIELD_FROM instruction. The above code uses a completely
hypothetical 'f_stack' property of frame objects to pull the
co-routine object which a co-routine is currently await-ing from the
stack. I've implemented a proof-of-concept f_stack property in the
frameobject.c just to test out the above code, and it seems to work.
With all that, some questions:
1) Does anyone else see value in trying to get the stack-trace down to
the actual yield point?
2) Is there a different way of doing it that doesn't require changes
to Python internals?
3) Assuming no to #2 is there a better way of getting the information
compared to the pretty hacking byte-code/stack inspection?
Gmail dumps patch review email in my junk box. The problem seems to be
the spoofed From: header.
Received: from psf.upfronthosting.co.za ([2a01:4f8:131:2480::3])
by mx.google.com with ESMTP id
Tue, 12 May 2015 00:20:38 -0700 (PDT)
Received-SPF: softfail (google.com: domain of transitioning
storchaka(a)gmail.com does not designate 2a01:4f8:131:2480::3 as permitted
Tracker reviews are the only false positives in my junk list. Otherwise,
I might stop reviewing. Verizon does not even deliver mail that fails
its junk test, so I would not be surprised if there are people who
simply do not get emailed reviews.
Tracker posts are sent from Person Name <report(a)bugs.python.org>
Perhaps reviews could come 'from' Person Name <review(a)bugs.python.org>
Even direct tracker posts just get a neutral score.
Received-SPF: neutral (google.com: 2a01:4f8:131:2480::3 is neither
permitted nor denied by best guess record for domain of
SPF is Sender Policy Framework
Checkins mail, for instance, gets an SPF 'pass' because python.org
designates mail.python.org as a permitted sender.
Terry Jan Reedy
As the subject says, I've been unable to build the regex module against
Python 3.5b1 for 32-bit. MingGW says:
skipping incompatible .../libpython35.a when searching for -lpython35
It builds without a problem against Python 3.5 for 64-bit.
Any ideas? Should I just wait until beta 2?
On 31 May 2015 at 19:07, Ludovic Gasc <gmludo(a)gmail.com> wrote:
> About Python 3 migration, I think that one of our best control stick is
> newcomers, and by extension, Python trainers/teachers.
> If newcomers learn first Python 3, when they will start to work
> professionally, they should help to rationalize the Python 3 migration
> inside existing dev teams, especially because they don't have an interest
> conflict based on the fact that they haven't written plenty of code with
> Python 2.
> 2020 is around the corner, 5 years shouldn't be enough to change the
> community mind, I don't know.
The education community started switching a while back - if you watch
Carrie-Anne Philbin's PyCon UK 2014 keynote, one of her requests for
the broader Python community was for everyone else to just catch up
already in order to reduce student's confusion (she phrased it more
politely than that, though). Educators need to tweak examples and
exercises to account for a version switch, but that's substantially
easier than migrating hundreds of thousands or even millions of lines
of production code.
And yes, if you learn Python 3 first, subsequently encountering Python
2's quirks and cruft is likely to encourage folks that know both
versions of the language to start advocating for a version upgrade :)
After accounting for the "Wow, the existing Python 2 install base is
even larger than we realised" factour, the migration is actually in a
pretty good place overall these days. The "enterprise" crowd really
are likely to be the only ones that might need the full remaining 5
years of migration time (and they may potentially have even more time,
if they're relying on a commercial redistributor).
Web frameworks have allowed Python 3 development for a while now, and
with Django switching their tutorial to Python 3 by default, Django
downloads via pip show one of the highest proportions of Python 3
adoption on PyPI. www.python.org itself is now a production Python 3
Django web service, and the next generation of pypi.python.org will be
a Pyramid application that's also running on Python 3.
The dedicated async/await syntax in 3.5 represents a decent carrot to
encourage migration for anyone currently using yield (or yield from)
based coroutines, since the distinct syntax not only allows for easier
local reasoning about whether something is an iterator or a coroutine,
it also provides a much improved user experience for asynchronous
iterators and context managers (including finally handling the
"asynchronous database transaction as a context manager" case, which
previous versions of Python couldn't really do at all).
The matrix multiplication operator is similarly a major improvement
for the science and data analysis part of the Python community.
In terms of reducing *barriers* to adoption, after inviting them to
speak at the 2014 language summit, we spent a fair bit of time with
the Twisted and Mercurial folks over the past year or so working
through "What's still missing from Python 3 for your use cases?", as
Python 3.4 was still missing some features for binary data
manipulation where we'd been a bit too ruthless in pruning back the
binary side of things when deciding what counted as text-only
features, and what was applicable to binary data as well. So 3.5
brings back binary interpolation, adds a hex() method to bytes, and
adds binary data support directly to a couple of standard library
modules (tempfile, difflib).
If I understand the situation correctly, the work Guido et al have
been doing on PEP 484 and type hinting standardisation is also aimed
at reducing barriers to Python 3 adoption, by making it possible to
develop better migration tools that are more semantically aware than
the existing syntax focused tools. The type hinting actually acts as a
carrot as well, since it's a feature that mainly shows its value when
attempting to scale a *team* to larger sizes (as it lets you delegate
more of the code review process to an automated tool, letting the
human reviewers spend more time focusing on higher level semantic
Finally, both Debian/Ubuntu and Fedora are well advanced in their
efforts to replace Python 2 with Python 3 in their respective default
images (but keeping Py2 available in their package repos). That work
is close to finished now (myself, Slavek Kabrda, Barry Warsaw, and
Matthias Klose had some good opportunities to discuss that at PyCon),
although there are still some significant rough edges to figure out
(such as coming up with a coherent cross-platform story for what we're
going to do with the Python symlink), as well as a few more key
projects to either migrate entirely, or at least finish porting to the
source compatible subset of Python 2 & 3 (e.g. Samba).
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
The buildbots currently live in a state of denial about the 3.5 branch.
Could someone whisper tenderly in their collective shell-like ears so
that they start building 3.5, in addition to 3.4 and trunk?
Donald Stufft wrote:
> Well Python 3.4.3 binary is 4kb for me, so you'd have that + your 1KB Python script + whatever other pieces you need.
For contrast, here are the things you need on Windows to be able to get to an interactive prompt (I don't know how other platforms get this down to 4KB...):
* python.exe (or some equivalent launcher) 39KB
* python35.dll 3,788KB
* vcruntime140.dll 87KB (the rest of the CRT is about 1MB, but is not redistributable so doesn't count here)
* 26 files in Lib 343KB
This gets you to ">>>", and basically everything after that is going to fail for some reason. That's an unavoidable 4,257KB.
The rest of the stdlib adds another ~16MB once you exclude the test suite, so a fully functioning Python is not cheap. (Using compressed .pyc's in a zip file can make a big difference here though, assuming you're willing to trade CPU for HDD.)