Python-Dev May 2015

python-dev@python.org

112 participants
85 discussions

[RELEASED] Python 3.5.0b1 is now available

by Larry Hastings

On behalf of the Python development community and the Python 3.5 release team, I'm pleased to announce the availability of Python 3.5.0b1. Python 3.5 has now entered "feature freeze". By default new features may no longer be added to Python 3.5. (However, there are a handful of features that weren't quite ready for Python 3.5.0 beta 1; these were granted exceptions to the freeze, and are scheduled to be added before beta 2.) This is a preview release, and its use is not recommended for production settings. Three important notes for Windows users about Python 3.5.0b1: * If you have previously installed Python 3.5.0a1, you may need to manually uninstall it before installing Python 3.5.0b1 (issue23612). * If installing Python 3.5.0b1 as a non-privileged user, you may need to escalate to administrator privileges to install an update to your C runtime libraries. * There is now a third type of Windows build for Python 3.5. In addition to the conventional installer and the web-based installer, Python 3.5 now has an embeddable release designed to be deployed as part of a larger application's installer for apps using or extending Python. During the 3.5 alpha releases, this was an executable installer; as of 3.5.0 beta 1 the embeddable build of Python is now shipped in a zip file. You can find Python 3.5.0b1 here: https://www.python.org/downloads/release/python-350b1/ Happy hacking, //arry/

5 years, 10 months

PEP 485: math.isclose()

by Chris Barker

I don't think I have permissions to comment on the issue,so I'm posting here. If there is a way for me to post to the issue, someone let me know... In the issue (http://bugs.python.org/issue24270) Tal wrote """ I have a question regarding complex values. The code (from Chris Barker) doesn't support complex values (only things that can be converted into doubles). However, the PEP states the following under "Non-float types": "complex : for complex, the absolute value of the complex values will be used for scaling and comparison. If a complex tolerance is passed in, the absolute value will be used as the tolerance." """ right -- that was written before it was decided that isclose() needed to be written in C -- the python version supported that. """ Should math.isclose() support complex values? """ nope -- the math module is all about floats. """ Should an equivalent function be added to cmath? """ I think so -- lets see if we can do that in time for 3.5 -- but first get the float one done. """ Should we just leave things as they are and remove mention of complex values from the PEP (it isn't mentioned in the docs)? """ I'll update the PEP. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

5 years, 10 months

Reminder: 3.5 now has its own branch! "default" branch is now 3.6!

by Larry Hastings

I've now pushed the 3.5.0 beta 1 release-engineering checkins to hg.python.org. At the same time I did this, I also created the 3.5 branch. Quick FAQ: Q: Where should I check in bugfixes for 3.5? A: In the "3.5" branch. You should also merge them forward into "default". Q: Where should I check in new features for 3.5? A: You sillyhead! New features aren't allowed for 3.5 anymore, it's in feature freeze. Q: What is "default" now? A: "default" is now 3.6. Meaning, you can now start on new features for 3.6! You don't have to wait until 3.5 final is released, like how we used to do it. Q: What's all this about bitbucket and push requests? A: We don't start doing that until 3.5.0 release candidate 1. Don't worry about it for now. When the time comes, I'll post instructions here and to the devguide. A new day is dawning, //arry/

5 years, 10 months

An yocto change proposal in logging module to simplify structured logs support

by Ludovic Gasc

Hi, 1. The problem For now, when you want to write a log message, you concatenate the data from your context to generate a string: In fact, you convert your structured data to a string. When a sysadmin needs to debug your logs when something is wrong, he must write regular expressions to extract interesting data. Often, he must find the beginning of the interesting log and follow the path. Sometimes, you can have several requests in the same time in the log, it's harder to find interesting log. In fact, with regular expressions, the sysadmin tries to convert the log lines strings to structured data. 2. A possible solution You should provide a set of regular expressions to your sysadmins to help them to find the right logs, however, another approach is possible: structured logs. Instead of to break your data structure to push in the log message, the idea is to keep the data structure, to attach that as metadata of the log message. For now, I know at least Logstash and Journald that can handle structured logs and provide a query tool to extract easily logs. 3. A concrete example with structured logs As most Web developers, we build HTTP daemons used by several different human clients in the same time. In the Python source code, to support structured logs, you don't have a big change, you can use "extra" parameter for that, example: [handle HTTP request] LOG.debug('Receive a create_or_update request', extra={'request_id': request.request_id, 'account_id': account_id, 'aiohttp_request': request, 'payload': str(payload)}) [create data in database] LOG.debug('Callflow created', extra={'account_id': account_id, 'request_id': request.request_id, 'aiopg_cursor': cur, 'results': row}) Now, if you want, you can enhance the structured log with a custom logging Handler, because the standard journald handler doesn't know how to handle aiohttp_request or aiopg_cursor. My example is based on journald, but you can write an equivalent version with python-logstash: #### from systemdream.journal.handler import JournalHandler class Handler(JournalHandler): # Tip: on a system without journald, use socat to test: # socat UNIX-RECV:/run/systemd/journal/socket STDIN def emit(self, record): if record.extra: # import ipdb; ipdb.set_trace() if 'aiohttp_request' in record.extra: record.extra['http_method'] = record.extra['aiohttp_request'].method record.extra['http_path'] = record.extra['aiohttp_request'].path record.extra['http_headers'] = str(record.extra['aiohttp_request'].headers) del(record.extra['aiohttp_request']) if 'aiopg_cursor' in record.extra: record.extra['pg_query'] = record.extra['aiopg_cursor'].query.decode('utf-8') record.extra['pg_status_message'] = record.extra['aiopg_cursor'].statusmessage record.extra['pg_rows_count'] = record.extra['aiopg_cursor'].rowcount del(record.extra['aiopg_cursor']) super().emit(record) #### And you can enable this custom handler in your logging config file like this: [handler_journald] class=XXXXXXXXXX.utils.logs.Handler args=() formatter=detailed And now, with journalctl, you can easily extract logs, some examples: Logs messages from 'lg' account: journalctl ACCOUNT_ID=lg All HTTP requests that modify the 'lg' account (PUT, POST and DELETE): journalctl ACCOUNT_ID=lg HTTP_METHOD=PUT HTTP_METHOD=POST HTTP_METHOD=DELETE Retrieve all logs from one specific HTTP request: journalctl REQUEST_ID=130b8fa0-6576-43b6-a624-4a4265a2fbdd All HTTP requests with a specific path: journalctl HTTP_PATH=/v1/accounts/lg/callflows All logs of "create" function in the file "example.py" journalctl CODE_FUNC=create CODE_FILE=/path/example.py If you already do a troubleshooting on a production system, you should understand the interest of this: In fact, it's like to have SQL queries capabilities, but it's logging oriented. We use that since a small time on one of our critical daemon that handles a lot of requests across several servers, it's already adopted from our support team. 4. The yocto issue with the Python logging module I don't explain here a small part of my professional life for my pleasure, but to help you to understand the context and the usages, because my patch for logging is very small. If you're an expert of Python logging, you already know that my Handler class example I provided above can't run on a classical Python logging, because LogRecord doesn't have an extra attribute. extra parameter exists in the Logger, but, in the LogRecord, it's merged as attributes of LogRecord: https://github.com/python/cpython/blob/master/Lib/logging/__init__.py#L1386 It means, that when the LogRecord is sent to the Handler, you can't retrieve the dict from the extra parameter of logger. The only way to do that without to patch Python logging, is to rebuild by yourself the dict with a list of official attributes of LogRecord, as is done in python-logstash: https://github.com/vklochan/python-logstash/blob/master/logstash/formatter.… At least to me, it's a little bit dirty. My quick'n'dirty patch I use for now on our CPython on production: diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py index 104b0be..30fa6ef 100644 --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -1382,6 +1382,7 @@ class Logger(Filterer): """ rv = _logRecordFactory(name, level, fn, lno, msg, args, exc_info, func, sinfo) + rv.extra = extra if extra is not None: for key in extra: if (key in ["message", "asctime"]) or (key in rv.__dict__): At least to me, it should be cleaner to add "extra" as parameter of _logRecordFactory, but I've no idea of side effects, I understand that logging module is critical, because it's used everywhere. However, except with python-logstash, to my knowledge, extra parameter isn't massively used. The only backward incompatibility I see with a new extra attribute of LogRecord, is that if you have a log like this: LOG.debug('message', extra={'extra': 'example'}) It will raise a KeyError("Attempt to overwrite 'extra' in LogRecord") exception, but, at least to me, the probability of this use case is near to 0. Instead of to "maintain" this yocto patch, even it's very small, I should prefer to have a clean solution in Python directly. Thanks for your remarks. Regards. -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/

5 years, 10 months

Re: [Python-Dev] Reminder: Python 3.5 beta 1 will be tagged tomorrow

by Chris Barker

Is it too late to get the isclose() code (PEP 485) into 3.5? I posted the code here, and got a tiny bit of review, but have not yet merged it into the source tree -- and don't know the process for getting it committed to the official source. So -- too late, or should I try to get that merge done soon -- if so, how? -Chris On Fri, May 22, 2015 at 12:53 PM, Larry Hastings <larry(a)hastings.org> wrote: > > > Howdy howdy. It's-a me, Larry, your friendly neighborhood Python 3.5 > Release Manager. > > Somewhere around 2 or 3pm tomorrow I expect to tag Python 3.5 beta 1. > We'll actually release beta 1 on Sunday, once the binary installers are all > built. > > Beta 1 is also feature-freeze, meaning no new features may be added to 3.5 > without my permission. Since it seems useful to have a specific cutoff > time, please stop adding features at ** 8pm Saturday UTC **. (That's 1pm > Pacific Daylight Time. It's also almost exactly 24 hours from... now.) > > I remind you that this time we're trying something new: we're going to > create the 3.5 branch when we release beta 1, allowing feature development > (for 3.6) to continue in trunk. At the point that I check in and push beta > 1, I'll also merge all checkins from trunk back into the 3.5 branch. After > that it'll be responsibility of the person checking in to check their bug > fixes in to the appropriate place. So please keep in mind: once the 3.5 > branch becomes generally available on Sunday, the usual rules for a release > branch will apply: bug fixes for 3.5 should be checked in to the 3.5 branch > and get merged forward into trunk. > > If you have new features you want to ship with Python 3.5, please check > them in as soon as possible! > > > Thank you for helping to make Python better, > > > */arry* > > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov > > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

5 years, 10 months

Re: [Python-Dev] devguide: Updated dev guide to reflect the new workflow we're trying for 3.5.

by Ned Deily

In article <5560B054.307(a)udel.edu>, Terry Reedy <tjreedy(a)udel.edu> wrote: > I somehow did not understand this last part before. Rather I thought > the need for pull requests would be highly restricted (and not affect me > ;-). 3.5 bugfixes (and idlelib patches, unclassified), especially those > applied to 3.4, should automatically be in the next 3.5 release. I > cannot imagine a reason not to so so. Otherwise, we could end up with > the awful anomaly that a bugfix (or Idle change) could be in 3.4.4 (the > final maintenance version that comes out after 3.5.0) but not in 3.5.0 > itself. The need for "pull requests" *is* highly restricted. Note that the "pull request" proposal appears in the Release Candidate section and only applies after 3.5.0rc1 is finalized. During the Beta phase that we're entering now, bugfixes should be checked into the new 3.5 branch and they will be released first in 3.5.0b2 or 3.5.0b3. After rc1, bugfixes checked into the 3.5 branch will be released first in 3.5.1 unless they are deemed release critical for 3.5.0 in which case the pull request would be needed. The goal is to have zero release critical fixes after rc1; usually there are very few. -- Ned Deily, nad(a)acm.org

5 years, 10 months

[RELEASE] Python 2.7.10

by Benjamin Peterson

The next bugfix release of the Python 2.7.x series, Python 2.7.10, has been released. The only interesting change since the release candidate is a fix for a regression in cookie parsing. Downloads are available at: https://www.python.org/downloads/release/python-2710/ Report bugs at: https://bugs.python.org Enjoy your 2 digit versions, Benjamin (on behalf of 2.7.10's contributors)

5 years, 10 months

Re: [Python-Dev] [Python-checkins] peps: PEP 489: The PEP is accepted.

by Brett Cannon

Are you also going to check the code in or is someone else doing it? On Fri, May 22, 2015, 17:47 eric.snow <python-checkins(a)python.org> wrote: > https://hg.python.org/peps/rev/1fbc23a1078c > changeset: 5874:1fbc23a1078c > user: Eric Snow <ericsnowcurrently(a)gmail.com> > date: Fri May 22 15:45:38 2015 -0600 > summary: > PEP 489: The PEP is accepted. > > files: > pep-0489.txt | 11 ++++++++--- > 1 files changed, 8 insertions(+), 3 deletions(-) > > > diff --git a/pep-0489.txt b/pep-0489.txt > --- a/pep-0489.txt > +++ b/pep-0489.txt > @@ -7,13 +7,13 @@ > Nick Coghlan <ncoghlan(a)gmail.com> > BDFL-Delegate: Eric Snow <ericsnowcurrently(a)gmail.com> > Discussions-To: import-sig(a)python.org > -Status: Draft > +Status: Final > Type: Standards Track > Content-Type: text/x-rst > Created: 11-Aug-2013 > Python-Version: 3.5 > -Post-History: 23-Aug-2013, 20-Feb-2015, 16-Apr-2015 > -Resolution: > +Post-History: 23-Aug-2013, 20-Feb-2015, 16-Apr-2015, 7-May-2015, > 18-May-2015 > +Resolution: > https://mail.python.org/pipermail/python-dev/2015-May/140108.html > > > Abstract > @@ -730,8 +730,13 @@ > > * PyModuleDef_Slot > > +Other changes: > + > PyModuleDef.m_reload changes to PyModuleDef.m_slots. > > +``BuiltinImporter`` and ``ExtensionFileLoader`` will now implement > +``create_module`` and ``exec_module``. > + > The internal ``_imp`` module will have backwards incompatible changes: > ``create_builtin``, ``create_dynamic``, and ``exec_dynamic`` will be > added; > ``init_builtin``, ``load_dynamic`` will be removed. > > -- > Repository URL: https://hg.python.org/peps > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > https://mail.python.org/mailman/listinfo/python-checkins >

5 years, 10 months

3.5 doc warnings

by Terry Reedy

35\Doc\whatsnew\3.5.rst:686: ERROR: Unknown interpreted text role "module". 35\Doc\library\typing.rst:: WARNING: document isn't included in any toctree from building html docs just now -- Terry Jan Reedy

5 years, 10 months

Fwd: Re: [python-committers] Reminder: Python 3.5 beta 1 will be tagged tomorrow

by Larry Hastings

Whoops, didn't send my reply to both lists. Forwarded, below. -------- Forwarded Message -------- Subject: Re: [python-committers] [Python-Dev] Reminder: Python 3.5 beta 1 will be tagged tomorrow Date: Sat, 23 May 2015 12:23:09 -0700 From: Larry Hastings <larry(a)hastings.org> To: python-committers(a)python.org On 05/23/2015 06:25 AM, Nick Coghlan wrote: > I filedhttp://bugs.python.org/issue24270 to track this, but there's a > fair bit of work to be done to integrate the changes into the existing > math module's code, tests and documentation. I'm willing to consider a feature freeze exception for this, as long as * it doesn't make invasive changes (it looks like it will literally add one new entry point, which is acceptable) * it's cleaned up in the way the core devs are proposing (integrate it into the math module, including tests and documentation) * it's done before beta 2 Somebody, please take that as an encouragement to get this cleaned up and ready for checkin. //arry/ p.s. Would it make sense to add a form of isclose to unittest?

5 years, 10 months

← Newer
1
2
3
4
5
6
7
8
9
Older →

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev May 2015