Mailman 3 February 2017 - Python-Dev

Re: [Python-Dev] Generator objects and list comprehensions?
by Anders Munch 03 Feb '17

03 Feb '17

Craig Rodrigues <rodrigc(a)freebsd.org>: > Make this return a list on Python 3, like in Python 2: [(yield 1) for x in range(10)] Give Python 2 a little more credit. Python 2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> [(yield 1) for x in range(10)] File "<stdin>", line 1 SyntaxError: 'yield' outside function regards, Anders

2 1

Heads up: possible double-comments on bpo for commits
by Brett Cannon 02 Feb '17

02 Feb '17

I've activated the webhook for receiving comments on issues when a commit lands mentioning an issue, so if you see a commit from our hg integration and another from GitHub, understand that's why (mention issues as "bpo NNNN" in commit messages if you want to see it in action). If it becomes too much of a hassle to see the duplicates before we migrate I can turn off the notifications, but obviously more testing the better. :)

3 5

re performance
by Sven R. Kunze 02 Feb '17

02 Feb '17

Hi folks, I recently refreshed regular expressions theoretical basics *indulging in reminiscences* So, I read https://swtch.com/~rsc/regexp/regexp1.html However, reaching the chart in the lower third of the article, I saw Python 2.4 measured against a naive Thompson matching implementation. And I was surprised about how bad it performed compared to an unoptimized version of an older than dirt algorithm. So, I decided to give it a try with Python2.7 and Python3.5. Eh, voilà, 100% cpu and no results so far. Nothing has changed at all since 2007. >>> import re >>> re.match(r'a?'*30 + r'a'*30, 'a'*30) .... (still waiting) Quoting from the article: " Some might argue that this test is unfair to the backtracking implementations, since it focuses on an uncommon corner case. This argument misses the point: given a choice between an implementation with a predictable, consistent, fast running time on all inputs or one that usually runs quickly but can take years of CPU time (or more) on some inputs, the decision should be easy." Victor, as the head of Python performance department, and Matthew, as the maintainer of the new regex module, what is your stance on this particular issue? From my perspective, I can say, that regular expressions might worth optimizing especially for web applications (url matching usually uses regexes) but also for other applications where I've seen many tight loops using regexes as well. So, I am probing interest on this topic here. Best, Sven

22 38

Investigating Python memory footprint of one real Web application
by INADA Naoki 02 Feb '17

02 Feb '17

Hi, all. After reading Instagram's blog article [1], I’m thinking about how Python can reduce memory usage of Web applications. My company creating API server with Flask, SQLAlchemy and typing. (sorry, it's closed source). So I can get some data from it's codebase. [1]: https://engineering.instagram.com/dismissing-python-garbage-collection-at-i… Report is here https://gist.github.com/methane/ce723adb9a4d32d32dc7525b738d3c31 My thoughts are: * Interning (None,) seems worth enough. * There are many empty dicts. Allocating ma_keys lazily may reduce memory usage. * Most large strings are docstring. Is it worth enough that option for trim docstrings, without disabling asserts? * typing may increase memory footprint, through functions __attributes__ and abc. * Can we add option to remove or lazy evaluate __attributes__ ? * Using string literal for annotating generic type may reduce WeakRef usage. * Since typing will be used very widely in this year. Need more investigating.

9 32

Why doesn't module finalization delete names as expected?
by Philippe Proulx 01 Feb '17

01 Feb '17

Hello. I'm not sure if I'm posting to the right list. If it's not the case, please tell me which one to post to. Using Python 3.5.2. I'm developing a C module with the help of SWIG. My library manages objects with reference counting, much like Python, except that it's deterministic: there's no GC. I create two Python objects like this: bakery = Bakery() bread = bakery.create_bread() Behind the scenes, the situation looks like this: +--------------------+ | UserBread obj (Py) | +----------^---+-----+ | : | : | : +------------------+ +---------+---V---+ | Bakery obj (lib) <----------------+ Bread obj (lib) | +--------^---+-----+ +--------^--------+ | : | | : | +--------+---V----+ +--------+-------+ | Bakery obj (Py) | | Bread obj (Py) | +---------^-------+ +------^---------+ | | | | + + bakery bread A pipe link means one "strong" reference and a colon link means one borrowed/weak reference. I have some ownership inversion magic for the Bakery lib. and Python objects to always coexist. So here it's pretty clear what can happen. I don't know which reference gets deleted first, but let's assume it's `bakery`. Then the situation looks like this: +--------------------+ | UserBread obj (Py) | +----------^---+-----+ | : | : | : +------------------+ +---------+---V---+ | Bakery obj (lib) <----------------+ Bread obj (lib) | +--------^---+-----+ +--------^--------+ : | | : | | +--------+---V----+ +--------+-------+ | Bakery obj (Py) | | Bread obj (Py) | +-----------------+ +------^---------+ | | + bread The Bakery Python object's __del__() drops the reference to its library object, but first its reference count is incremented during this call (so it's not really destroyed) and it's marked as only owned by the library object from now on. When `bread` gets deleted: 1. The Bread Python object's __del__() method gets called: the reference to its library object is dropped. 2. The Bread library object's destroy function drops its reference to the Bakery library object. 3. The Bakery library object's destroy function drops its reference to the Bakery Python object. 4. The Bakery Python object's __del__() method does nothing this time, since the object is marked as only owned by its library object (inverted ownership). 5. The Bread library object's destroy function then drops its reference to the UserBread Python object. In the end, everything is correctly destroyed and released. This also works if `bread` is deleted before `bakery`. My problem is that this works as expected when used like this: def func(): bakery = Bakery() bread = bakery.create_bread() if __name__ == '__main__': func() but NOTHING is destroyed when used like this: bakery = Bakery() bread = bakery.create_bread() That is, directly during the module's initialization. It works, however, if I delete `bread` manually: bakery = Bakery() bread = bakery.create_bread() del bread It also works with `bakery` only: bakery = Bakery() My question is: what could explain this? My guess is that my logic is correct since it works fine in the function call situation. It feels like `bread` is never deleted in the module initialization situation, but I don't know why: the only reference to the Bread Python object is this `bread` name in the module... what could prevent this object's __del__() method to be called? It works when I call `del bread` manually: I would expect that the module finalization does the exact same thing? Am I missing anything? Thanks, Phil

2 1

SSL certificates recommendations for downstream python packagers
by David Cournapeau 01 Feb '17

01 Feb '17

Hi, I am managing the team responsible for providing python packaging at Enthought, and I would like to make sure we are providing a good (and secure) out of the box experience for SSL. My understanding is that PEP 476 is the latest PEP that concerns this issue, and that PEP recommends using the system store: https://www.python.org/dev/peps/pep-0476/#trust-database. But looking at binary python distributions from python.org, that does not seem to a.ways be the case. I looked at the following: * 3.5.3 from python.org for OS X (64 bits): this uses the old, system openssl * 3.6.0 from python.org for OS X: this embeds a recent openssl, but ssl seems to be configured to use non existing paths (ssl..get_default_verify_paths()), and indeed, cert validation seems to fail by default with those installers * 3.6.0 from python.org for windows: I have not found how the ssl module finds the certificate, but certification seems to work Are there any official recommendations for downstream packagers beyond PEP 476 ? Is it "acceptable" for downstream packagers to patch python's default cert locations ? David

5 18

Core Python Projects for GSoC 2017?
by Terri Oda 01 Feb '17

01 Feb '17

Hey all, it's Google Summer of Code time again! For those not familiar with it, Google Summer of Code is a mentoring program where Google provides stipends to pay students to work with selected open source organizations. Python has participated for many years now, and we're hoping to be selected to participate again. One of the things we have to do to apply is have a bunch of interesting ideas for students to work on and a bunch of mentors who are excited about working with students. The PSF page is http://python-gsoc.org/ That has contact information and answers to questions like "what does it take to be a mentor?" and please feel free to ask if there's more you want to know! A good project idea is one that is useful to the community and can be completed by a student with the help of mentors during the 3-month duration of the program. We want at least two mentors available per idea (so that it's not a big deal if someone's at a conference or sick) and at least one of the mentors (usually the primary one) should be able to review code and work with the community to get code merged upstream when it's ready. So... is anyone here interested in mentoring this year, and does anyone have any great ideas of things that we need and can support in Core Python? Brainstorming appreciated! (Although only ideas with actual mentoring support will make the final cut.) I've got a stub of a Core Python ideas page here: https://wiki.python.org/moin/SummerOfCode/2017/python-core (If you can't edit the wiki, I can add people's usernames to the editor lists, or I can add ideas that are emailed to me.) And our ideas page template that has a checklist of things students usually want to know is here: https://wiki.python.org/moin/SummerOfCode/OrgIdeasPageTemplate If you're interested in mentoring, email gsoc-admins(a)python.org so I can make sure you're on the list of people I ping with reminders. We need a good set of ideas by Feb 7 for our application, and then if we're accepted by Google, we can add a few other ideas up until Feb 28 or so. Terri

1 0