Mailman 3 September 2015 - Python-Dev

The changes I am planning to NumPy I'd like to make only available on Python 3
by Travis Oliphant Sept. 12, 2015

Sept. 12, 2015

If it helps anyone in their interest level. My intention would be to make these changes to NumPy only available on Python 3 as a way to help continue adoption of Python 3. -Travis -- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io

1 0

Looking for a developer who will work with me for at least 6 months to fix NumPy's dtype system.
by Travis Oliphant Sept. 12, 2015

Sept. 12, 2015

Hi all, Apologies for cross-posting, but I need to get the word out and twitter doesn't provide enough explanation. I've been working on a second edition of my "Guide to NumPy" book. It's been a time-pressured activity, but it's helped me put more meat around my ideas for how to fix NumPy's dtype system -- which I've been contemplating off an on for 8 years. I'm pretty sure I know exactly how to do it --- in a way that fits more cleanly into Python. It will take 3-6 months and will have … [View More]residual efforts needed that will last another 6 months --- making more types available with NumPy, improving calculations etc. This work will be done completely in public view and allow for public comment. It will not solve *all* of NumPy's problems, but it will put NumPy's dtype system on the footing it in retrospect should have been put on in the first place (if I had known then what I know now). It won't be a grandiose rewrite. It will be a pretty surgical fix to a few key places in the code. However, it will break the ABI and require recompilation of NumPy extensions (and so would need to be called NumPy 2.0). This is unavoidable, but I don't see any problem with breaking the ABI today given how easy it is to get distributions of Python these days from a variety of sources (including using conda --- but not only using conda). For those that remember what happened in Python dev land, the changes will be similar to when Guido changed Python 1.5.2 to Python 2.0. I can mentor and work closely with someone who will work on this and we will invite full participation and feedback from whomever in the community also wants to participate --- but I can't do it myself full time (and it needs someone full time+). Fortunately, I can pay someone to do it if they are willing to commit at least 6 months (it is not required to work at Continuum for this, but you can have a job at Continuum if you want one). I'm only looking for people who have enough experience with C or preferably the Python C-API. You also have to *want* to work on this. You need to be willing to work with me on the project directly and work to have a mind-meld with my ideas which will undoubtedly give rise to additional perspectives and ideas for later work for you. When I wrote NumPy 1.0, I put in 80+ hour weeks for about 6 months or more and then 60+ weeks for another year. I was pretty obsessed with it. This won't need quite that effort, but it will need something like it. Being able to move to Austin is a plus but not required. I can sponsor a visa for the right candidate as well (though it's not guaranteed you will get one with the immigration policies what they are). This is a labor of love for so many of us and my desire to help the dtype situation in NumPy comes from the same space that my desire to work on NumPy in the first place came. I will be interviewing people to work on this as not everyone who may want to will really be qualified to do it --- especially with so many people writing Cython these days instead of good-ole C-API code :-) Feel free to spread the news to anyone you can. I won't say more until I've found someone to work with me on this --- because I won't have the time to follow-up with any questions or comments. Even if I can't find someone I will publish the ideas --- but that also takes time and effort that is in short supply for me right now. If there is someone willing to fund this work, please let me know as well -- that could free up more of my time. Best, -Travis -- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io [View Less]

1 0

Choosing an official stance towards module deprecation in Python 3
by Brett Cannon Sept. 12, 2015

Sept. 12, 2015

There are two discussions going on in the issue tracker about deprecating some modules and it has led to the inevitable discussion of Python 2/3 compatibility (I'm not even going to bother mentioning the issue #s as this thread is not about the modules specifically but module deprecation in general). Because I'm tired of rehashing the same discussion every time a module deprecation comes up I would like to make an official decision that we can follow any time we decide to deprecate a module. … [View More]

10 15

Partial function application
by Herbert Kruitbosch Sept. 11, 2015

Sept. 11, 2015

Dear developers, First of all, I'm a programmer for a data science company and I recently graduated. That being said, I have wondered why python does not have syntactical support (like syntax sugar) for partial function application. I think partial function application is a powerful concept, but also think that the implementation in functional.partial as described here: https://www.python.org/dev/peps/pep-0309/ is too verbose. Moreover I think the functional programming paradigm is a … [View More]

3 3

Summary of Python tracker Issues
by Python tracker Sept. 11, 2015

Sept. 11, 2015

ACTIVITY SUMMARY (2015-09-04 - 2015-09-11) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5076 (+19) closed 31753 (+49) total 36829 (+68) Open issues with patches: 2258 Issues opened (46) ================== #8232: webbrowser.open incomplete on Windows http://bugs.python.org/issue8232 reopened by larry #24199: Idle: remove idlelib.idlever.py and … [View More]

1 0

Re: [Python-Dev] cpython (3.5): whatsnew/3.5: Editorialization pass on library section
by Serhiy Storchaka Sept. 10, 2015

Sept. 10, 2015

On 11.09.15 00:36, yury.selivanov wrote: > https://hg.python.org/cpython/rev/3265f33df731 > changeset: 97874:3265f33df731 > branch: 3.5 > parent: 97872:70c97a626c41 > user: Yury Selivanov <yselivanov(a)sprymix.com> > date: Thu Sep 10 17:35:38 2015 -0400 > summary: > whatsnew/3.5: Editorialization pass on library section > > Patch by Elvis Pranskevichus > io > -- > > -* New Python implementation of :class:`io.… [View More]

2 1

Yet another "A better story for multi-core Python" comment
by Gary Robinson Sept. 10, 2015

Sept. 10, 2015

Folks, If it’s out of line in some way for me to make this comment on this list, let me know and I’ll stop! But I do feel strongly about one issue and think it’s worth mentioning, so here goes. I read the "A better story for multi-core Python” with great interest because the GIL has actually been a major hindrance to me. I know that for many uses, it’s a non-issue. But it was for me. My situation was that I had a huge (technically mutable, but unchanging) data structure which needed a lot of … [View More]analysis. CPU time was a major factor — things took days to run. But even so, my time as a programmer was much more important than CPU time. I needed to prototype different algorithms very quickly. Even Cython would have slowed me down too much. Also, I had a lot of reason to want to make use of the many great statistical functions in SciPy, so Python was an excellent choice for me in that way. So, even though pure Python might not be the right choice for this program in a production environment, it was the right choice for me at the time. And, if I could have accessed as many cores as I wanted, it may have been good enough in production too. But my work was hampered by one thing: There was a huge data structure that all the analysis needed to access. Using a database would have slowed things down too much. Ideally, I needed to access this same structure from many cores at once. On a Power8 system, for example, with its larger number of cores, performance may well have been good enough for production. In any case, my experimentation and prototyping would have gone more quickly with more cores. But this data structure was simply too big. Replicating it in different processes used memory far too quickly and was the limiting factor on the number of cores I could use. (I could fork with the big data structure already in memory, but copy-on-write issues due to reference counting caused multiple copies to exist anyway.) So, one thing I am hoping comes out of any effort in the “A better story” direction would be a way to share large data structures between processes. Two possible solutions: 1) More the reference counts away from data structures, so copy-on-write isn’t an issue. That sounds like a lot of work — I have no idea whether it’s practical. It has been mentioned in the “A better story” discussion, but I wanted to bring it up again in the context of my specific use-case. Also, it seems worth reiterating that even though copy-on-write forking is a Unix thing, the midipix project appears to bring it to Windows as well. (http://midipix.org) 2) Have a mode where a particular data structure is not reference counted or garbage collected. The programmer would be entirely responsible for manually calling del on the structure if he wants to free that memory. I would imagine this would be controversial because Python is currently designed in a very different way. However, I see no actual risk if one were to use an @manual_memory_management decorator or some technique like that to make it very clear that the programmer is taking responsibility. I.e., in general, information sharing between subinterpreters would occur through message passing. But there would be the option of the programmer taking responsibility of memory management for a particular structure. In my case, the amount of work required for this would have been approximately zero — once the structure was created, it was needed for the lifetime of the process. Under this second solution, there would be little need to actually remove the reference counts from the data structures — they just wouldn’t be accessed. Maybe it’s not a practical solution, if only because of the overhead of Python needing to check whether a given structure is manually managed or not. In that case, the first solution makes more sense. In any case I thought this was worth mentioning, because it has been a real problem for me, and I assume it has been a real problem for other people as well. If a solution is both possible and practical, that would be great. Thank you for listening, Gary -- Gary Robinson garyrob(a)me.com http://www.garyrobinson.net [View Less]

12 16

Can't post to bugs.python.org
by Serhiy Storchaka Sept. 10, 2015

Sept. 10, 2015

I can't neither post a message to existing issue nor open a new issue. The irker854 bot on IRC channel #python-dev cites my message and the tracker updates activity time of existing issue, but doesn't show my message and doesn't reflect changes of status. Posting via e-mail doesn't work as well. Web server replies: An error has occurred A problem was encountered processing your request. The tracker maintainers have been notified of the problem. E-mail server replies: Subject: Failed … [View More]

2 2

Embedding Python 3.5
by Paul Moore Sept. 9, 2015

Sept. 9, 2015

First of all, an apology. I know this is probably going to be of limited interest for many on python-dev, but I'm honestly not sure where there's a better audience for the question. As it's related to how people should use the new "embeddable" distribution of Python 3.5, I'm hoping it's sufficiently relevant - but if not, feel free to direct me to a better forum. I've been trying to use the new embeddable distribution to build a Windows installation of vim that includes a Python interpreter, … [View More]rather than relying on the user having a system install of Python on PATH. This strikes me as precisely the sort of usage that the embeddable distribution would be great for. And indeed, it works fine for that. But when it comes to distribution, I have an issue. The problem is that for vim to find the Python DLL, it needs to be alongside vim.exe (there's a small complication here I'll come to later, but let's ignore that for now). So I unzip the embeddable distribution into the directory containing vim.exe. However, I want that directory on PATH (so I can run vim from the command line). Now, as the embeddable distribution includes python.exe, this pollutes my PATH with an extra copy of python, which may or may not be picked up in preference to my "real" python, depending on how I install python. That's bad - I can delete python.exe and pythonw.exe from my vim installation, but I still have a load of Python DLLs on PATH that could potentially mess up other Python installations on my machine (I honestly don't know how likely that is, particularly if I distribute my vim build to other users with different setups). What I'd like to do is to put the Python stuff in a subdirectory of the Vim installation, so it's safely isolated. In the case of vim, I can do this, as vim has an option to dynamically load the Python DLL via LoadLibrary, and I can patch vim to look in <vim dir>\python before searching PATH. But the LoadLibrary stuff is horribly fragile, and I doubt anyone else embedding Pthon in their application is likely to do that - they will probably just link to python35.dll at load-time. I'm not aware of any way to adjust the loader's search path for DLLs to include a subdirectory that's not on PATH, so it seems to me that there's no good way to avoid having the embeddable distribution visible via the user's PATH. For vim, I'll probably go for the dynamic loading approach with a patch to look in a subdirectory. But have I missed any other approaches that make an embedded Python more isolated from the user's system? Paul [View Less]

3 4

Re: [Python-Dev] devguide: Drop myself from issue assignment list
by Serhiy Storchaka Sept. 9, 2015

Sept. 9, 2015

On 28.06.15 09:14, nick.coghlan wrote: > https://hg.python.org/devguide/rev/452f840bac9f > changeset: 749:452f840bac9f > user: Nick Coghlan <ncoghlan(a)gmail.com> > date: Sun Jun 28 16:13:54 2015 +1000 > summary: > Drop myself from issue assignment list > > files: > experts.rst | 6 +++--- > 1 files changed, 3 insertions(+), 3 deletions(-) > > > diff --git a/experts.rst b/experts.rst > --- a/experts.rst > +++ b/experts.… [View More]

1 2