Please be more precise when commenting on PEP 611.
Hi everyone, Thanks again for all your comments on PEP 611. I would like to ask a favour; please be more specific in your comments. Ideally state which part of the PEP you are disagreeing with and why you disagree with the relevant part of the rationale/motivation. Also, when asking for limits to be raised or removed entirely, could you state what you perceive to be the costs and benefits of larger limits. What do you believe is an acceptable cost in memory or runtime for larger limits? For example, you might say that the limit of one million lines of code per module is too small, and that it is worth a small, say 1%, impact on speed to allow a larger of limit of 100 million. If you believe a limit would have no cost, then please give a explanation of why that is so. Merely saying that you would like a larger limit is pointless. If there were no cost to arbitrarily large limits, then I wouldn't have proposed the PEP in the first place. Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few. Cheers, Mark.
On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon <mark@hotpy.org> wrote:
Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few.
Can we get some stats on what the costs of higher limits (or having no limit at all) is? Or, putting it the other way: Since CPython currently does not enforce limits, what are the benefits of placing those limits? Merely having a limit doesn't in itself give a benefit (and is a (minor) cost); it would help the discussion if we knew exactly WHAT the costs of the higher limits were. ChrisA
Also, I've pointed at two of those limits that worry me some - namely number of classes and number of co-routines, and had not seen so far any feedback even whether they actually make sense. A simple data analysis tasks that create a co-routine per row, and submit those for workers with more than 1 million rows is something that just works today, and is not even a burden in a desktop machine would hit such limits. Is there an easy way to locate the data-structures that would be changed on cpython that would need to be changed to limit classes and co-routines? I'd like to have at least a little more concrete idea of what could possibly be optimized if they were capped. js -><- On Mon, 9 Dec 2019 at 11:22, Chris Angelico <rosuav@gmail.com> wrote:
On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon <mark@hotpy.org> wrote:
Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few.
Can we get some stats on what the costs of higher limits (or having no limit at all) is? Or, putting it the other way: Since CPython currently does not enforce limits, what are the benefits of placing those limits? Merely having a limit doesn't in itself give a benefit (and is a (minor) cost); it would help the discussion if we knew exactly WHAT the costs of the higher limits were.
ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4IKBEMNG... Code of Conduct: http://python.org/psf/codeofconduct/
On 09/12/2019 2:15 pm, Chris Angelico wrote:
On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon <mark@hotpy.org> wrote:
Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few.
Can we get some stats on what the costs of higher limits (or having no limit at all) is? Or, putting it the other way: Since CPython currently does not enforce limits, what are the benefits of placing those limits? Merely having a limit doesn't in itself give a benefit (and is a (minor) cost); it would help the discussion if we knew exactly WHAT the costs of the higher limits were.
Given there is an infinite number of potential optimizations that it would enable, it is a bit hard to put a number on it :) It is also impossible to put precise numbers on the speedups of a particular optimizations unless it is implemented. I suspect no one is going to do that unless paid to do so, or are guaranteed that the work won't be thrown away because the PEP is rejected. Cheers, Mark.
ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4IKBEMNG... Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Dec 10, 2019 at 4:56 AM Mark Shannon <mark@hotpy.org> wrote:
On 09/12/2019 2:15 pm, Chris Angelico wrote:
On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon <mark@hotpy.org> wrote:
Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few.
Can we get some stats on what the costs of higher limits (or having no limit at all) is? Or, putting it the other way: Since CPython currently does not enforce limits, what are the benefits of placing those limits? Merely having a limit doesn't in itself give a benefit (and is a (minor) cost); it would help the discussion if we knew exactly WHAT the costs of the higher limits were.
Given there is an infinite number of potential optimizations that it would enable, it is a bit hard to put a number on it :)
Then pick one optimization and show us what it can do. Let's say you pick "1,000,000 lines in a single source file", and show that this limit (or rather, changing code to *assume* this limit) improves the performance of, say, printing of tracebacks. A few benchmarks showing how small files benefit from the banning of gigantic files would be extremely helpful.
It is also impossible to put precise numbers on the speedups of a particular optimizations unless it is implemented. I suspect no one is going to do that unless paid to do so, or are guaranteed that the work won't be thrown away because the PEP is rejected.
Fair, but even imprecise numbers would be extremely helpful. At the moment, what we have is (paraphrased): You: "We should limit things. Stuff will be faster." Others: "Really? Because bit masking is work. It'll be slower." You: "Maybe we limit it somewhere else, whatever. It'll be faster." Others: "Will it? How much faster?" You: "It'll be faster." I'm not asking you to prove that CPython 3.9 could be exactly 7.183% faster with these changes, just some indication that this will actually improve things. You've asked us to say what we believe is "an acceptable cost in memory or runtime", but at the moment, we're not paying ANY cost or getting ANY benefit, so it's really hard to randomly guess at things. ChrisA
On 09/12/2019 2:15 pm, Chris Angelico wrote: You: "We should limit things. Stuff will be faster." Others: "Really? Because bit masking is work. It'll be slower." You: "Maybe we limit it somewhere else, whatever. It'll be faster." Others: "Will it? How much faster?" You: "It'll be faster."
Mark, possibly you want to re-frame the PEP to be more like "this is good for correctness and enabling robust reasoning about the interpreter, which has a variety of benefits (and possibly speed will be one of them eventually)"? My impression is that you see speedups as a secondary motivation, while other people are getting the impression that speedups are the entire motivation, so one way or the other the text is confusing people. In particular, right now the most detailed example is the compacted object header bit, which makes it a magnet for critique. Also, I don't understand how this idea would work at all :-). So I'd either remove it or else make it more detailed, one or the other. -n -- Nathaniel J. Smith -- https://vorpus.org
On Mon, Dec 09, 2019 at 02:12:37PM -0800, Nathaniel Smith wrote:
On 09/12/2019 2:15 pm, Chris Angelico wrote: You: "We should limit things. Stuff will be faster." Others: "Really? Because bit masking is work. It'll be slower."
I'm not an expert, but the impression I've got from various discussions on performance over the last few years is that the single biggest bottleneck for CPU performance is memory locality. Cache misses are so expensive, and CPU instructions so fast, that memory locality is king and the cost of bit masking is insignificant. In other words, worrying about the cost of bit masking in C code is so very 1990s. I could be wrong of course: I'm not an expert. And I don't think we should take it for granted that this is the case, unless some experts on modern CPUs speak up and say that Mark is so obviously correct that a demonstration is unnecessary. Failing that, I think it would be good for the PEP to give some evidence that more compact data structures will be faster despite the cost of bit masking: references to reliable sources (let's say, a similar discussion on the Linux kernel mailing list), or some sample C code that skeptics can run.
You: "Maybe we limit it somewhere else, whatever. It'll be faster."
That's a totally unfair and inaccurate representation of Mark's position. The PEP doesn't say "let's put in arbitrary limits in random places for the lols", he proposed seven concrete limits and gave reasons for why he expects that they will improve memory efficiency, safety, performance or all three.
Others: "Will it? How much faster?" You: "It'll be faster."
Mark, possibly you want to re-frame the PEP to be more like "this is good for correctness and enabling robust reasoning about the interpreter, which has a variety of benefits (and possibly speed will be one of them eventually)"? My impression is that you see speedups as a secondary motivation, while other people are getting the impression that speedups are the entire motivation, so one way or the other the text is confusing people.
In fairness to Mark, the PEP is *already* framed in terms of safety and memory use. One has to read all the way to literally the very last sentence of the "Motivation" sentence before "speed" is mentioned: "There is also the potential for a more efficient instruction format, speeding up interpreter dispatch." To get to that point, you have to go past: - at least five references to efficiency; - at least four references to safety (vulnerability to overflow); - a reference to memory locality being the bottleneck on modern CPUs. In the rest of the document, Mark refers to speed three more times: - "instructions can be represented in a compact 64 bit form allowing very fast passes over the instruction sequence." - "Using super-instructions would make that the 32 bit format almost as compact as the 16 bit format, and significantly faster." - and a reference to optionally switching to a saturating ref count mechanism, which would allow the use of unlimited memory but would be "a little bit slower". (The above counts are based on Mark's initial post to the list, archived here: https://mail.python.org/archives/list/python-dev@python.org/thread/QM4QUJOBQ... and may not reflect the current state of the PEP.) Having said that though, I think you are right that the PEP could do with a bit more detail on the current status quo and existing limits, and how the proposed changes will improve safety and memory use. -- Steven
On Tue, Dec 10, 2019 at 10:51 AM Steven D'Aprano <steve@pearwood.info> wrote:
On Mon, Dec 09, 2019 at 02:12:37PM -0800, Nathaniel Smith wrote:
On 09/12/2019 2:15 pm, Chris Angelico wrote: You: "We should limit things. Stuff will be faster." Others: "Really? Because bit masking is work. It'll be slower."
I'm not an expert, but the impression I've got from various discussions on performance over the last few years is that the single biggest bottleneck for CPU performance is memory locality. Cache misses are so expensive, and CPU instructions so fast, that memory locality is king and the cost of bit masking is insignificant. In other words, worrying about the cost of bit masking in C code is so very 1990s.
I could be wrong of course: I'm not an expert. And I don't think we should take it for granted that this is the case, unless some experts on modern CPUs speak up and say that Mark is so obviously correct that a demonstration is unnecessary.
And the speculation continues.
You: "Maybe we limit it somewhere else, whatever. It'll be faster."
That's a totally unfair and inaccurate representation of Mark's position. The PEP doesn't say "let's put in arbitrary limits in random places for the lols", he proposed seven concrete limits and gave reasons for why he expects that they will improve memory efficiency, safety, performance or all three.
*He expects that*. That's what I'm talking about. We have lots of speculation and no evidence either way.
Having said that though, I think you are right that the PEP could do with a bit more detail on the current status quo and existing limits, and how the proposed changes will improve safety and memory use.
Exactly. Yes, I know that I massively oversimplified things in that post. But you nonetheless acknowledge here that we are *still* quite lacking in any actual evidence. We have people who believe that a bit mask will slow things down, others who claim that improved cache locality will speed things up, and Mark asks us to please justify our objections with numbers. But surely it's up to Mark to show numbers first? ChrisA
Chris Angelico wrote:
We have people who believe that a bit mask will slow things down, others who claim that improved cache locality will speed things up, and Mark asks us to please justify our objections with numbers. But surely it's up to Mark to show numbers first?
+1. While it would be helpful for criticisms of the PEP to be more specific and provide more of a cost-benefit analysis, the burden of proof for demonstrating the benefits provided are ultimately up to the author(s) of the PEP. We require far more justification for making a change to impose the limits in the first place than we do for maintaining the status quo. Personally, I don't think it would be reasonable to impose any of these limits without some form of concrete evidence that doing so will provide an improvement; in any combination of efficiency, performance, and/or security, as the PEP suggests would happen. But, I don't think exact numbers are needed. (I.E. adding these limits will improve performance across all Python programs by an average of 22.359%!). On Mon, Dec 9, 2019 at 7:15 PM Chris Angelico <rosuav@gmail.com> wrote:
On Tue, Dec 10, 2019 at 10:51 AM Steven D'Aprano <steve@pearwood.info> wrote:
On Mon, Dec 09, 2019 at 02:12:37PM -0800, Nathaniel Smith wrote:
On 09/12/2019 2:15 pm, Chris Angelico wrote: You: "We should limit things. Stuff will be faster." Others: "Really? Because bit masking is work. It'll be slower."
I'm not an expert, but the impression I've got from various discussions on performance over the last few years is that the single biggest bottleneck for CPU performance is memory locality. Cache misses are so expensive, and CPU instructions so fast, that memory locality is king and the cost of bit masking is insignificant. In other words, worrying about the cost of bit masking in C code is so very 1990s.
I could be wrong of course: I'm not an expert. And I don't think we should take it for granted that this is the case, unless some experts on modern CPUs speak up and say that Mark is so obviously correct that a demonstration is unnecessary.
And the speculation continues.
You: "Maybe we limit it somewhere else, whatever. It'll be faster."
That's a totally unfair and inaccurate representation of Mark's position. The PEP doesn't say "let's put in arbitrary limits in random places for the lols", he proposed seven concrete limits and gave reasons for why he expects that they will improve memory efficiency, safety, performance or all three.
*He expects that*. That's what I'm talking about. We have lots of speculation and no evidence either way.
Having said that though, I think you are right that the PEP could do with a bit more detail on the current status quo and existing limits, and how the proposed changes will improve safety and memory use.
Exactly. Yes, I know that I massively oversimplified things in that post. But you nonetheless acknowledge here that we are *still* quite lacking in any actual evidence. We have people who believe that a bit mask will slow things down, others who claim that improved cache locality will speed things up, and Mark asks us to please justify our objections with numbers. But surely it's up to Mark to show numbers first?
ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YN2BDJFP... Code of Conduct: http://python.org/psf/codeofconduct/
On 10/12/2019 7:24 am, Kyle Stanley wrote:
Chris Angelico wrote:
We have people who believe that a bit mask will slow things down, others who claim that improved cache locality will speed things up, and Mark asks us to please justify our objections with numbers. But surely it's up to Mark to show numbers first?
+1. While it would be helpful for criticisms of the PEP to be more specific and provide more of a cost-benefit analysis, the burden of proof for demonstrating the benefits provided are ultimately up to the author(s) of the PEP. We require far more justification for making a change to impose the limits in the first place than we do for maintaining the status quo.
I agree that we require justification for changing the status quo. If the status quo were the result of considered decision, then it would of course need considerable justification. If, as is the case here, the status quo is a result of historical accident and/or implementation details, then I think a weaker justification is OK.
Personally, I don't think it would be reasonable to impose any of these limits without some form of concrete evidence that doing so will provide an improvement; in any combination of efficiency, performance, and/or security, as the PEP suggests would happen. But, I don't think exact numbers are needed. (I.E. adding these limits will improve performance across all Python programs by an average of 22.359%!).
On Mon, Dec 9, 2019 at 7:15 PM Chris Angelico <rosuav@gmail.com <mailto:rosuav@gmail.com>> wrote:
On Tue, Dec 10, 2019 at 10:51 AM Steven D'Aprano <steve@pearwood.info <mailto:steve@pearwood.info>> wrote: > > On Mon, Dec 09, 2019 at 02:12:37PM -0800, Nathaniel Smith wrote: > > > > On 09/12/2019 2:15 pm, Chris Angelico wrote: > > > You: "We should limit things. Stuff will be faster." > > > Others: "Really? Because bit masking is work. It'll be slower." > > I'm not an expert, but the impression I've got from various discussions > on performance over the last few years is that the single biggest > bottleneck for CPU performance is memory locality. Cache misses are so > expensive, and CPU instructions so fast, that memory locality is king > and the cost of bit masking is insignificant. In other words, worrying > about the cost of bit masking in C code is so very 1990s. > > I could be wrong of course: I'm not an expert. And I don't think we > should take it for granted that this is the case, unless some experts on > modern CPUs speak up and say that Mark is so obviously correct that a > demonstration is unnecessary.
And the speculation continues.
> > > You: "Maybe we limit it somewhere else, whatever. It'll be faster." > > That's a totally unfair and inaccurate representation of Mark's > position. The PEP doesn't say "let's put in arbitrary limits in random > places for the lols", he proposed seven concrete limits and gave reasons > for why he expects that they will improve memory efficiency, safety, > performance or all three.
*He expects that*. That's what I'm talking about. We have lots of speculation and no evidence either way.
> Having said that though, I think you are right that the PEP could do > with a bit more detail on the current status quo and existing limits, > and how the proposed changes will improve safety and memory use.
Exactly. Yes, I know that I massively oversimplified things in that post. But you nonetheless acknowledge here that we are *still* quite lacking in any actual evidence. We have people who believe that a bit mask will slow things down, others who claim that improved cache locality will speed things up, and Mark asks us to please justify our objections with numbers. But surely it's up to Mark to show numbers first?
ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org <mailto:python-dev@python.org> To unsubscribe send an email to python-dev-leave@python.org <mailto:python-dev-leave@python.org> https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YN2BDJFP... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EDZOEKHL... Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Dec 11, 2019 at 2:14 AM Mark Shannon <mark@hotpy.org> wrote:
If the status quo were the result of considered decision, then it would of course need considerable justification. If, as is the case here, the status quo is a result of historical accident and/or implementation details, then I think a weaker justification is OK.
Whoa. The lack of limits in the status quo (no limits on various things except indirectly, through available memory) is most definitely the result of an intentional decision. "No arbitrary limits" was part of Python's initial design philosophy. We didn't always succeed (parse tree depth and call recursion depth come to mind) but that was definitely the philosophy. It was in contrast to other languages that did have arbitrary limits (e.g. Pascal's 255-char limit on strings, or C's machine-dependent integer size) and in several cases the implementation went through great lengths to avoid limits (e.g. we could have avoided a lot of dynamic memory (re)allocation if we'd limited line lengths or file sizes). You have an extreme need to justify why we should change now. "An infinite number of potential optimizations" does not cut it. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
I've been restraining myself from commenting in this threads, as I don't know enough details of the implementation of Python to make reasoned arguments on how much such limits would help in the efficiency for running a Python program. I will way that in my many years of programming experience I can't think of any great cases where a language as part of the language definition limited to 'size' of a program to good effect, and generally such limits relegate a language into being seen as a 'toy language'. The biggest issue is that computers are growing more powerful every day, and programs follow in getting bigger, so any limit that we think of as more than sufficient soon becomes too small (No one will need more than 640k of RAM). I COULD easily see on the other hand, the language stating that some implementations might impose certain limits like those being mentioned, and maybe provide lower limits of those limits, that barring system resource limits a program can expect to be able to use. If a program needs higher limits, it can use a less limiting implementation. I could see a fork of the current CPython implementation happening, where one path keeps the current lack of limits, and another branch tries imposing the limits to see how much benefit you really get. After seeing, one of the forks might just die off as not being needed any more. I would see those limits being added to the specific implementation that provide identifiable advantage to making the system better. Being done, at least initially, in a fork, there might not need to be a lot of discussion on exactly what limits to try imposing, as 'Python' as a base language wouldn't be changing, just some limits in a particular branch implementation. it still might make a lot of sense for some discussion to occur to find out if the limits being discussed are reasonable. Perhaps the one big disadvantage of this idea is that the person wanting to see what these limits might be able to do for the language would need to be more involved in seeing it happen, as there won't be the easy opportunity to propose a somewhat vague idea, get it approved, and have someone else stuck with the job of getting it done. -- Richard Damon
On Wed, Dec 11, 2019 at 10:30:15PM -0500, Richard Damon wrote:
I will way that in my many years of programming experience I can't think of any great cases where a language as part of the language definition limited to 'size' of a program to good effect,
Good thing that's not what the PEP proposes then :-) The PEP talks about CPython implementation, not hard limits on all Python compilers/interpreters.
and generally such limits relegate a language into being seen as a 'toy language'.
The designers of C recognised that, in practice, compilers will have limits. Rather than demanding "NO ARBITRARY LIMITS!!!" they specified *minimum* levels for compliant compilers. Often quite low limits. Actual compilers often impose their own limits: https://gcc.gnu.org/onlinedocs/cpp/Implementation-limits.html https://www.cs.auckland.ac.nz/references/unix/digital/AQTLTBTE/DOCU_012.HTM So if Mark's proposal relegates Python to a "toy language", we'll be in good company with other "toys" that have implementation, or even language, limits: https://stackoverflow.com/questions/5689798/why-does-java-limit-the-size-of-... https://web.archive.org/web/20160304023522/http://programmers.stackexchange.... https://www.sqlite.org/limits.html (If you read only one of those five links, please read the last.)
The biggest issue is that computers are growing more powerful every day, and programs follow in getting bigger, so any limit that we think of as more than sufficient soon becomes too small (No one will need more than 640k of RAM).
The beauty of this proposal is that since its an implementation limit, not a law of nature, if and when computers get more powerful and machines routinely have multiple zettabyte memories *wink* we can always update the implementation. I'm not entirely being facetious here. There's a serious point. Unlike languages like C and Java, where changes have to go through a long, slow, difficult process, we have a much more agile process. If the PEP is accepted, that doesn't mean we're locked into that decision for life. Relaxing limits in the future doesn't break backwards compatibility. "What if computers get more powerful? Our limits will be obsolete!!!" Naturally. Do you still expect to be using Python 3.9 in ten years time with your fancy new uber-hyper-quantum ten thousand gigabyte computer? Probably not. As hardware grows, and our needs grow, so can the hypothetical limits. What is valuable are *concrete*, actual (not theoretical) examples of where Mark's proposed limits are too low, so that we can get a realistic view of where the potential tradeoffs lie: * lose this much functionality (code that did run, or might have run, but that won't run under the PEP) * in order to gain this much in safety, maintainability, efficiency. And before people jump down my throat again, I've already said -- on multiple occassions -- that the onus is on Mark to demonstrate the plausibility of any such gains. Thank you for reading :-) -- Steven
On 12/13/19 12:27 AM, Steven D'Aprano wrote:
On Wed, Dec 11, 2019 at 10:30:15PM -0500, Richard Damon wrote:
I will way that in my many years of programming experience I can't think of any great cases where a language as part of the language definition limited to 'size' of a program to good effect, Good thing that's not what the PEP proposes then :-)
The PEP talks about CPython implementation, not hard limits on all Python compilers/interpreters.
Not the way I read the PEP. It talks about changing the language. Under 'Enforcement' it admits it can't 'force' other implementations but says other implementations should generate the same errors unless doing so hurts performance, so it is a language change, not just limits for a given implementation. As I said originally, I have no problem with the idea of creating a variant implementation of CPython with this sort or documented limits to demonstrate that it does provide a real benefit. We would then be in a good place to determine the real costs and benefits, and one of the branches might just wither and die because it isn't useful enough anymore.
and generally such limits relegate a language into being seen as a 'toy language'. The designers of C recognised that, in practice, compilers will have limits. Rather than demanding "NO ARBITRARY LIMITS!!!" they specified *minimum* levels for compliant compilers. Often quite low limits.
Actual compilers often impose their own limits:
https://gcc.gnu.org/onlinedocs/cpp/Implementation-limits.html
https://www.cs.auckland.ac.nz/references/unix/digital/AQTLTBTE/DOCU_012.HTM
Yes, as I said above, there is a big definition between an implementation of a language documenting some of its limits and the language definition itself limiting what the language can do.
So if Mark's proposal relegates Python to a "toy language", we'll be in good company with other "toys" that have implementation, or even language, limits:
https://stackoverflow.com/questions/5689798/why-does-java-limit-the-size-of-...
https://web.archive.org/web/20160304023522/http://programmers.stackexchange.... Yes, Java made a decision early in its life cycle to lock itself into fixed sized types, and https://www.sqlite.org/limits.html These are a very different type of limits. These are defines that the
There is a big difference between limiting the size of a single method and limiting the total number of classes allowed in a program. The first can be largely gotten around by refactoring the method to be implemented in multiple methods, the latter can't be. programmer can change to establish what the various limits are. They can be increased or decreased as desired by the programmer (with natural upper limits based on the size of certain fundamental types).
(If you read only one of those five links, please read the last.)
The biggest issue is that computers are growing more powerful every day, and programs follow in getting bigger, so any limit that we think of as more than sufficient soon becomes too small (No one will need more than 640k of RAM). The beauty of this proposal is that since its an implementation limit, not a law of nature, if and when computers get more powerful and machines routinely have multiple zettabyte memories *wink* we can always update the implementation.
I'm not entirely being facetious here. There's a serious point. Unlike languages like C and Java, where changes have to go through a long, slow, difficult process, we have a much more agile process. If the PEP is accepted, that doesn't mean we're locked into that decision for life. Relaxing limits in the future doesn't break backwards compatibility.
"What if computers get more powerful? Our limits will be obsolete!!!" Naturally. Do you still expect to be using Python 3.9 in ten years time with your fancy new uber-hyper-quantum ten thousand gigabyte computer? Probably not. As hardware grows, and our needs grow, so can the hypothetical limits.
What is valuable are *concrete*, actual (not theoretical) examples of where Mark's proposed limits are too low, so that we can get a realistic view of where the potential tradeoffs lie:
* lose this much functionality (code that did run, or might have run, but that won't run under the PEP)
* in order to gain this much in safety, maintainability, efficiency.
And before people jump down my throat again, I've already said -- on multiple occassions -- that the onus is on Mark to demonstrate the plausibility of any such gains.
Thank you for reading :-)
As I said, the proposal as listed on Python.org is a language change, not a proposal for an implementation that has a set of limits. To do the former, really needs much more evidence to make it a reasonable course of action, the latter, if really done on an independent fork probably doesn't even really need a PEP, as because Python is open source, taking the existing code base, cloning it, and making the changes is something anyone is allowed to do. It just won't be the 'official C-Python distribution'. It does mean that the proposer needs to do (or find someone to do) the work. What might make sense through a PEP would be see if it would be reasonable to make a common code base variant, which might make sense if most of the code base can be untouched, and there are just isolated chunks of code that need to vary (structure definitions and limited setup code). My first feeling is that it is enough of a change, with unproven results, that it doesn't really make sense to commit to it for the main release without doing a test version to see what gains can be gotten, thus the idea of a 'second' version with the limits built in. Maybe part of the issue is many people think Python IS CPython and CPython is Python, and thus to propose an idea for changing CPython would seem to be a change in the base Python specification. I would also suggest that the presence of the 'Reference Implementation' section in the PEP format suggests that what is expected for things like this is that the proposer does fork the current implementation, makes at least a sample of the proposed changes and provides that as a demonstration of the advantages of the proposal. Perhaps one big disadvantage to how this PEP is written, is it presumes that we first add restrictions to the various limits in Python starting in version 3.9, but they are phased in, so we can't actually depend on them being present till 3.12 (several years later) so we are very limited in what performance we can gain for years. Making it a fork says we can apply the restrictions NOW, if a given program doesn't work under the limits, it just needs to be run under the unlimited version of Python, which will still be available. We also can make a direct apples to apples comparison of programs that do run under the improvements we can make from the limits. -- Richard Damon
On 11/12/2019 3:56 pm, Guido van Rossum wrote:
On Wed, Dec 11, 2019 at 2:14 AM Mark Shannon <mark@hotpy.org <mailto:mark@hotpy.org>> wrote:
If the status quo were the result of considered decision, then it would of course need considerable justification. If, as is the case here, the status quo is a result of historical accident and/or implementation details, then I think a weaker justification is OK.
Whoa. The lack of limits in the status quo (no limits on various things except indirectly, through available memory) is most definitely the result of an intentional decision. "No arbitrary limits" was part of Python's initial design philosophy. We didn't always succeed (parse tree depth and call recursion depth come to mind) but that was definitely the philosophy. It was in contrast to other languages that did have arbitrary limits (e.g. Pascal's 255-char limit on strings, or C's machine-dependent integer size) and in several cases the implementation went through great lengths to avoid limits (e.g. we could have avoided a lot of dynamic memory (re)allocation if we'd limited line lengths or file sizes).
Sure, there are few *designed* in limits in Python at the moment. But in reality Python has lots of limits. Because CPython is the reference implementation, Python is what CPython does; and CPython has lots of limits. Classes. -------- One suggested way, by which a million classes might be created was using namedtuple:
from collections import namedtuple l = [ namedtuple(f"nt{i}", "a,b,c,d,e,f,g,h,i,j") for i in range(1000_000) ]
This takes 6Gb of resident memory on my machine (3.9alpha). Most machines have roughly 4Gb per core (if you buy a bigger machine you get more CPUs and more RAM, roughly in that proportion). So RAM effectively limits the number of classes to less than one million already (assuming you want to use your cores efficiently). Imposing a limit on the number of classes and reducing memory footprint would allow more classes in practice, and would allow a lot more objects in more sensible programs. Instructions per code-object ---------------------------- CPython will crash if this is in the 2**31 to 2**32 range as the compiler treats addresses as unsigned, but the interpreter treats them as signed. Obviously this can be fixed, but it is an example of the sort of lurking bug that implicit limits can cause. These things are very expensive to test as you need a machine with hundreds of gigabytes of memory. Explicit limits are much easier to test. Does code outside the limit fail in the expected fashion and code just under the limit work correctly? What I want, is to allow more efficient use of resources without inconveniently low or unspecified limits. There will always be some limits on finite machines. If they aren't specified, they still exist, we just don't know what they are or how they will manifest themselves. Cheers, Mark.
You have an extreme need to justify why we should change now. "An infinite number of potential optimizations" does not cut it. -- --Guido van Rossum (python.org/~guido <http://python.org/~guido>) /Pronouns: he/him //(why is my pronoun here?)/ <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Thu., 12 Dec. 2019, 10:03 pm Mark Shannon, <mark@hotpy.org> wrote:
Explicit limits are much easier to test. Does code outside the limit fail in the expected fashion and code just under the limit work correctly?
What I want, is to allow more efficient use of resources without inconveniently low or unspecified limits. There will always be some limits on finite machines. If they aren't specified, they still exist, we just don't know what they are or how they will manifest themselves.
This is the main reason I'd like to see the PEP split into two distinct aspects: * articulating & testing specific limits (we already have some bigmem tests that can only be run at full scale on server class machines with 20+ GiB of RAM - don't forget that CPython runs on supercomputers, not just regular machines). * potentially lowering some specific limits in the CPython implementation The first part is clearly valuable as a general capability, especially if there's an API that lets different implementations report different limits. For example, something like a "sys.implementation.limits" informational API, modelled on the way that the top level "sys.implementation" data structure already works. Even when the limits are large, we have potential access to machines that can accommodate them for testing purposes (e.g. I know at least one Linux vendor that we're on good terms with that has servers with more than a terabyte of RAM in their test fleet, and I presume other OS vendors and cloud platform providers have similar capabilities). By contrast, the second part really needs to be tackled on a case-by-case basis, with a clear understanding of the kinds of failure modes we're aiming to handle more gracefully, which kinds of performance improvements we're aiming to enable, and which existing capabilities we'd be reducing. Cheers, Nick.
On 10/12/2019 00:01, Chris Angelico wrote:
Exactly. Yes, I know that I massively oversimplified things in that post. But you nonetheless acknowledge here that we are*still* quite lacking in any actual evidence. We have people who believe that a bit mask will slow things down, others who claim that improved cache locality will speed things up, and Mark asks us to please justify our objections with numbers. But surely it's up to Mark to show numbers first?
+1 -- Rhodri James *-* Kynesim Ltd
I think a much more sensible approach than mandating a limit because "who knows, it might speed something up" would be finding the speedup first. Probably that means one limit at a time too. E.g. maybe some patch imposes the 1 million LOC limit and demonstrates a repeatable benchmark improvement because of some coffee change that allows. That cold be interesting. Even them, I wouldn't want some arbitrary round number just for its own sake. For example, if taking 10 bits away from a word that holds a LOC index speeds something up, make the LOC limit 4,194,304 (2**22)... Maybe. If you only need 9 bits for that use, make the limit twice as much. On Mon, Dec 9, 2019, 12:53 PM Mark Shannon <mark@hotpy.org> wrote:
On 09/12/2019 2:15 pm, Chris Angelico wrote:
On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon <mark@hotpy.org> wrote:
Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few.
Can we get some stats on what the costs of higher limits (or having no limit at all) is? Or, putting it the other way: Since CPython currently does not enforce limits, what are the benefits of placing those limits? Merely having a limit doesn't in itself give a benefit (and is a (minor) cost); it would help the discussion if we knew exactly WHAT the costs of the higher limits were.
Given there is an infinite number of potential optimizations that it would enable, it is a bit hard to put a number on it :)
It is also impossible to put precise numbers on the speedups of a particular optimizations unless it is implemented. I suspect no one is going to do that unless paid to do so, or are guaranteed that the work won't be thrown away because the PEP is rejected.
Cheers, Mark.
ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/4IKBEMNG...
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BWTOEHKY... Code of Conduct: http://python.org/psf/codeofconduct/
Mark wrote, in reply to me:
On 09/12/2019 3:01 pm, Paddy McCarthy wrote:
"Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few."
Is there some evidence for the above statement? One of the issues with C programming is the need for limits Its hard to choose the right limits leading to so many failures. You propose limiting Python for performance gains but loose in ease of use and security.
How does this make anything less secure?
Many basic exploits have components taking advantage of limits known as buffer overflows Suppose we can improve the interpreter in such a way that it takes
advantage of the one million bytecode per code object and speeds it up by 0.1% (I can do a *lot* better than that). What would the performance gain save globally? $100k, $1M p.a.? What does it cost? A few developer hours.
You are looking to unproven execution gains. Adding an arbitrary limit might make it 0.1% faster to get the *wrong *result . Optimisation isn't just speedThe less mind-clutter spent on such* micro optimisations* might leave one able to think of a better algorithm or a better data structure that could give thousands of % speedups. You should only quote C or C++ compiler limitations as a warning of what *not *to do. Python wins by being different; by being more orthoganal; by having *less *arbitrary restrictions. Case in point: http://paddy3118.blogspot.com/2019/11/quasi-random-sobol-sequence.html Itook decade old C++ library, converted it to straight python, Python allowed me to see the opportunity for improvement leading to a Python version that is half the speed of the C++ code and generates points on demand. That's how Python wins big time. Having to worry about stifling limits is the wrong direction for the language to take. On Mon, 9 Dec 2019 at 21:26, David Mertz <mertz@gnosis.cx> wrote:
I think a much more sensible approach than mandating a limit because "who knows, it might speed something up" would be finding the speedup first.
Probably that means one limit at a time too. E.g. maybe some patch imposes the 1 million LOC limit and demonstrates a repeatable benchmark improvement because of some coffee change that allows. That cold be interesting.
Even them, I wouldn't want some arbitrary round number just for its own sake. For example, if taking 10 bits away from a word that holds a LOC index speeds something up, make the LOC limit 4,194,304 (2**22)... Maybe. If you only need 9 bits for that use, make the limit twice as much.
On Mon, Dec 9, 2019, 12:53 PM Mark Shannon <mark@hotpy.org> wrote:
On 09/12/2019 2:15 pm, Chris Angelico wrote:
On Tue, Dec 10, 2019 at 1:09 AM Mark Shannon <mark@hotpy.org> wrote:
Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few.
Can we get some stats on what the costs of higher limits (or having no limit at all) is? Or, putting it the other way: Since CPython currently does not enforce limits, what are the benefits of placing those limits? Merely having a limit doesn't in itself give a benefit (and is a (minor) cost); it would help the discussion if we knew exactly WHAT the costs of the higher limits were.
Given there is an infinite number of potential optimizations that it would enable, it is a bit hard to put a number on it :)
It is also impossible to put precise numbers on the speedups of a particular optimizations unless it is implemented. I suspect no one is going to do that unless paid to do so, or are guaranteed that the work won't be thrown away because the PEP is rejected.
Cheers, Mark.
ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/4IKBEMNG...
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BWTOEHKY... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MAOXN66J... Code of Conduct: http://python.org/psf/codeofconduct/
I have not seen any benchmarks supporting the claim that proposed limits would ACTUALLY have any positive performance effect. While some of the claims of performance gain are *plausible*, I also would not be very surprised if some of them caused performance loss. For example, requiring bit masking to get information from words feels like it probably costs more than using extra words in the first place. On Mon, Dec 9, 2019, 9:08 AM Mark Shannon <mark@hotpy.org> wrote:
Hi everyone,
Thanks again for all your comments on PEP 611.
I would like to ask a favour; please be more specific in your comments.
Ideally state which part of the PEP you are disagreeing with and why you disagree with the relevant part of the rationale/motivation.
Also, when asking for limits to be raised or removed entirely, could you state what you perceive to be the costs and benefits of larger limits. What do you believe is an acceptable cost in memory or runtime for larger limits?
For example, you might say that the limit of one million lines of code per module is too small, and that it is worth a small, say 1%, impact on speed to allow a larger of limit of 100 million.
If you believe a limit would have no cost, then please give a explanation of why that is so.
Merely saying that you would like a larger limit is pointless. If there were no cost to arbitrarily large limits, then I wouldn't have proposed the PEP in the first place.
Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few.
Cheers, Mark. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZXDAJKRV... Code of Conduct: http://python.org/psf/codeofconduct/
On 09/12/2019 14:05, Mark Shannon wrote:
I would like to ask a favour; please be more specific in your comments.
Ideally state which part of the PEP you are disagreeing with and why you disagree with the relevant part of the rationale/motivation.
This is rather hard to do when I don't think you have made the case for limits yet. Certainly not convincingly. Without that argument, all I can really do is say that I think you're wrong. -- Rhodri James *-* Kynesim Ltd
My overall problem with the PEP and a reason I'd reject it by default it that it is asking to pre-emptively impose limits on things, some of which we believe would cause problems to existing long running applications (limiting the total number of types for example), without having actually demonstrated practical benefits from such a change. Having an implementation that takes limits and runs with them to practical effect would provide motivation to consider adopting them. Otherwise it doesn't feel like it solves a real problem, and could cause some existing users pain. So what's our real motivation? picking on some nits within the PEP 611 text:
Motivation: "poorly generated code could cause values to exceed 2^32"
Malicious or poorly generated code can always do anything so this isn't a strong motivation. If you want a nice failure when we come near actual existing physical limits, that could be done today as a PR without a PEP. If correctness is the motivation, we could be correct without changing the existing unchecked limits.
: "there are two ways to use a 32 bit refcount on a 64 bit machine. One is to limit each sub-interpreter to 32Gb of memory. The other is to use a saturating reference count, which would be a little bit slower, but allow unlimited memory allocation."
Please do not arbitrarily cap sub-interpreter memory usage to a small value. 32GiB is very small today. Also, at least one existing eternal refcount implementation I've had experience with demonstrated a notable hit to interpreter cpu performance as it required an additional test+branch within the extremely widely used Py_INCREF and Py_DECREF macro code. -gps On Mon, Dec 9, 2019 at 6:10 AM Mark Shannon <mark@hotpy.org> wrote:
Hi everyone,
Thanks again for all your comments on PEP 611.
I would like to ask a favour; please be more specific in your comments.
Ideally state which part of the PEP you are disagreeing with and why you disagree with the relevant part of the rationale/motivation.
Also, when asking for limits to be raised or removed entirely, could you state what you perceive to be the costs and benefits of larger limits. What do you believe is an acceptable cost in memory or runtime for larger limits?
For example, you might say that the limit of one million lines of code per module is too small, and that it is worth a small, say 1%, impact on speed to allow a larger of limit of 100 million.
If you believe a limit would have no cost, then please give a explanation of why that is so.
Merely saying that you would like a larger limit is pointless. If there were no cost to arbitrarily large limits, then I wouldn't have proposed the PEP in the first place.
Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few.
Cheers, Mark. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZXDAJKRV... Code of Conduct: http://python.org/psf/codeofconduct/
On 09/12/2019 10:29 pm, Gregory P. Smith wrote:
My overall problem with the PEP and a reason I'd reject it by default it that it is asking to pre-emptively impose limits on things, some of which we believe would cause problems to existing long running applications (limiting the total number of types for example), without having actually demonstrated practical benefits from such a change. Having an implementation that takes limits and runs with them to practical effect would provide motivation to consider adopting them.
Otherwise it doesn't feel like it solves a real problem, and could cause some existing users pain. So what's our real motivation?
picking on some nits within the PEP 611 text:
Motivation: "poorly generated code could cause values to exceed 2^32"
Malicious or poorly generated code can always do anything so this isn't a strong motivation. If you want a nice failure when we come near actual existing physical limits, that could be done today as a PR without a PEP.
If correctness is the motivation, we could be correct without changing the existing unchecked limits.
: "there are two ways to use a 32 bit refcount on a 64 bit machine. One is to limit each sub-interpreter to 32Gb of memory. The other is to use a saturating reference count, which would be a little bit slower, but allow unlimited memory allocation."
Please do not arbitrarily cap sub-interpreter memory usage to a small value. 32GiB is very small today.
I wasn't planning on doing so, merely pointing out that there is a very simple way to make a 32 bit reference count correct. Limiting the number of objects or using a saturating count would be more sensible.
Also, at least one existing eternal refcount implementation I've had experience with demonstrated a notable hit to interpreter cpu performance as it required an additional test+branch within the extremely widely used Py_INCREF and Py_DECREF macro code.
Did they also reduce the size of object header, improving cache locality? Do you have a reference?
-gps
On Mon, Dec 9, 2019 at 6:10 AM Mark Shannon <mark@hotpy.org <mailto:mark@hotpy.org>> wrote:
Hi everyone,
Thanks again for all your comments on PEP 611.
I would like to ask a favour; please be more specific in your comments.
Ideally state which part of the PEP you are disagreeing with and why you disagree with the relevant part of the rationale/motivation.
Also, when asking for limits to be raised or removed entirely, could you state what you perceive to be the costs and benefits of larger limits. What do you believe is an acceptable cost in memory or runtime for larger limits?
For example, you might say that the limit of one million lines of code per module is too small, and that it is worth a small, say 1%, impact on speed to allow a larger of limit of 100 million.
If you believe a limit would have no cost, then please give a explanation of why that is so.
Merely saying that you would like a larger limit is pointless. If there were no cost to arbitrarily large limits, then I wouldn't have proposed the PEP in the first place.
Bear in mind that the costs of higher limits are paid by everyone, but the benefits are gained by few.
Cheers, Mark. _______________________________________________ Python-Dev mailing list -- python-dev@python.org <mailto:python-dev@python.org> To unsubscribe send an email to python-dev-leave@python.org <mailto:python-dev-leave@python.org> https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZXDAJKRV... Code of Conduct: http://python.org/psf/codeofconduct/
participants (13)
-
Chris Angelico -
David Mertz -
Gregory P. Smith -
Guido van Rossum -
Joao S. O. Bueno -
Kyle Stanley -
Mark Shannon -
Nathaniel Smith -
Nick Coghlan -
Paddy McCarthy -
Rhodri James -
Richard Damon -
Steven D'Aprano