Re: [Python-ideas] Shuffled
And if random.shuffle() returned a new list, other people would be bitten because they expected it to be in-place.
No one proposes that. shuffled() should be a new function.
One moderately stong piece of evidence would be if this function is widely available in third-party libraries and other languages.
Wrong. Python is NOT known for doing everything by how it was done before. My father used statically typed variables and his father used statically typed variables so I will use...
If so, then I would suggest than each list provides a .shuffle method as well (just as sorted/sort does).
Well, that could be problematic because std lib would have to use a random generator. We should leave things that require a source of randomness somewhere in the random module. Would you agree on that? Randomized unit testing is obvious example. Besides, any argument that can be used for having shuffle() could be also made for adding shuffled(). Randomizing lists has been used for a long time. We only need an non mutating analog. I am willing to make a patch. pozdrawiam, Arkadiusz Bulski
On Tue, Sep 06, 2016 at 05:29:18PM +0200, Arek Bulski wrote: [I think Arek is quoting me here]
One moderately stong piece of evidence would be if this function is widely available in third-party libraries and other languages.
Wrong. Python is NOT known for doing everything by how it was done before.
That's not what I said. I'm telling you what you need to do to have the best chance of your proposal being accepted. It isn't enough to just write a patch. Python is not *your* language where you get to decide what goes in and out, it is Guido's language, and he trusts the community to come to a consensus for these things. Part of that process is to trust respected core developers like Raymond Hettinger and Tim Peters to make decisions. I have been following the Python-Ideas mailing list for, oh, probably ten years. Maybe fifteen. I've seen hundreds, maybe thousands of suggestions and ideas rejected, and dozens accepted, including some of my own. If your only argument is to continue to insist that Python should have a shuffled() function and you'll write a patch, it will go no where. First you have to convince people that the patch is needed.
My father used statically typed variables and his father used statically typed variables so I will use...
And my grandfather and my father used dynamically typed variables. Lisp had dynamically typed variables back in the 1950s, and Smalltalk had them in the 1980s. So what? -- Steve
Steven D'Aprano writes:
If your only argument is to continue to insist that Python should have a shuffled() function and you'll write a patch, it will go no where. First you have to convince people that the patch is needed.
It's not worth the effort. The previous attempt (issue26393) to get a shuffled function had a patch (which reduces to turning the expression "random.Random.sample(x, len(x))" into a function), and it was summarily rejected then, too. By different people, not quite evidence of a consensus, but put it this way: you're spitting into a gale. That patch gets 95%, btw ... it had docs, tests, and a NEWS entry first try. I'd want a few stylistic changes, but altogether a copacetic patch. Even so, no mercy was shown.
Hi Steven, On 06.09.2016 18:32, Steven D'Aprano wrote:
On Tue, Sep 06, 2016 at 05:29:18PM +0200, Arek Bulski wrote:
[I think Arek is quoting me here]
One moderately stong piece of evidence would be if this function is widely available in third-party libraries and other languages. Wrong. Python is NOT known for doing everything by how it was done before. That's not what I said.
I'm telling you what you need to do to have the best chance of your proposal being accepted. It isn't enough to just write a patch.
Python is not *your* language where you get to decide what goes in and out, it is Guido's language, and he trusts the community to come to a consensus for these things. Part of that process is to trust respected core developers like Raymond Hettinger and Tim Peters to make decisions.
I have been following the Python-Ideas mailing list for, oh, probably ten years. Maybe fifteen. I've seen hundreds, maybe thousands of suggestions and ideas rejected, and dozens accepted, including some of my own.
If your only argument is to continue to insist that Python should have a shuffled() function and you'll write a patch, it will go no where. First you have to convince people that the patch is needed.
He already convinced some people. Just not some venerable Python devs, which doesn't necessarily mean something at all. Their response: "Oh, I don't need it, let's close it." Arek: "But I need it." So, who's right now? I would suggest not being so protective. It's community project after all. Maybe, the patch will help him understand that it's not so easy, or other people will see the benefit after reading it. If the patch isn't worth including, he learned something in the process of creating it.
My father used statically typed variables and his father used statically typed variables so I will use... And my grandfather and my father used dynamically typed variables. Lisp had dynamically typed variables back in the 1950s, and Smalltalk had them in the 1980s. So what?
Best, Sven
Naive shuffled() can be emulated using a single expression: sorted(lst, key=lambda _: random()) So there's even less incentive for standardization. ~Elazar
Are you serious? The reason for "we don't need standardization" == "there is a solution, long, and with a lot of special characters". Holy c***. I remember Python being the least verbose language on the planet. I suspect this argument didn't lead to status quo. It's like saying, we don't need variable annotations because we have comments. On 06.09.2016 20:25, אלעזר wrote:
Naive shuffled() can be emulated using a single expression:
sorted(lst, key=lambda _: random())
So there's even less incentive for standardization.
Besides being a silly argument, it's an interesting solution. Does it really work? I remember Microsoft utilizing a similar approach for their browser selection tool which led to a skewed probability distribution. Maybe, I wrong here though. Cheer, Sven
On 09/06/2016 03:37 PM, Sven R. Kunze wrote:
Besides being a silly argument, it's an interesting solution.
Does it really work? I remember Microsoft utilizing a similar approach for their browser selection tool which led to a skewed probability distribution. Maybe, I wrong here though.
Yes. The key is evaluated only once, so each element gets a pseudo-random number. Sorting this list leads to a shuffle. However, a super-linear shuffle, whilst Fisher-Yates is a linear solution and also requires less additional memory. Lastly, although it is obvious, from a software engineering standpoint, this is a mere weird hack.
(Just to be clear, I wasn't trying to suggest this as more than an ad-hoc solution for a throwaway script. But to me, "sorted by random key" is almost as obvious as "shuffled", perhaps more so for non english speakers with little background in CS terms; the words "sorted" and "random" jumps to the eye, and sometimes you don't need more than that) ~Elazar בתאריך יום ג׳, 6 בספט' 2016, 21:48, מאת Bernardo Sulzbach < mafagafogigante@gmail.com>:
On 09/06/2016 03:37 PM, Sven R. Kunze wrote:
Besides being a silly argument, it's an interesting solution.
Does it really work? I remember Microsoft utilizing a similar approach for their browser selection tool which led to a skewed probability distribution. Maybe, I wrong here though.
Yes. The key is evaluated only once, so each element gets a pseudo-random number. Sorting this list leads to a shuffle.
However, a super-linear shuffle, whilst Fisher-Yates is a linear solution and also requires less additional memory.
Lastly, although it is obvious, from a software engineering standpoint, this is a mere weird hack.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 06.09.2016 20:46, Bernardo Sulzbach wrote:
On 09/06/2016 03:37 PM, Sven R. Kunze wrote:
Besides being a silly argument, it's an interesting solution.
Does it really work? I remember Microsoft utilizing a similar approach for their browser selection tool which led to a skewed probability distribution. Maybe, I wrong here though.
Yes. The key is evaluated only once, so each element gets a pseudo-random number. Sorting this list leads to a shuffle.
Ah yes, that might make it work.
However, a super-linear shuffle, whilst Fisher-Yates is a linear solution and also requires less additional memory.
Lastly, although it is obvious, from a software engineering standpoint, this is a mere weird hack.
Sure, but it works. ;) Though I agree that's not really intuitive if you are looking for "just shuffle that list, please". Cheers, Sven PS: here's the statistical analysis of the browser ballot issue with some explanations on the different algorithms on shuffling: http://www.robweir.com/blog/2010/02/microsoft-random-browser-ballot.html
[Sven R. Kunze <srkunze@mail.de>]
... He already convinced some people. Just not some venerable Python devs, which doesn't necessarily mean something at all.
Their response: "Oh, I don't need it, let's close it." Arek: "But I need it."
So, who's right now?
By default, the venerable Python devs ;-) As I said in my comments on the issue report, I searched through all my code for instances of `shuffle()`, and didn't find any that could be improved by `shuffled()`. Nor could I dream up a compelling use case. So it wasn't just a knee-jerk "oh, I don't need it", the "I don't need it" was the outcome of some research and thought. In contrast, the original request was a bare "But I need it", without so much as a single concrete use case to justify it ("testing" is an application area, not a concrete use case - I've made extensive use of shuffling in testing too, but - as I said - found no existing code where `shuffled()` would have helped). In the absence of compelling concrete use cases, new ideas have approximately no chance of being adopted. Otherwise the language & libraries bloat in counterproductive ways. Just ask any venerable Python dev ;-)
On 06.09.2016 20:46, Tim Peters wrote:
[Sven R. Kunze <srkunze@mail.de>]
... He already convinced some people. Just not some venerable Python devs, which doesn't necessarily mean something at all.
Their response: "Oh, I don't need it, let's close it." Arek: "But I need it."
So, who's right now? By default, the venerable Python devs ;-)
Well, little bit arrogant? I hope technical arguments weigh more than status here. ;)
[I don't need it in my source code which is under version control.]
It's okay. You don't need to reiterate that you don't need it. I also don't need it when looking at my checked-in source code (but it seems there are people who do). But I remember using it in interactive sessions (damn, why didn't I check that into git?). And there I remember finding it quite irritating not having a return value at all.
random.shuffle([1,2,3,4])
Just gives me: nothing. Of course, the period of irritation lasted short and reading __doc__ helped a lot, but, well, I could have saved me some time. You get the idea. Cheers, Sven
[Sven R. Kunze <srkunze@mail.de>]
... He already convinced some people. Just not some venerable Python devs, which>> doesn't necessarily mean something at all.
Their response: "Oh, I don't need it, let's close it." Arek: "But I need it."
So, who's right now?
[Tim]
By default, the venerable Python devs ;-)
[Sven]
Well, little bit arrogant? I hope technical arguments weigh more than status here. ;)
I haven't seen any technical arguments here. You snipped the parts of my reply where I emphasized the need for concrete use cases. In the absence of anything approaching that, it's a matter of taste, and then - yes - decades of Python development experience do - and should - outweigh a newcomer's wish list. Especially when, as appears to be the case here, those with truly extensive experience agree.
... But I remember using it in interactive sessions (damn, why didn't I check that into git?). And there I remember finding it quite irritating not having a return value at all.
random.shuffle([1,2,3,4])
Just gives me: nothing.
Of course, the period of irritation lasted short and reading __doc__ helped a lot, but, well, I could have saved me some time. You get the idea.
No, I don't. It's an obvious feature of Python's list object design that mutating methods always return None. Which means "obvious" after someone has learned the language, not obvious the first time they try it. StackOverflow is equally full of newbie complaints that, e.g., some_list.sort().append(100) gives an "incomprehensible" AttributeError: 'NoneType' object has no attribute 'append' error. Once "ah, mutating methods generally return None" sinks in, they never post any complaint like that again. I would be far more annoyed if, e.g.,
random.shuffle(some_million_element_list)
swamped my terminal with mountains of output. You can't both behaviors simultaneously, so the status quo wins. Indeed, the venerable status quo ;-)
On 06.09.2016 21:29, Tim Peters wrote:
In the absence of anything approaching [use-cases], it's a matter of taste, and then - yes - decades of Python development experience do - and should - outweigh a newcomer's wish list. Especially when, as appears to be the case here, those with truly extensive experience agree.
I am not questioning experience which everyone in a team can benefit from. BUT experienced devs also need to recognize and respect the fact that younger/unexperienced developers are just better in detecting inconsistencies and bloody work-arounds. They simply haven't had to live with them for so long. Experienced devs just are stuck in a rut/are routine-blinded: "we've done that for years", "there's no better way". That's the way we do it in our teams. Employing the new guys as some sort of inconsistency detectors. This way, they learn to find their way around the code base and they can improve it by doing so. And I would never allow it in my team, to dismiss this kind of observation from new colleagues. It's invaluable as they will become routine-blinded as well.
[...] I would be far more annoyed if, e.g.,
random.shuffle(some_million_element_list) swamped my terminal with mountains of output.
But you readily accept this behavior for "sorted"? That makes no sense at all.
You can't both behaviors simultaneously, so the status quo wins. Indeed, the venerable status quo ;-)
Nobody said to change "shuffle". Sven
[Sven R. Kunze <srkunze@mail.de>]
I am not questioning experience which everyone in a team can benefit from.
BUT experienced devs also need to recognize and respect the fact that younger/unexperienced developers are just better in detecting inconsistencies and bloody work-arounds. They simply haven't had to live with them for so long. Experienced devs just are stuck in a rut/are routine-blinded: "we've done that for years", "there's no better way".
That's the way we do it in our teams. Employing the new guys as some sort of inconsistency detectors. This way, they learn to find their way around the code base and they can improve it by doing so.
And I would never allow it in my team, to dismiss this kind of observation from new colleagues. It's invaluable as they will become routine-blinded as well.
I have been more than willing to discuss it, and I did not close the issue report. I did say I was opposed to it, but that's simply because I am, and I explained there too _why_ I was opposed. Do you have anything to say about the specific proposal? I doubt either of us has found this meta-discussion useful. I'm still looking for a compelling use case. The only concrete thing anyone has noted in `shuffled()`'s favor so far is that sometimes they're surprised by the behavior of random.shuffle(list) returning None in an interactive shell (noted by you, and by another, and I cheerfully own up to being a bit surprised by that too long ago). But that's an observation about `random.shuffle()`, not about the proposed `shuffled()`.
[...] I would be far more annoyed if, e.g.,
random.shuffle(some_million_element_list)
swamped my terminal with mountains of output.
But you readily accept this behavior for "sorted"? That makes no sense at all.
Of course it does. The only analogy to random.shuffle(big_list) returning None that makes a lick of sense here is that big_list.sort() also returns None. IF a `shuffled()` function is introduced, then of course it should return its result - just like `sorted()` returns its result.
You can't both behaviors simultaneously, so the status quo wins. Indeed, the venerable status quo ;-)
Nobody said to change "shuffle".
A verbatim quote from the first message in this thread: "Also shuffle() should return self so mutating methods could be chained."
Maybe, there's a misunderstanding here and I hope I didn't waste too much of your and my time. Not sure where I got this from but my last status was that Arek would like to have a "shuffled" function that does exactly what you described. Maybe, that doesn't fit the reports description but his pull request will tell the truth. :) About the confusion of returning None. It's not confusing in the sense of "o my god, I did it wrong, I need to learn it", but more, like "I used shuffle, BECAUSE it's the only one I could find in a hurry" and then ascertain "why the heck is there no alternative returning the shuffled result instead of overwriting my list? I would expect this from Python as it already provides both alternatives for sorting". Sven On 08.09.2016 00:02, Tim Peters wrote:
[Sven R. Kunze <srkunze@mail.de>]
I am not questioning experience which everyone in a team can benefit from.
BUT experienced devs also need to recognize and respect the fact that younger/unexperienced developers are just better in detecting inconsistencies and bloody work-arounds. They simply haven't had to live with them for so long. Experienced devs just are stuck in a rut/are routine-blinded: "we've done that for years", "there's no better way".
That's the way we do it in our teams. Employing the new guys as some sort of inconsistency detectors. This way, they learn to find their way around the code base and they can improve it by doing so.
And I would never allow it in my team, to dismiss this kind of observation from new colleagues. It's invaluable as they will become routine-blinded as well. I have been more than willing to discuss it, and I did not close the issue report. I did say I was opposed to it, but that's simply because I am, and I explained there too _why_ I was opposed.
Do you have anything to say about the specific proposal? I doubt either of us has found this meta-discussion useful. I'm still looking for a compelling use case. The only concrete thing anyone has noted in `shuffled()`'s favor so far is that sometimes they're surprised by the behavior of random.shuffle(list) returning None in an interactive shell (noted by you, and by another, and I cheerfully own up to being a bit surprised by that too long ago). But that's an observation about `random.shuffle()`, not about the proposed `shuffled()`.
[...] I would be far more annoyed if, e.g.,
random.shuffle(some_million_element_list) swamped my terminal with mountains of output. But you readily accept this behavior for "sorted"? That makes no sense at all. Of course it does. The only analogy to random.shuffle(big_list) returning None that makes a lick of sense here is that big_list.sort() also returns None. IF a `shuffled()` function is introduced, then of course it should return its result - just like `sorted()` returns its result.
You can't both behaviors simultaneously, so the status quo wins. Indeed, the venerable status quo ;-) Nobody said to change "shuffle". A verbatim quote from the first message in this thread:
"Also shuffle() should return self so mutating methods could be chained."
I lurk around here a lot more than I actually post -- But in this case I find myself weighing in firmly on the side of "please don't add this". It'd just add noise in the documentation and __builtins__ namespace. There are millions of useful functions that *could* be added to the `list` object (or to the `__builtins__` namespace). It's the job of the core dev team to decide which are the most useful for everybody and this is one where (in my opinion) the usefulness doesn't justify the additional clutter. It seems like the primary argument for this is that some users would like to have this functionality baked into an interactive session. If that's the case, then I'd like to point out that if you find yourself wanting this in an interactive session frequently there is an interactive startup file <https://docs.python.org/3/tutorial/appendix.html#the-interactive-startup-fil...> that you can modify so that this function will always be available to *you* whenever you start up an interactive session (and you can put whatever else in there that you like as well :-). I hope that helps. On Wed, Sep 7, 2016 at 3:22 PM, Sven R. Kunze <srkunze@mail.de> wrote:
Maybe, there's a misunderstanding here and I hope I didn't waste too much of your and my time.
Not sure where I got this from but my last status was that Arek would like to have a "shuffled" function that does exactly what you described. Maybe, that doesn't fit the reports description but his pull request will tell the truth. :)
About the confusion of returning None. It's not confusing in the sense of "o my god, I did it wrong, I need to learn it", but more, like "I used shuffle, BECAUSE it's the only one I could find in a hurry" and then ascertain "why the heck is there no alternative returning the shuffled result instead of overwriting my list? I would expect this from Python as it already provides both alternatives for sorting".
Sven
On 08.09.2016 00:02, Tim Peters wrote:
[Sven R. Kunze <srkunze@mail.de>]
I am not questioning experience which everyone in a team can benefit from.
BUT experienced devs also need to recognize and respect the fact that younger/unexperienced developers are just better in detecting inconsistencies and bloody work-arounds. They simply haven't had to live with them for so long. Experienced devs just are stuck in a rut/are routine-blinded: "we've done that for years", "there's no better way".
That's the way we do it in our teams. Employing the new guys as some sort of inconsistency detectors. This way, they learn to find their way around the code base and they can improve it by doing so.
And I would never allow it in my team, to dismiss this kind of observation from new colleagues. It's invaluable as they will become routine-blinded as well.
I have been more than willing to discuss it, and I did not close the issue report. I did say I was opposed to it, but that's simply because I am, and I explained there too _why_ I was opposed.
Do you have anything to say about the specific proposal? I doubt either of us has found this meta-discussion useful. I'm still looking for a compelling use case. The only concrete thing anyone has noted in `shuffled()`'s favor so far is that sometimes they're surprised by the behavior of random.shuffle(list) returning None in an interactive shell (noted by you, and by another, and I cheerfully own up to being a bit surprised by that too long ago). But that's an observation about `random.shuffle()`, not about the proposed `shuffled()`.
[...] I would be far more annoyed if, e.g.,
random.shuffle(some_million_element_list)
> swamped my terminal with mountains of output.
But you readily accept this behavior for "sorted"? That makes no sense at all.
Of course it does. The only analogy to random.shuffle(big_list) returning None that makes a lick of sense here is that big_list.sort() also returns None. IF a `shuffled()` function is introduced, then of course it should return its result - just like `sorted()` returns its result.
You can't both behaviors simultaneously, so the status quo wins.
Indeed, the venerable status quo ;-)
Nobody said to change "shuffle".
A verbatim quote from the first message in this thread:
"Also shuffle() should return self so mutating methods could be chained."
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- [image: pattern-sig.png] Matt Gilson // SOFTWARE ENGINEER E: matt@getpattern.com // P: 603.892.7736 We’re looking for beta testers. Go here <https://www.getpattern.com/meetpattern> to sign up!
On 08.09.2016 00:41, Matt Gilson wrote:
I lurk around here a lot more than I actually post -- But in this case I find myself weighing in firmly on the side of "please don't add this". It'd just add noise in the documentation and __builtins__ namespace. There are millions of useful functions that /could/ be added to the `list` object (or to the `__builtins__` namespace). It's the job of the core dev team to decide which are the most useful for everybody and this is one where (in my opinion) the usefulness doesn't justify the additional clutter.
It seems like the primary argument for this is that some users would like to have this functionality baked into an interactive session.
Maybe, it wasn't obvious (despite I thought so), but I also don't see it in the built-ins. Another function in the "random" module would just suffice. But still, this is not my proposal, so I am gonna wait for what Arek brings up. Sven
If that's the case, then I'd like to point out that if you find yourself wanting this in an interactive session frequently there is an interactive startup file <https://docs.python.org/3/tutorial/appendix.html#the-interactive-startup-fil...> that you can modify so that this function will always be available to /you/ whenever you start up an interactive session (and you can put whatever else in there that you like as well :-).
I hope that helps.
On Wed, Sep 7, 2016 at 3:22 PM, Sven R. Kunze <srkunze@mail.de <mailto:srkunze@mail.de>> wrote:
Maybe, there's a misunderstanding here and I hope I didn't waste too much of your and my time.
Not sure where I got this from but my last status was that Arek would like to have a "shuffled" function that does exactly what you described. Maybe, that doesn't fit the reports description but his pull request will tell the truth. :)
About the confusion of returning None. It's not confusing in the sense of "o my god, I did it wrong, I need to learn it", but more, like "I used shuffle, BECAUSE it's the only one I could find in a hurry" and then ascertain "why the heck is there no alternative returning the shuffled result instead of overwriting my list? I would expect this from Python as it already provides both alternatives for sorting".
Sven
On 08.09.2016 00:02, Tim Peters wrote:
[Sven R. Kunze <srkunze@mail.de <mailto:srkunze@mail.de>>]
I am not questioning experience which everyone in a team can benefit from.
BUT experienced devs also need to recognize and respect the fact that younger/unexperienced developers are just better in detecting inconsistencies and bloody work-arounds. They simply haven't had to live with them for so long. Experienced devs just are stuck in a rut/are routine-blinded: "we've done that for years", "there's no better way".
That's the way we do it in our teams. Employing the new guys as some sort of inconsistency detectors. This way, they learn to find their way around the code base and they can improve it by doing so.
And I would never allow it in my team, to dismiss this kind of observation from new colleagues. It's invaluable as they will become routine-blinded as well.
I have been more than willing to discuss it, and I did not close the issue report. I did say I was opposed to it, but that's simply because I am, and I explained there too _why_ I was opposed.
Do you have anything to say about the specific proposal? I doubt either of us has found this meta-discussion useful. I'm still looking for a compelling use case. The only concrete thing anyone has noted in `shuffled()`'s favor so far is that sometimes they're surprised by the behavior of random.shuffle(list) returning None in an interactive shell (noted by you, and by another, and I cheerfully own up to being a bit surprised by that too long ago). But that's an observation about `random.shuffle()`, not about the proposed `shuffled()`.
[...] I would be far more annoyed if, e.g.,
random.shuffle(some_million_element_list)
swamped my terminal with mountains of output.
But you readily accept this behavior for "sorted"? That makes no sense at all.
Of course it does. The only analogy to random.shuffle(big_list) returning None that makes a lick of sense here is that big_list.sort() also returns None. IF a `shuffled()` function is introduced, then of course it should return its result - just like `sorted()` returns its result.
You can't both behaviors simultaneously, so the status quo wins. Indeed, the venerable status quo ;-)
Nobody said to change "shuffle".
A verbatim quote from the first message in this thread:
"Also shuffle() should return self so mutating methods could be chained."
_______________________________________________ Python-ideas mailing list Python-ideas@python.org <mailto:Python-ideas@python.org> https://mail.python.org/mailman/listinfo/python-ideas <https://mail.python.org/mailman/listinfo/python-ideas> Code of Conduct: http://python.org/psf/codeofconduct/ <http://python.org/psf/codeofconduct/>
--
pattern-sig.png
Matt Gilson// SOFTWARE ENGINEER
E: matt@getpattern.com <mailto:matt@getpattern.com>// P: 603.892.7736
We’re looking for beta testers. Go here <https://www.getpattern.com/meetpattern>to sign up!
On 07/09/2016 23:41, Matt Gilson wrote:
I lurk around here a lot more than I actually post Very wise. I am a tournament Bridge player, and one of the things I say to my partners is: "Bidding is very simple. When you have something to say, say it (unless it really seems too dangerous). When you have nothing to say, Pass". -- But in this case I find myself weighing in firmly on the side of "please don't add this". It'd just add noise in the documentation and __builtins__ namespace. There are millions of useful functions that /could/ be added to the `list` object (or to the `__builtins__` namespace). It's the job of the core dev team to decide which are the most useful for everybody and this is one where (in my opinion) the usefulness doesn't justify the additional clutter.
It seems like the primary argument for this is that some users would like to have this functionality baked into an interactive session. If that's the case, then I'd like to point out that if you find yourself wanting this in an interactive session frequently there is an interactive startup file <https://docs.python.org/3/tutorial/appendix.html#the-interactive-startup-fil...> that you can modify so that this function will always be available to /you/ whenever you start up an interactive session (and you can put whatever else in there that you like as well :-).
I hope that helps. +1 And incidentally, thank you for reminding me of this. I do actually use |PYTHONSTARTUP| (mostly to pre-import modules that I use frequently: sys, time, datetime, os, etc. - YMMV). But your reminder may well prod me into finding new uses. Rob Cliffe
On Thu, Sep 08, 2016 at 12:22:28AM +0200, Sven R. Kunze wrote:
About the confusion of returning None. It's not confusing in the sense of "o my god, I did it wrong, I need to learn it", but more, like "I used shuffle, BECAUSE it's the only one I could find in a hurry"
Then don't be in such a hurry. Honestly, there is no substitute for professionalism. Outside of fantasy novels and movies, there has never been a "ticking time bomb" situation where a programmer has to write code to shuffle a list in six seconds or the world will be destroyed. There is *always* time to at least glance at the docs, and it is never more than a handful of keypresses away in the interactive interpreter: help(random.shuffle) And if you don't read the docs? Oh well, you find out soon enough once you actually test your code or have a code review. No harm done. If the programmer doesn't do *any* of those things -- doesn't read the docs, doesn't test their code, doesn't do code reviews -- then there's no hope for them. We could provide shuffled() and shuffle() and they'll still end up picking the wrong one fifty percent of the time. "My wild guess of how a function works was wrong, so now I'm cheesed off that the function doesn't do what I expect" is NOT a good reason to add things to the standard library.
I would expect this from Python as it already provides both alternatives for sorting".
Does that mean you expect Python to provide appended() as well as append(), extended() as well as extend(), popped() as well as pop(), cleared() as well as clear(), inserted() as well as insert(), removed() as well as remove()? If your response is to say "Why do we need all those? Why on earth would anyone need to use list.cleared()?" then perhaps you can now understand why shuffled() has to prove itself too. Functions don't get added just because they *can* be added. They have to be worth it. Every function has costs as well as benefits. Each function adds: - more code to download, compile, maintain - more tests - more documentation - more for the user to learn - more choices for the user to decide between - more complexity - more risk of bugs Unless the proposal can demonstrate that the benefit outweighs the costs, it should not be added. Merely stating that you need it doesn't demonstate that a four-line helper function at the top of your module isn't sufficient. For what's it is worth, if it were *my* call, I'd accept that the costs of adding this are low, but the benefits are just a *tiny* bit higher. But that's a judgement call, and if Raymond see the cost:benefit ratio going the other way, I have no objective argument to change his mind. -- Steve
Though I agree with the argument that inexperienced developers are [usually] worse, that's not the case here, unless anyone here is really trying to say the ones arguing for or against "shuffled" are inexperienced. These ad hominem won't bring us anywhere. No one seem to be arguing if "shuffled" is inconsistent, but the "shuffle" consistency is a subject that didn't reach consensus. And the point now seem to be whether "shuffled" is useful or not. If I understood correctly, that proposal is about writing code in a more functional style (or a expression-oriented programming style), with a pure function for shuffling. I think there are other related subjects that can be said about sorted/shuffled. For example: why a list? Why not a generator? Something like random.choice is useful to get one single value, but if one just want to see a few random values from a large input without repeating the values, what's the best approach? Does that mean you expect Python to provide appended() [...] extended()
[...]
Doesn't the "+" operator do that?
[...] list.cleared()?
Huh? OP is clearly talking about avoiding state, avoiding "in-place" operations. A clear list would be an empty list []. The idea of "popped", "inserted" and "removed" are also simply off-topic. 2016-09-07 23:31 GMT-03:00 Steven D'Aprano <steve@pearwood.info>:
On Thu, Sep 08, 2016 at 12:22:28AM +0200, Sven R. Kunze wrote:
About the confusion of returning None. It's not confusing in the sense of "o my god, I did it wrong, I need to learn it", but more, like "I used shuffle, BECAUSE it's the only one I could find in a hurry"
Then don't be in such a hurry.
Honestly, there is no substitute for professionalism. Outside of fantasy novels and movies, there has never been a "ticking time bomb" situation where a programmer has to write code to shuffle a list in six seconds or the world will be destroyed. There is *always* time to at least glance at the docs, and it is never more than a handful of keypresses away in the interactive interpreter:
help(random.shuffle)
And if you don't read the docs? Oh well, you find out soon enough once you actually test your code or have a code review. No harm done.
If the programmer doesn't do *any* of those things -- doesn't read the docs, doesn't test their code, doesn't do code reviews -- then there's no hope for them. We could provide shuffled() and shuffle() and they'll still end up picking the wrong one fifty percent of the time.
"My wild guess of how a function works was wrong, so now I'm cheesed off that the function doesn't do what I expect" is NOT a good reason to add things to the standard library.
I would expect this from Python as it already provides both alternatives for sorting".
Does that mean you expect Python to provide appended() as well as append(), extended() as well as extend(), popped() as well as pop(), cleared() as well as clear(), inserted() as well as insert(), removed() as well as remove()?
If your response is to say "Why do we need all those? Why on earth would anyone need to use list.cleared()?" then perhaps you can now understand why shuffled() has to prove itself too. Functions don't get added just because they *can* be added. They have to be worth it. Every function has costs as well as benefits. Each function adds:
- more code to download, compile, maintain - more tests - more documentation - more for the user to learn - more choices for the user to decide between - more complexity - more risk of bugs
Unless the proposal can demonstrate that the benefit outweighs the costs, it should not be added. Merely stating that you need it doesn't demonstate that a four-line helper function at the top of your module isn't sufficient.
For what's it is worth, if it were *my* call, I'd accept that the costs of adding this are low, but the benefits are just a *tiny* bit higher. But that's a judgement call, and if Raymond see the cost:benefit ratio going the other way, I have no objective argument to change his mind.
-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Danilo J. S. Bellini --------------- "*It is not our business to set up prohibitions, but to arrive at conventions.*" (R. Carnap)
On Wed, Sep 07, 2016 at 11:47:49PM -0300, Danilo J. S. Bellini wrote:
Though I agree with the argument that inexperienced developers are [usually] worse, that's not the case here, unless anyone here is really trying to say the ones arguing for or against "shuffled" are inexperienced. These ad hominem won't bring us anywhere.
That is not what I said. Please be careful of accusing others of things they didn't say. It was Sven who made the claim that inexperienced developers are better at seeing inconsistancies. I was responding specifically to that claim. I'm not saying anything about the experience, or lack of, of either Sven or Arek.
No one seem to be arguing if "shuffled" is inconsistent, but the "shuffle" consistency is a subject that didn't reach consensus. And the point now seem to be whether "shuffled" is useful or not. If I understood correctly, that proposal is about writing code in a more functional style (or a expression-oriented programming style), with a pure function for shuffling.
I think there are other related subjects that can be said about sorted/shuffled. For example: why a list? Why not a generator?
You cannot sort a sequence lazily. You have to sort the entire sequence before you can be sure which item comes first. Making sorted() an iterator by default doesn't give you any advantage: it still has to sort the entire sequence before it can yield the first item.
Something like random.choice is useful to get one single value, but if one just want to see a few random values from a large input without repeating the values, what's the best approach?
random.sample()
Does that mean you expect Python to provide appended() [...] extended()
[...]
Doesn't the "+" operator do that?
Of course it does. But the "consistency" argument would be "why is append a method not an operator? Why isn't + called appended?". That's just spelling. It isn't that important.
[...] list.cleared()?
Huh? OP is clearly talking about avoiding state, avoiding "in-place" operations. A clear list would be an empty list [].
Of course. We can see that. But again, that's "inconsistent": * the in-place operation is spelled `list.clear()` * the functional operation is spelled `[]` There is nothing wrong with that. The value of consistency is grossly exaggerated. Sometimes it really doesn't matter if similar operations are spelled differently. -- Steve
On 8 September 2016 at 12:47, Danilo J. S. Bellini <danilo.bellini@gmail.com> wrote:
Though I agree with the argument that inexperienced developers are [usually] worse, that's not the case here, unless anyone here is really trying to say the ones arguing for or against "shuffled" are inexperienced. These ad hominem won't bring us anywhere.
No one seem to be arguing if "shuffled" is inconsistent, but the "shuffle" consistency is a subject that didn't reach consensus. And the point now seem to be whether "shuffled" is useful or not. If I understood correctly, that proposal is about writing code in a more functional style (or a expression-oriented programming style), with a pure function for shuffling.
A pure function for shuffling already exists - random.sample() So the question at hand boils down to whether it makes sense to provide: def shuffled(self, container): return self.sample(container, len(container)) as a method on Random objects, and equivalently as a top-level module function. I'd personally be in favour of that as a learning bridge between shuffling and sampling (shuffle in place -> shuffle out of place -> sample a subset rather than shuffling out of place and slicing), but I'm neither a professional educator nor a maintainer or heavy user of the module in question, so my preference in the matter should be weighted pretty low.
I think there are other related subjects that can be said about sorted/shuffled. For example: why a list? Why not a generator?
This is covered in the random.sample() docs - returning a list is useful, as you can then easily use slices and tuple unpacking to partition the result: winners = random.sample(participants, 10) first, second, *runners_up = winners Beyond that practical benefit, if you want random-sampling-with-replacement, then "map(random.choice, container)" already has you covered, while random-sampling-without-replacement inherently needs to maintain a set of already produced values so it can avoid repeating them, which makes the apparent memory efficiency of using a generator instead is somewhat illusory.
Something like random.choice is useful to get one single value, but if one just want to see a few random values from a large input without repeating the values, what's the best approach?
Use random.sample(), that's what it's for. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 8 September 2016 at 13:23, Nick Coghlan <ncoghlan@gmail.com> wrote:
Beyond that practical benefit, if you want random-sampling-with-replacement, then "map(random.choice, container)"
Oops, that was supposed to be "map(random.choice, itertools.repeat(container))". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nice to know about random.sample! =) I think what OP said can then be reduced to having the default k in random.sample to be the iterable size. The existance of random.sample is a very strong argument against "shuffled", and the only "feature" shuffled would have that random.sample doesn't have is that default size. You cannot sort a sequence lazily
You can, but it probably wouldn't be efficient if you need all the values. On the other hand, if you need just the 3 smaller values of a huge list... well, that's another topic. 2016-09-08 0:28 GMT-03:00 Nick Coghlan <ncoghlan@gmail.com>:
On 8 September 2016 at 13:23, Nick Coghlan <ncoghlan@gmail.com> wrote:
Beyond that practical benefit, if you want random-sampling-with-replacement, then "map(random.choice, container)"
Oops, that was supposed to be "map(random.choice, itertools.repeat(container))".
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-- Danilo J. S. Bellini --------------- "*It is not our business to set up prohibitions, but to arrive at conventions.*" (R. Carnap)
On 8 September 2016 at 13:33, Danilo J. S. Bellini <danilo.bellini@gmail.com> wrote:
Nice to know about random.sample! =)
I think what OP said can then be reduced to having the default k in random.sample to be the iterable size. The existance of random.sample is a very strong argument against "shuffled", and the only "feature" shuffled would have that random.sample doesn't have is that default size.
There are a few reasons I don't think defining a default for random.sample() would be a good answer to Arek's question: 1. The cognitive leap between shuffling and sampling isn't small 2. "shuffled" would be a more logical name for an out-of-place shuffle than "sample" 3. "Copy the whole container" would be a surprising default for a function called "sample" 4. With a default, random.sample becomes more easily confused with random.choice For the last two concerns, if I saw "result = random.sample(container)" and didn't already know about random.choice(), I'd expect it to behave like random.choice(). Even knowing they're different, I'd still need to do a double-take to make sure I was remembering which was which correctly. By contrast, "random.sample(container, 1)", "random.sample(container, k)", "random.sample(container, len(container))" are all clearly different from the single result primitive "random.choice(container)" One interesting (to me anyway) aspect of an out-of-place shuffle is that you can readily implement it based on *either* of the more primitive operations (in-place shuffle or random sample): def shuffled(container): result = list(container) random.shuffle(result) return result def shuffled(container): return random.sample(container, len(container)) Writing down those two examples does highlight a potential refactoring benefit to having "out-of-place shuffle" as an explicitly represented concept in the core Random API: it can be defined in terms of shuffle or sample *on the same Random instance*, and hence automatically benefit when code switches from using the global random number generator to using a purpose-specific Random instance that allows greater control over the reproducibility of results (which can be very important for testing, games, modeling & simulation). The above helper functions are both flawed on that front: they hardcode the use of the default global random number generator. To permit use of a specific random instance, they need to be changed to: def shuffled(container, random=random): result = list(container) random.shuffle(result) return result def shuffled(container, random=random): return random.sample(container, len(container)) and then used as "result = shuffled(original, my_random_instance)" if you decide to switch away from the global API. By contrast, a method based implementation could be refactored the exact same way as any other random.Random method: result = my_random_instance.shuffled(original) I'll reiterate that I don't have a use case for this myself, but I'll cite the key arguments I see in favour: - an out-of-place shuffle may be useful as a bridging concept between in place shuffling and out of place sampling (in either direction) - the presence of "shuffled" becomes a reminder that "shuffle" itself is an in-place operation - "write it yourself" isn't as simple as it first sounds due to the common migration away from the random module functions to a custom Random instance Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
1. The cognitive leap between shuffling and sampling isn't small
I don't think so, actually the Fisher-Yates shuffle algorithm algorithm is an iterative sampling algorithm: https://gist.github.com/danilo bellini/6384872
2. "shuffled" would be a more logical name for an out-of-place shuffle than "sample"
Agreed.
3. "Copy the whole container" would be a surprising default for a function called "sample"
Perhaps "sample the whole population" sounds strange. Indeed, statisticians probably wouldn't be happy with that. That reminds me the variance estimator with the extra unbiasing "-1" in the denominator versus the population variance.
4. With a default, random.sample becomes more easily confused with random.choice
I don't think k=1 would be a good default sample size from a statistics point of view, but I get the point (I'm from a DSP background, where "a sample" means one single "value"). Controling the random function is required for the function to be really pure, else its output won't depend only on the inputs (and there would be some "state" in that "implicit input"). That would also be a great feature when non-uniform (or external) random number generators are to be used. This seem to be something that only shuffle gives some control (among the functions we're talking about), or am I missing something? 2016-09-08 1:26 GMT-03:00 Nick Coghlan <ncoghlan@gmail.com>:
On 8 September 2016 at 13:33, Danilo J. S. Bellini <danilo.bellini@gmail.com> wrote:
Nice to know about random.sample! =)
I think what OP said can then be reduced to having the default k in random.sample to be the iterable size. The existance of random.sample is a very strong argument against "shuffled", and the only "feature" shuffled would have that random.sample doesn't have is that default size.
There are a few reasons I don't think defining a default for random.sample() would be a good answer to Arek's question:
1. The cognitive leap between shuffling and sampling isn't small 2. "shuffled" would be a more logical name for an out-of-place shuffle than "sample" 3. "Copy the whole container" would be a surprising default for a function called "sample" 4. With a default, random.sample becomes more easily confused with random.choice
For the last two concerns, if I saw "result = random.sample(container)" and didn't already know about random.choice(), I'd expect it to behave like random.choice(). Even knowing they're different, I'd still need to do a double-take to make sure I was remembering which was which correctly. By contrast, "random.sample(container, 1)", "random.sample(container, k)", "random.sample(container, len(container))" are all clearly different from the single result primitive "random.choice(container)"
One interesting (to me anyway) aspect of an out-of-place shuffle is that you can readily implement it based on *either* of the more primitive operations (in-place shuffle or random sample):
def shuffled(container): result = list(container) random.shuffle(result) return result
def shuffled(container): return random.sample(container, len(container))
Writing down those two examples does highlight a potential refactoring benefit to having "out-of-place shuffle" as an explicitly represented concept in the core Random API: it can be defined in terms of shuffle or sample *on the same Random instance*, and hence automatically benefit when code switches from using the global random number generator to using a purpose-specific Random instance that allows greater control over the reproducibility of results (which can be very important for testing, games, modeling & simulation).
The above helper functions are both flawed on that front: they hardcode the use of the default global random number generator. To permit use of a specific random instance, they need to be changed to:
def shuffled(container, random=random): result = list(container) random.shuffle(result) return result
def shuffled(container, random=random): return random.sample(container, len(container))
and then used as "result = shuffled(original, my_random_instance)" if you decide to switch away from the global API.
By contrast, a method based implementation could be refactored the exact same way as any other random.Random method:
result = my_random_instance.shuffled(original)
I'll reiterate that I don't have a use case for this myself, but I'll cite the key arguments I see in favour:
- an out-of-place shuffle may be useful as a bridging concept between in place shuffling and out of place sampling (in either direction) - the presence of "shuffled" becomes a reminder that "shuffle" itself is an in-place operation - "write it yourself" isn't as simple as it first sounds due to the common migration away from the random module functions to a custom Random instance
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-- Danilo J. S. Bellini --------------- "*It is not our business to set up prohibitions, but to arrive at conventions.*" (R. Carnap)
On 8 September 2016 at 15:05, Danilo J. S. Bellini <danilo.bellini@gmail.com> wrote:
1. The cognitive leap between shuffling and sampling isn't small
I don't think so, actually the Fisher-Yates shuffle algorithm algorithm is an iterative sampling algorithm: https://gist.github.com/danilobellini/6384872
I'm not talking about mathematical equivalence, I'm talking about making an unaided leap from "shuffle this deck of cards" (random.shuffle) and "pick a card, any card" (random.choice) to "choose a random sample from this population" (random.sample). I can see people following that logic given suitable instruction (since they really are closely related operations), but it's a tough connection to see on your own.
4. With a default, random.sample becomes more easily confused with random.choice
I don't think k=1 would be a good default sample size from a statistics point of view, but I get the point (I'm from a DSP background, where "a sample" means one single "value").
Likewise - it isn't that I think "1" would be a reasonable default, it's that without the second argument being there, I lapse back into DSP terminology rather than statistical terminology. Requiring the second argument as random.sample() does today keeps everything nicely unambiguous.
Controling the random function is required for the function to be really pure, else its output won't depend only on the inputs (and there would be some "state" in that "implicit input"). That would also be a great feature when non-uniform (or external) random number generators are to be used. This seem to be something that only shuffle gives some control (among the functions we're talking about), or am I missing something?
The module level "functions" in random are just bound methods for a default global random.Random() instance, so they're not truly pure - there's interdependence there via the shared PRNG state. However, by creating your *own* Random instance, or another object that provides the same API, you can get a lot more control over things, including reproducible behaviour for a given seed. I'm not familiar with how shuffle works internally, but presumably passing a non-uniform distribution is a way let you bias the shuffle (the docs don't actually explain *why* you'd want to use a randomiser other than the default). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Wed, Sep 07, 2016 at 11:43:59PM +0200, Sven R. Kunze wrote:
BUT experienced devs also need to recognize and respect the fact that younger/unexperienced developers are just better in detecting inconsistencies and bloody work-arounds.
That is not a fact. It is the opposite of a fact -- inexperienced developers are WORSE at spotting inconsistencies, because they don't recognise deep consistencies. For example, the on-going controversy about mutable default arguments: def func(x=0, y=[]): x += 1 y.append(1) print(x, y) To the inexperienced Python developer this makes no sense. Why does Pythod reset the value of x by default, while y remembers the value it had from the previous call? That's an inconsistency. But the more experienced dev realises that there is no inconsistency. Python behaves exactly the same way in both cases. Both x and y are treated the same: they both are treated as "Early Binding". The only difference is that 0 is immutable, and so x += 1 binds a new value to x, while [] is mutable, and y.append(1) modifies that value in place. Try this instead: def func(x=0, y=[]): x += 1 y = y + [1] # don't use += print(x, y) The more experienced programmer might not *like* this behaviour, they may wish that Python used Late Binding instead, but it takes a very subtle and deep knowledge of the language to understand that this is not an inconsistency despite the superficial appearances. -- Steve
On 8 September 2016 at 12:00, Steven D'Aprano <steve@pearwood.info> wrote:
On Wed, Sep 07, 2016 at 11:43:59PM +0200, Sven R. Kunze wrote:
BUT experienced devs also need to recognize and respect the fact that younger/unexperienced developers are just better in detecting inconsistencies and bloody work-arounds.
That is not a fact. It is the opposite of a fact -- inexperienced developers are WORSE at spotting inconsistencies, because they don't recognise deep consistencies.
There are also cases where we're internally inconsistent or lacking internal integration because we're being consistent with different external environments. So we have both binary floating point and decimal floating point because there are inherent conflicts in the underlying computational models. We have both the abc module (which deals strictly with runtime classes) and the typing module (which deals strictly with type categories) as those things are similar, but not the same. We have both asyncio and traditional synchronous APIs as there are essential conflicts between blocking and non-blocking models of control flow. Cross-platform secure network connectivity handling is a pain because many of the underlying platform APIs are unreliably implemented and poorly documented moving targets. Cross-platform software distribution is a pain largely because major platform vendors are typically competing to lock in developers, so their incentives are aligned with making what we're trying to do difficult rather than offering to help out. For these kinds of cases, frustrations on the part of experienced developers arise when folks ask entirely reasonable questions "Why is <topic area> so complicated?", but then refuse to accept the answer of "Because it's genuinely complex". Pretending complexity doesn't exist doesn't help anyone - it means they're left to their own devices to come up with answers, rather than receiving more explicit guidance at the language level. More directly relevant to the current discussion is the principle of "Not every three line function needs to be a builtin (or even in the standard library)". There are ways to collect objective data on the relative usage of particular constructs (especially as hosting platforms like GitHub team up with analytics platforms like Google Big Query to make their data searchable, and initiatives like Software Heritage aim to bring that kind of capability to all published open source software), but one of the *simplest* criteria to apply, and the one that experienced core developers can typically answer without further research, is "Would the standard library benefit from this?". When the answer to that is "No", then the default answer to requests for API additions that are relatively easy for people to implement for themselves is always going to be "No" - otherwise the stdlib will collapse under its own weight (and the persistent "up and to the right" trend on http://bugs.python.org/issue?@template=stats means that there's a solid argument to be made that the standard library is already bigger than we can effectively maintain at current maintenance funding levels). Cheers, Nick. P.S. I wrote http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-... about this a few years ago, and I still think it's an accurate description of the underlying causes of these conflicts -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 08.09.2016 04:00, Steven D'Aprano wrote:
On Wed, Sep 07, 2016 at 11:43:59PM +0200, Sven R. Kunze wrote:
BUT experienced devs also need to recognize and respect the fact that younger/unexperienced developers are just better in detecting inconsistencies and bloody work-arounds. That is not a fact. It is the opposite of a fact
You might have heard of: "There are no such things as facts, just opinions. Everything, we see is a perspective not the truth." See below, why this applies here as well.
-- inexperienced developers are WORSE at spotting inconsistencies, because they don't recognise deep consistencies.
Your example (default arguments) might make sense when going one or two level deeper. However: Is going deep really necessary at all? People program for different reasons, to have fun, to create value for others, to educate, or for reasons we both cannot even think of. Why should they leave their level? Because of you or me? Because they need to know what Turing-completeness means? What calling conventions are? I don't think so. They wanna solve problems and get things done whether or not they know every single bit of the language they use. If they decide to go deeper, that's wonderful, but if they don't, don't force them. So, an inconsistency at level 1 might be a **result of a consistency at level 0** BUT it nevertheless is and stays an inconsistency at level 1 no matter how sophisticated the consistency at level 0 is. And please note, some even want to go on level up, so inconsistencies at level 1 just plain suck then. People then simply don't care about how the current flows at cpu level or about your carefully handcrafted bits and pieces on level 0 which you are so proud of and which are so consistent there. Cheers.
On 10 September 2016 at 22:09, Sven R. Kunze <srkunze@mail.de> wrote:
Is going deep really necessary at all?
People program for different reasons, to have fun, to create value for others, to educate, or for reasons we both cannot even think of. Why should they leave their level? Because of you or me? Because they need to know what Turing-completeness means? What calling conventions are? I don't think so. They wanna solve problems and get things done whether or not they know every single bit of the language they use. If they decide to go deeper, that's wonderful, but if they don't, don't force them.
If people don't want to actively learn about the enormous complexities of real world programming language design, then they need to be willing to defer to those that have put in that time and effort. Deliberately choosing to argue from a position of wilful ignorance is disrespectful of everyone else's time and attention, and has no place on python-ideas or python-dev. It's fine for newcomers not to understand those complexities yet - many of these problems don't have obvious answers, and explaining them to someone else is a great way to better formulate them in our own minds (hence the martial arts maxim: "If you want to learn, teach"). However, it's utterly unacceptable to refuse to acknowledge "It's more complicated than you currently realise" as a reasonable answer, especially when that answer is accompanied by explanations of some of those complexities. If folks aren't willing to adhere to those terms of participation, they can save themselves and everyone else a lot of irritation by unsubscribing voluntarily, rather than waiting until they annoy other list participants enough to earn a suspension of their posting privileges (and potentially even a permanent ban if they're persistent enough, although that step has only had to happen once to date). Regards, Nick. P.S. If folks want to vent about the arrogance and intransigence of the Python core development team because we collectively disagree with them about something, they have the entire rest of the internet to do that without interfering with the collaborative process. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Tue, Sep 6, 2016 at 1:15 PM, Sven R. Kunze <srkunze@mail.de> wrote:
Their response: "Oh, I don't need it, let's close it." Arek: "But I need it."
This definitely feels like a case of "put it on PyPI." Actually, maybe contribute to `boltons`, it feels like it might fit as a utility function there. While I wouldn't mind being able to type `from random import shuffled`, I don't have a bit problem instead typing `from boltons import shuffled`, nor even `from arek_utils import shuffled`. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Tue, Sep 6, 2016 at 2:48 PM, David Mertz <mertz@gnosis.cx> wrote:
On Tue, Sep 6, 2016 at 1:15 PM, Sven R. Kunze <srkunze@mail.de> wrote:
Their response: "Oh, I don't need it, let's close it." Arek: "But I need it."
This definitely feels like a case of "put it on PyPI." Actually, maybe contribute to `boltons`, it feels like it might fit as a utility function there.
While I wouldn't mind being able to type `from random import shuffled`, I don't have a bit problem instead typing `from boltons import shuffled`, nor even `from arek_utils import shuffled`.
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[David Mertz <mertz@gnosis.cx>]
This definitely feels like a case of "put it on PyPI." Actually, maybe contribute to `boltons`, it feels like it might fit as a utility function there.
It's trivial to write such a function if it's truly needed - it would be easier to write it from scratch than to remember which module it's hiding in. There's a "clever" 1-liner, but with no imagination at all it's still dead obvious: def shuffled(xs): from random import shuffle xs = xs[:] shuffle(xs) return xs `boltons` doesn't typically bother with screamingly obvious things.
While I wouldn't mind being able to type `from random import shuffled`, I don't have a bit problem instead typing `from boltons import shuffled`, nor even `from arek_utils import shuffled`.
But would you _use_ it? I'm still asking for use cases. When, e.g., I'm randomizing permutations for testing, the last thing I want is: while whatever: result = function_of_xs(shuffled(xs)) check result and complain if it's wrong Why not? Because no trace remains of _which_ permutation provoked the failure when a failure occurs. Instead code looks like this: while whatever: shuffle(xs) result = function_of_xs(xs) # or xs[:] if the function mutates its arg check result and complain that `xs` specifically provoked a failure Indeed, the only clear "use case" that makes sense I've been able to think of is: xs = shuffled(xs) But that's written more easily and efficiently today as shuffle(xs) This is in stark contrast to sorted(), where clear use cases abound. That didn't get in because it's hard to mimic (it's basically as easy as shuffled()), but because it's exactly what's wanted in all kinds of contexts in all kinds of code.
On Tue, Sep 6, 2016 at 3:04 PM, Tim Peters <tim.peters@gmail.com> wrote:
But would you _use_ it? I'm still asking for use cases.
When, e.g., I'm randomizing permutations for testing, the last thing I want is:
while whatever: result = function_of_xs(shuffled(xs)) check result and complain if it's wrong
Why not? Because no trace remains of _which_ permutation provoked the failure when a failure occurs. Instead code looks like this:
while whatever: shuffle(xs) result = function_of_xs(xs) # or xs[:] if the function mutates its arg check result and complain that `xs` specifically provoked a failure
Indeed, the only clear "use case" that makes sense I've been able to think of is:
xs = shuffled(xs)
But that's written more easily and efficiently today as
shuffle(xs)
This is in stark contrast to sorted(), where clear use cases abound. That didn't get in because it's hard to mimic (it's basically as easy as shuffled()), but because it's exactly what's wanted in all kinds of contexts in all kinds of code.
I'll weigh in and say that I've had a few cases where I've wanted a shuffled function, but not many. The vast majority are interactive uses, where I want to get a sampling of data, and in those cases I'm normally just printing the output (often, by letting the REPL handle it). I'm fairly sure I've never wanted a shuffled in the actual code, and shuffle is almost always what I want (or just pulling items at random). Probably the most common case is to produce a list of random numbers in a (small) range. The current code looks roughly like: import random items = list(range(10)) random.shuffle(items) items # this is interactive, so this prints it for me As this does not come up often, I almost invariably write the following first: import random random.shuffle(range(10)) Then get no output and write the first form. It is not a major difference, and only comes up maybe a few times a year at most for me. Chris
On 07.09.2016 02:49, Chris Kaynor wrote:
I'll weigh in and say that I've had a few cases where I've wanted a shuffled function, but not many. The vast majority are interactive uses, where I want to get a sampling of data, and in those cases I'm normally just printing the output (often, by letting the REPL handle it).
I'm fairly sure I've never wanted a shuffled in the actual code, and shuffle is almost always what I want (or just pulling items at random).
Probably the most common case is to produce a list of random numbers in a (small) range. The current code looks roughly like:
import random items = list(range(10)) random.shuffle(items) items # this is interactive, so this prints it for me
As this does not come up often, I almost invariably write the following first: import random random.shuffle(range(10))
Then get no output and write the first form. It is not a major difference, and only comes up maybe a few times a year at most for me.
That sounds extremely familiar. I would say the interactive session is the clear use-case here. But don't ask me why I need a shuffled list of somethings (probably for some highly vicious and immoral reasons ;-) ). @David Your idea of a PyPI package could almost work. However, in an interactive python console, I expect as much batteries included as possible to make it as quick as possible. And considering how simple such wrapper would be, it almost does not warrant the download of a third-party package. @Tim Of course it's easy to write a wrapper function. But from what I gather here, this is not the point. In interactive sessions, I find it highly annoying when I need to define my own functions. Especially because I need to do it again in a new python session. Somebody provided a one-line hack using "sorted" to emulate "shuffled". The statement basically is: shuffling is a special kind of sorting. So, I would expect the interface to work the same. That at least suffices for me to understand Arek's point of view. I would even go so far as to say: shuffled(my_list) # returns a new shuffled list my_list.shuffle() # shuffles in-place Allowing to plug-in the RNG, when needed. Sven
On Wed, Sep 7, 2016 at 3:10 PM, Sven R. Kunze <srkunze@mail.de> wrote:
@David Your idea of a PyPI package could almost work. However, in an interactive python console, I expect as much batteries included as possible to make it as quick as possible. And considering how simple such wrapper would be, it almost does not warrant the download of a third-party package.
Of course I don't think it's worth having a PyPI package for this one function. That's why I suggested it could possibly live in `boltons` (that already contains a lot of other simple functions that aren't in the stdlib) or in `arek_utils` (which I haven't described except implicitly as a collection of "a bunch of useful things"). Tim argues that this functionality is even too simple for inclusion in boltons; that might be true. But either way, it's not a huge burden in the interactive console to type a first line of `from my_utils import *` before getting down to work. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On 09/06/2016 11:15 AM, Sven R. Kunze wrote:
On 06.09.2016 18:32, Steven D'Aprano wrote:
If your only argument is to continue to insist that Python should have a shuffled() function and you'll write a patch, it will go no where. First you have to convince people that the patch is needed.
He already convinced some people. Just not some venerable Python devs, which doesn't necessarily mean something at all.
It means it's not going to happen.
Their response: "Oh, I don't need it, let's close it." Arek: "But I need it."
So, who's right now?
Everyone: - if the python devs say it doesn't need to be in the stdlib, it's not going in. - if Arek (or anyone) needs it, then Arek (and those who need it) can write it and put it in their own personal utility library.
I would suggest not being so protective. It's community project after all.
No, it isn't. It is Guido's project.
Maybe, the patch will help him understand that it's not so easy, or other people will see the benefit after reading it. If the patch isn't worth including, he learned something in the process of creating it.
Sure. The point is simply that writing and submitting a patch does not grant automatic acceptance. -- ~Ethan~
participants (15)
-
Arek Bulski -
Bernardo Sulzbach -
Chris Angelico -
Chris Kaynor -
Danilo J. S. Bellini -
David Mertz -
Ethan Furman -
Matt Gilson -
Nick Coghlan -
Rob Cliffe -
Stephen J. Turnbull -
Steven D'Aprano -
Sven R. Kunze -
Tim Peters -
אלעזר