Le mardi 20 décembre 2011 à 12:01 +0200, Maciej Fijalkowski a écrit :

If this documentation is to be used by other python implementations, then mentions of performance are outright harmful, since the performance characteristics differ quite drastically. Written in C is also not a part of specification as far as I know :)

But that's basically the only reason to invoke the `operator.attrgetter("foo")` ugliness, instead of writing the explicit and obvious `lambda x: x.foo`. So not mentioning that it provides a speed benefit on CPython hides the primary reason for using the operator module. Overwise it's just a bunch of useless wrappers. --------- More generally, not talking about performance at all is more harmful than making CPython-specific comments in the documentation. Implementation details *deserve* to be documented when they have an impact on behaviour (including performance / resource usage). Python is not just a platonic ideal. Do you suggest we also remove this part: http://docs.python.org/dev/library/io.html#performance ? Regards Antoine.

On Tue, 20 Dec 2011 11:14:15 +0100 Dirkjan Ochtman wrote:

On Tue, Dec 20, 2011 at 11:08, Antoine Pitrou wrote:

...

...

If this documentation is to be used by other python implementations, then mentions of performance are outright harmful, since the performance characteristics differ quite drastically. Written in C is also not a part of specification as far as I know :)

But that's basically the only reason to invoke the `operator.attrgetter("foo")` ugliness, instead of writing the explicit and obvious `lambda x: x.foo`. So not mentioning that it provides a speed benefit on CPython hides the primary reason for using the operator module. Overwise it's just a bunch of useless wrappers.

So the question is if the docs are Python documentation or CPython documentation? On PyPy, I'm guessing lambda x: x.foo might (some day) be just as fast as operator.attrgetter("foo").

I would expect it to be just as fast right now, although that's just an uninformed guess. That said, CPython is both the dominant implementation and the only one (AFAIR) to have stable 3.2 support.

...

Implementation details *deserve* to be documented when they have an impact on behaviour (including performance / resource usage). Python is not just a platonic ideal. Do you suggest we also remove this part: http://docs.python.org/dev/library/io.html#performance ?

I agree that it's good to document some implementation details, but it seems like the paragraph, as it was before, documented too many details. It seems like a paragraph that mentions the specificity of this aspect for CPython and omits the reference to C as the VM implementation should be acceptable to all parties.

Agreed. The original wording was poor since it mentioned C while what is really significant is performance. There are probably Python programmers who don't even know what C is. Regards Antoine.

On 2011-12-20, at 11:08 , Antoine Pitrou wrote:

But that's basically the only reason to invoke the `operator.attrgetter("foo")` ugliness, instead of writing the explicit and obvious `lambda x: x.foo`. I don't agree with this, an attrgetter in the current namespace can be clearer than an explicit lambda in place, and more importantly when trying to fetch more than one attribute attrgetter is far superior to lambdas as far as I'm concerned.

I don't think I've ever seen `attrgetter` (or any of the other `operator` functions) advocated on basis of speed. This mention does not even exist in the Python 2 docs, which does not prevent people from using `operator`.

On Tue, 20 Dec 2011 05:27:41 -0500 Terry Reedy wrote:

...

I disagree with this change. Knowing that they are written in C is important when deciding to pass them to e.g. sort() or sorted(), because you know it will be faster than an arbitrary pure Python function.

You could tag it as a "CPython implementation detail" if you want, or talk about performance rather than mention "C".

The existence of operator and the behavior of its functions is not a C implementation detail.

And?

I think a programmer can assume that they are are written in the implementation language to be as fast as possible.

Yeah, you can assume anything, and then get bitten by the fact that e.g. OrderedDict is pure Python and thus massively slower than dict. But at least you've achieved some platonic ideal of how documentation should not talk about implementation details, which is great, right? Why you think we should leave users in the dark rather than inform them is beyond me. While we certainly should find a good compromise between readability and completeness, and should certainly tweak the doc's wording and layout adequately, removing useful information is nonsense.

For instance, there is nothing is the library manual that I can see that specifies that the builtin functions and types are written in C (for CPython).

I guess everyone expects builtin functions and types to be reasonably fast, regardless of the language or implementation. (even though I did see some beginner code rewrite its own slow "list" wrapper, so it's probably not an universal expectation)

Perhaps Python Setup and Usage could be renamed CPython Setup and Usage and expanded with more info on gc (ref counting), O() notes, Python vs. C code, etc.

Really? That's a perfectly inappropriate place to talk about performance details of *any* implementation. Regards Antoine.

On Tue, Dec 20, 2011 at 11:27, Terry Reedy wrote:

And I remember that Guido has asked that the manual not discuss big O() behavior of the methods of builtin classes.

Do you know when/where he did that? It seems useful to know that on CPython, list.insert(0, x) will become slow as the list grows... It probably shouldn't be upfront, but O() hints for some of the core stuff seems useful (though again, in some cases they should probably be limited to CPython). Cheers, Dirkjan

On Tue, Dec 20, 2011 at 6:24 AM, Dirkjan Ochtman wrote:

On Tue, Dec 20, 2011 at 11:27, Terry Reedy wrote:

...

And I remember that Guido has asked that the manual not discuss big O() behavior of the methods of builtin classes.

Do you know when/where he did that? It seems useful to know that on CPython, list.insert(0, x) will become slow as the list grows... It probably shouldn't be upfront, but O() hints for some of the core stuff seems useful (though again, in some cases they should probably be limited to CPython).

I think the question of the day is whether the documentation is targeting those who wish to have an understanding of what is happening under the hood, or those that want to take such details for granted. I much prefer the little notes and performance hints. - John

On 12/20/2011 11:15 AM, Benjamin Peterson wrote:

2011/12/20 Antoine Pitrou:

...

Le mardi 20 décembre 2011 à 10:57 -0500, Benjamin Peterson a écrit :

...

In that case, I would rather speak of "fast" functions rather than "implemented in C" functions (a la the itertools docs). Would that be acceptable?

Definitely.

Done.

I like what you did too. -- Terry Jan Reedy