IIRC it's an old micro-optimization; this was a common idiom at Zope. But I think it's way overused -- people believe it works so they do it all the time, even for code that's not performance sensitive, just because it's become a habit. (Like "register" in C in the '80s.) On Tue, Oct 27, 2015 at 10:55 AM, Yury Selivanov wrote:

Serhiy,

On 2015-10-27 1:45 PM, Serhiy Storchaka wrote:

...

There is known trick to optimize a function:

def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<

It has a side effect: change function's signature. Would be nice to have a way to set function's local variables at creation time without affecting a signature.

I see this a lot in all kinds of code. In my experience it doesn't actually speed things up in a measurable way.

Is the below code really much slower?

def foo(x, y=0): pack=struct.pack maxsize=1<

If the #CODE is a tight long-running loop - then no, because the loop will probably run much longer than an extra attribute lookup + one extra bit shift on each "foo()" call. And if there is no tight loop - then you won't probably notice those optimizations anyways.

I think that adding a "const" statement deserves some discussion, but not from the standpoint of micro-optimizations.

Thanks, Yury

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

On Tue, Oct 27, 2015 at 11:18 AM, Guido van Rossum wrote:

IIRC it's an old micro-optimization; this was a common idiom at Zope. But I think it's way overused -- people believe it works so they do it all the time, even for code that's not performance sensitive, just because it's become a habit. (Like "register" in C in the '80s.)

On Tue, Oct 27, 2015 at 10:55 AM, Yury Selivanov wrote:

...

Serhiy,

On 2015-10-27 1:45 PM, Serhiy Storchaka wrote:

...

There is known trick to optimize a function:

def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<

It's not a micro-optimization in this case:

def foo(x, SENTINEL=object()): ... I don't like mangling function signatures to do this. What I really want is the equivalent of C's static here: def foo(x): static SENTINEL = object() ... This has two important semantics: (1) the scope of the SENTINEL variable is limited to the function foo; (2) it is only initialized once. Hosting the value out of the function into a decorator like @asconstants is fine. --- Bruce Check out my new puzzle book: http://J.mp/ingToConclusions http://j.mp/ingToConclusions Get it free here: http://J.mp/ingToConclusionsFree http://j.mp/ingToConclusionsFree (available on iOS)

On Oct 27 2015, Yury Selivanov wrote:

On 2015-10-27 3:02 PM, Bruce Leban wrote:

...

It's not a micro-optimization in this case:

def foo(x, SENTINEL=object()): ...

I don't like mangling function signatures to do this. What I really want is the equivalent of C's static here:

def foo(x): static SENTINEL = object() ...

Why not

SENTINEL=object() def foo(x): ...

?

Because it leaks into the enclosing scope. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«

On 27.10.2015 21:30, Yury Selivanov wrote:

On 2015-10-27 4:22 PM, Nikolaus Rath wrote:

...

Because it leaks into the enclosing scope.

Leaks what? Name? Name it _SENTINEL then.

I think because it's missing the relationship with foo. So, better do this foo._sentinel = object() It's coupled to foo and moved out of the outer scope. Best, Sven

On Tue, Oct 27, 2015 at 4:36 PM, Steven D'Aprano wrote:

On Tue, Oct 27, 2015 at 04:30:41PM -0400, Yury Selivanov wrote:

...

On 2015-10-27 4:22 PM, Nikolaus Rath wrote:

...

Because it leaks into the enclosing scope.

Leaks what? Name? Name it _SENTINEL then.

It is still in the enclosing scope.

Bruce is right that what we really want is something like "static". Lacking that feature, one work-around is "make it a global variable, and give it a leading underscore", but that's just a work-around.

Consider:

_SENTINEL = object()

# Later: def spam(): ... def eggs(): ... def cheese(): ... def aardvark(): ...

As the module user, you can see the leading underscore in _SENTINEL and immediately forget all about it. But as the module maintainer, you cannot ignore _SENTINEL. Leading underscore or not, it is still part of the implementation, and maintenance is all about the implementation.

Which of the functions uses _SENTINEL? You can assume that *at least* one function uses it, but it may be more. Is it safe to rename it? Change it's value? To the maintainer, _SENTINEL is just another global variable, with all the disadvantages that has.

Better to put it inside the function, so the maintainer knows that it is local to the function:

def spam(): SENTINEL = object() ...

That's not bad, but what if the value is something harder to calculate?

def spam(): SENTINEL = next_prime_number(2**512) ...

At the moment, Python has an obvious way to calculate a value once only (make it a module-level global), and there is an obvious way to make a value local to a function (put it inside the function body). But there's no obvious way to do both together:

"Calculate this thing once only, AND make it local to this scope."

The unobvious and ugly way is to put the calculation in the function declaration as a default value:

def spam(SENTINEL=next_prime_number(2**512)): ...

which complicates the function signature and risks errors if the caller accidentally calls the function with too many arguments.

Python intentionally doesn't have this feature, and the argument you present sounds pretty weak (compared to the compelling argument *against* abusing the default argument syntax -- which also has a maintenance cost, as you have to explain it over and over). -- --Guido van Rossum (python.org/~guido)

On 27.10.15 19:55, Yury Selivanov wrote:

I see this a lot in all kinds of code. In my experience it doesn't actually speed things up in a measurable way.

I think that adding a "const" statement deserves some discussion, but not from the standpoint of micro-optimizations.

Agree, we slowly gots rid of such microoptimizations. Seems that builtins and globals lookup is much faster in current CPython than was when such microoptimizations were added. Yet one application of this trick is binding globals and module's members in __del__ methods and functions that can be called at shutdown time after module's content cleared. But this is very limited application.

On Tue, Oct 27, 2015 at 12:51 PM, Mark Lawrence wrote:

On 27/10/2015 17:55, Yury Selivanov wrote:

...

Serhiy,

On 2015-10-27 1:45 PM, Serhiy Storchaka wrote:

...

There is known trick to optimize a function:

def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<

It has a side effect: change function's signature. Would be nice to have a way to set function's local variables at creation time without affecting a signature.

I see this a lot in all kinds of code. In my experience it doesn't actually speed things up in a measurable way.

Is the below code really much slower?

def foo(x, y=0): pack=struct.pack maxsize=1<

If the #CODE is a tight long-running loop - then no, because the loop will probably run much longer than an extra attribute lookup + one extra bit shift on each "foo()" call. And if there is no tight loop - then you won't probably notice those optimizations anyways.

I think that adding a "const" statement deserves some discussion, but not from the standpoint of micro-optimizations.

Thanks, Yury

From my very naive perspective I'd have thought that the only real difference between the two implementations is that Yury's has the optimization hard coded within the function body, while Serhiy's allows you to override the hard coded defaults at run time. Am I hot, warm, tepid, cold or approaching 0 Kelvin?

Yury's code has to look up the global value of struct, and then get its pack attribute, every time the function is called. Serhiy's code only does this once, when the function is created, and when it is called the local value is loaded directly from the function defaults, which amounts to a single tuple lookup rather than two sequential dict lookups.