[Python-ideas] Using functools.lru_cache only on some arguments of a function

[Ryan Gonzalez]

http://thecodelesscode.com/topics/caching

This smells like something that could cause trouble to me... If it's in the stdlib, it's easier for someone to use it and then be confused when something doesn't work correctly (maybe someone else will make partial_state affect the result, not realizing that that argument is ignored).

Too buggy to me.

On December 4, 2015 3:44:18 PM CST, Bill Winslow <bunslow at gmail.com> wrote:
>This is a question I posed to reddit, with no real resolution:
>https://www.reddit.com/r/learnpython/comments/3v75g4/using_functoolslru_cache_only_on_some_arguments/
>
>The summary for people here is the following:
>
>Here's a pattern I'm using for my code:
>
>def deterministic_recursive_calculation(input, partial_state=None):
>    condition = do_some_calculations(input)
>    if condition:
>        return deterministic_recursive_calculation(reduced_input,
>some_state)
>
>Basically, in calculating the results of the subproblem, the subproblem
>can
>be calculated quicker by including/sharing some partial results from
>the
>superproblem. (Calling the subproblem without the partial state still
>gives
>the same result, but takes substantially longer.)
>
>I want to memoize this function for obvious reasons, but I need the
>lru_cache to ignore the partial_state argument, for its value does not
>affect the output, only the computation expense.
>
>Is there any reasonable way to do this?
>
>Things such as memoizing a wrapper don't actually solve the problem.
>About
>the only way I can think of with current technology is either to have a
>hidden singleton class which maintains state, or a hidden global
>variable,
>which amount to the same thing of storing the partial state outside the
>function. But those strike me as unwieldy and unpythonic.
>
>What I'd really like to do is to have some way to tell
>functools.lru_cache
>to ignore some arguments of a function it's memoizing for the purposes
>of
>caching.
>
>One way would be to add an "arg_filter" argument, which for purposes of
>this example would be used like so:
>
>@lru_cache(arg_filter=lambda args, kwargs: args[:1], {})
>def deterministic_recursive_calculation(input, partial_state=None):
>    condition = do_some_calculations(input)
>    if condition:
>        return deterministic_recursive_calculation(reduced_input,
>some_state)
>
>This effectively ignores all arguments except the first positional one
>for
>the purposes of caching. Such an option could be implemented as in the
>diff
>below (provided only for discussion purposes).
>
>So what I want to know is:
>1) Am I sane? Is there no particularly good way currently to go about
>caching functions following the given pattern?
>2) Assuming the answer to 1) is "Yes I am sane, and there's no good way
>currently", is my proposal a reasonable one? Mostly on philosophical
>grounds, not necessarily specifics of how it works.
>
>Thank you for your time and consideration.
>
>Bill
>
>----------------------------------------------------------------------------------------------------------------------------------
>https://hg.python.org/cpython/file/3.5/Lib/functools.py
>
>diff functools.py functools.py.orig
>363c363
>< def _make_key(args, kwds, typed, arg_filter,
>---
>> def _make_key(args, kwds, typed,
>377,378d376
><     if arg_filter is not None:
><         args, kwds = arg_filter(args, kwds)
>393c391
>< def lru_cache(maxsize=128, typed=False, arg_filter=None):
>---
>> def lru_cache(maxsize=128, typed=False):
>403,406d400
><     *arg_filter* is an optional function which filters
>user-speicified
>portions
><     of the arguments from the caching key (e.g. if an argument
>affects the
><     computation but not the final result).
><
>428,430d421
><     if arg_filter is not None and not callable(arg_filter):
><         raise TypeError('Expected arg_filter to be a callable')
><
>432,433c423
><         wrapper = _lru_cache_wrapper(user_function, maxsize, typed,
>arg_filter,
><                                      _CacheInfo)
>---
>>         wrapper = _lru_cache_wrapper(user_function, maxsize, typed,
>_CacheInfo)
>438c428
>< def _lru_cache_wrapper(user_function, maxsize, typed, arg_filter,
>_CacheInfo):
>---
>> def _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo):
>466c456
><             key = make_key(args, kwds, typed, arg_filter)
>---
>>             key = make_key(args, kwds, typed)
>481c471
><             key = make_key(args, kwds, typed, arg_filter)
>---
>>             key = make_key(args, kwds, typed)
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151204/619733e3/attachment.html>