@lazy decorator an alternative to functools.partial ?
Hello all! I've been playing with string templating quite a bit lately and wrote some logic to help me with this. I found it a bit more flexible than using partial and wanted to get feedback to see if it would be worth formalizing this code into the functools module. It's pretty short so I don't think it deserves its own module, and it behaves similarly to partial but has a bit more flexibility. Thanks for your time! ```python from functools import wraps from typing import Any, Callable, Union def lazy(func: Callable[..., Any]) -> Callable[..., Any]: """ A decorator that buffers args and kwargs gathering and function execution suspending processing until the function is called without any args or kwargs. This allows any function to support partials for decomposition out of the box. Also, supports variable length arguments and kw_arguments. Example: import math @lazy def velocity(height_m: float, acceleration_m2: float) -> float: return math.sqrt(2 * acceleration_m2 * height_m) g = -9.81 # acceleration due to gravity in m/s^2 earth = velocity(acceleration=g) v = earth(height=1)() print("The velocity is:", v, "m/s") v = earth(height=10)() print("The velocity is:", v, "m/s") v = earth(height=100)() print("The falling velocity is:", v, "m/s") """ @wraps(func) def collect(*args: Any, **kwargs: Any) -> Any: if not args and not kwargs: return func(*args, **kwargs) return ( lambda *next_args, **next_kwargs: func( *(args + next_args), **{**kwargs, **next_kwargs} ) if not next_args and not next_kwargs else collect(*(args + next_args), **{**kwargs, **next_kwargs}) ) return collect @lazy def f_string_head(f_str: str, **kwargs) -> Union[Callable[[], str], str]: """ Turns an f-string into a curry-able function, will return head to end or first KeyError when finally evaluated with no arguments. """ try: results = f_str.format_map({} if kwargs is None else kwargs) return results except KeyError as e: sub_template = f_str.split("{" + e.args[0] + "}", 1)[0] results = sub_template.format_map({} if kwargs is None else kwargs) return results def test_f_string() -> None: """ Tests the f_string function. """ template = """{one}{two}{three}""" f_one = f_string_head(template, one="1") f_one_three = f_one(three="3") f_one_two_three = f_one_three(two="2") assert f_one() == "1" assert f_one_three() == "1" assert f_one_two_three() == "123" def test_curring() -> None: @lazy def my_function(*args, **kwargs): print( "Function evaluated with arguments:", args, "and keyword arguments:", kwargs ) assert args == (1, 2, 3, 4) assert kwargs == {"a": 2, "b": 5} return "Function passed" partial_function = my_function(1, 2, a=3) result = partial_function(3, 4, b=5, a=2) results = result() print(results) def test_add() -> None: @lazy def add(*args: int, **kwargs: int) -> int: """ A function that takes any number of integer arguments and returns their sum. Args: *args: Integer arguments to be summed. **kwargs: Integer keyword arguments to be summed. Returns: The sum of the integer arguments. """ return sum(args) + sum(kwargs.values()) result = add(1)(2, 3)(4)(5, six=6)() print(f"Add={result}") def test_velocity() -> None: import math @lazy def velocity(height_m: float, acceleration_m2: float) -> float: return math.sqrt(2 * acceleration_m2 * height_m) g = 9.81 # acceleration due to gravity in m/s^2 earth = velocity(acceleration_m2=g) v = earth(height_m=5)() print("The falling velocity is:", v, "m/s") v = earth(height_m=15)() print("The falling velocity is:", v, "m/s") v = earth(height_m=30)() print("The falling velocity is:", v, "m/s") def main() -> None: """ The main function that runs the program. """ test_curring() test_add() test_f_string() test_velocity() if __name__ == "__main__": main() ```
Seems fine, but a pypi library seems better than the standard library. Separately, I think this is usually called "currying", and there are already libraries which implement this functionality, eg toolz: https://toolz.readthedocs.io/en/latest/curry.html Best wishes, Lucas Wiman
Thanks for the reference, Lucas. I wasn't familiar with toolz and it looks similar to a package I'm contributing to with a similar purpose of filling holes in the standard library, as such shouldn't some of this stuff be targeted for integration? I'm a bit dubious about the pypi suggestion as packages are being regularly poisoned with malware ( e.g. New KEKW malware infects open-source Python Wheel files via a PyPI distribution | SC Media (scmagazine.com) <https://www.scmagazine.com/news/devops/kekw-malware-infects-open-source-python-wheel-files> ) and support issues keep happening with package management tools. As a result, I have been hesitant to onboard new packages without at least a code review (not that would help me against future deltas) and would prefer to import as few modules outside the standard library as a result. Also, I was under the impression that Python was "batteries included" so why wouldn't it include:lazy, currying, monads...directly in the standard library? On Sat, May 13, 2023 at 6:39 PM Lucas Wiman <lucas.wiman@gmail.com> wrote:
Seems fine, but a pypi library seems better than the standard library. Separately, I think this is usually called "currying", and there are already libraries which implement this functionality, eg toolz: https://toolz.readthedocs.io/en/latest/curry.html
Best wishes, Lucas Wiman
On Wed, May 17, 2023 at 2:22 PM Daniel Guffey <daniel.guffey@gmail.com> wrote:
I'm a bit dubious about the pypi suggestion as packages are being regularly poisoned with malware ( e.g. New KEKW malware infects open-source Python Wheel files via a PyPI distribution | SC Media (scmagazine.com) <https://www.scmagazine.com/news/devops/kekw-malware-infects-open-source-python-wheel-files> ) and support issues keep happening with package management tools.
This is an absurd complaint. For one, the PyPA dealt with that very quickly. But more relevantly, Toolz is a package with many years of development by well-trusted people. Yes, getting a brand new malware onto PyPI is a danger, but that's a completely unrelated issue than using well-established and signed packages from known people. If you weirdly distrust PyPI, you can equally get the same thing via GitHub... I guess unless you also distrust those repos. It's not absurd to suggest a new decorator for the standard library. But "I don't trust PyPI" isn't going to win you any support for the idea. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
Apologies, I didn't mean to imply PyPI was inherently untrustworthy, unusable, or irrelevant. Clearly, it has a place and I use it for packages that I am familiar with and trust. The frame I'm trying to convey is that: 1. Developers are not the only consumers. e.g. If you're in an organization with a security team then adding new PyPI packages without review may not even be an option. 2. The scope of the standard library is debatable, however, I'm trying to focus on functionality that I think should be standard or can reasonably argue such, and in this case, I'm talking about basic functional language features. 3. There is a difference between being included in the standard library and not. Trust, visibility, availability, and keeping people from `re-inventing the wheel`. 4. The provided example is hardly an isolated case, but a fish in the sea of security threats. 5. Reducing external dependencies is generally beneficial. The toolz Heritage — Toolz 0.10.0 documentation <https://toolz.readthedocs.io/en/latest/heritage.html> seems to even reflect my point that these are core operations. On Wed, May 17, 2023 at 1:36 PM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
On Wed, May 17, 2023 at 2:22 PM Daniel Guffey <daniel.guffey@gmail.com> wrote:
I'm a bit dubious about the pypi suggestion as packages are being regularly poisoned with malware ( e.g. New KEKW malware infects open-source Python Wheel files via a PyPI distribution | SC Media (scmagazine.com) <https://www.scmagazine.com/news/devops/kekw-malware-infects-open-source-python-wheel-files> ) and support issues keep happening with package management tools.
This is an absurd complaint. For one, the PyPA dealt with that very quickly. But more relevantly, Toolz is a package with many years of development by well-trusted people. Yes, getting a brand new malware onto PyPI is a danger, but that's a completely unrelated issue than using well-established and signed packages from known people.
If you weirdly distrust PyPI, you can equally get the same thing via GitHub... I guess unless you also distrust those repos.
It's not absurd to suggest a new decorator for the standard library. But "I don't trust PyPI" isn't going to win you any support for the idea.
-- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On Thu, 18 May 2023 at 05:14, Daniel Guffey <daniel.guffey@gmail.com> wrote:
Apologies, I didn't mean to imply PyPI was inherently untrustworthy, unusable, or irrelevant. Clearly, it has a place and I use it for packages that I am familiar with and trust.
Then I would advise being careful how you word things, since what you said was a pretty clear recommendation against ever using it. ChrisA
The problem with "batteries included" is that what exactly is a battery is hard to define, and everyone has a different opinion about it. It's my observation that in recent (~10?) years, the core devs are pretty carfeul about including anything new in teh stdlib -- it has its upsides, for sure, but also downsides, tying maintenance, etc to the core library. YOu can read the various archives for the discussion. So the recommendation is usually to build a package, distribute it as you see fit (though anyway other than PiPy will get very little uptake), and if that demonstrates that it's useful, you can propose to ad it ot the stdlib -- that has the advantage of review and proof of usefulness. However, there is a challenge here -- a PyPi package with a single function (or decorator) will be hard to gain traction with (leftpad, anyone?) -- but there's a solution there, too. What works better is a package that's a collection of related tools. I'm not familiar with toolz, but it sounds like that's a good candidate -- if your idea is not already part of it, then I suggest you propose a contribution -- if it is well received, you'll be in a good position to propose an addition to the stdlib. -CHB On Wed, May 17, 2023 at 12:24 PM Chris Angelico <rosuav@gmail.com> wrote:
On Thu, 18 May 2023 at 05:14, Daniel Guffey <daniel.guffey@gmail.com> wrote:
Apologies, I didn't mean to imply PyPI was inherently untrustworthy,
unusable, or irrelevant. Clearly, it has a place and I use it for packages that I am familiar with and trust.
Then I would advise being careful how you word things, since what you said was a pretty clear recommendation against ever using it.
ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XI2JLN... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On 19/05/23 4:16 am, Christopher Barker wrote:
The problem with "batteries included" is that what exactly is a battery is hard to define, and everyone has a different opinion about it.
To my mind, "batteries included" means you can do *something* useful with it out of the box. It doesn't mean you can do *everything* you might want to do. -- Greg
On 2023-05-19 00:54, Greg Ewing wrote:
On 19/05/23 4:16 am, Christopher Barker wrote:
The problem with "batteries included" is that what exactly is a battery is hard to define, and everyone has a different opinion about it.
To my mind, "batteries included" means you can do *something* useful with it out of the box. It doesn't mean you can do *everything* you might want to do.
As Tim Peters has said: "Batteries included - but not nuclear reactors".
participants (7)
-
Chris Angelico
-
Christopher Barker
-
Daniel Guffey
-
David Mertz, Ph.D.
-
Greg Ewing
-
Lucas Wiman
-
MRAB