[Python-ideas] Partial operator (and 'third-party methods' and 'piping') [was Re: Function composition (was no subject)]

[Nick Coghlan]

On 12 May 2015 at 08:12, Gregory Salvan <apieum at gmail.com> wrote:
> Sorry the fun part: the more you write code the less you have to write
> tests.

I think this is the key for the folks hoping to make the case for
increased support for function composition in the future (it's
definitely too late in the cycle for 3.5): focus on the *pragmatic*
benefits in testability, and argue that this makes up for the *loss*
of readability. "It's easier to read" is *not* a true statement for
anyone that hasn't already learned to think functionally, and "It is
worth your while to learn to think functionally, even if it takes you
years" is a very *different* statement.

The human brain tends to think procedurally by default (presumably
because our stream of consciousness is typically experienced as a
linear series of events), while object oriented programming can
benefit from analogies with physical objects (especially when taught
via robotics or other embodied systems), and message passing based
concurrent systems can benefit from analogies with human
communications. By contrast, there aren't any easy "interaction with
the physical world" analogies to draw on for functional programming,
so it takes extensive training and practice to teach people to think
in functional terms. Folks with a strong mathematical background
(especially in formal mathematical proofs) often already have that
training (even if they're only novice programmers), while the vast
majority of software developers (even professional ones), don't.

As a result, I think the more useful perspective to take is the one
taken for the PEP 484 type hinting PEP: positioning function
composition as an advanced tool for providing increased correctness
guarantees for critical components by building them up from
independently tested composable parts, rather than relying on ad hoc
procedural logic that may itself be a source of bugs. Aside from more
accurately reflecting the appropriate role of function composition in
Pythonic development (i.e. as a high barrier to entry technique that
is nevertheless sometimes worth the additional conceptual complexity,
akin to deciding to use metaclasses to solve a problem), it's also
likely to prove beneficial that Guido's recently been on the other
side of this kind of argument when it comes to both type hinting in
PEP 484 and async/await in PEP 492. I assume he'll still remain
skeptical of the value of the trade-off when it comes to further
improvements to Python's functional programming support, but at least
he'll be familiar with the form of the argument :)

On the "pragmatic benefits in testability" front, I believe one key
tool to focus on is the Quick Check test case generator
(https://wiki.haskell.org/Introduction_to_QuickCheck1) which lets the
test generator take care of determining appropriate boundary
conditions to check based on a specification of the desired externally
visible behaviour of a function, rather than relying on the developer
to manually specify those boundary conditions as particular test
cases.

I personally learned about that approach earlier this year through a
talk that Fraser Tweedale gave at LCA in January:
https://speakerdeck.com/frasertweedale/the-best-test-data-is-random-test-data
& https://www.youtube.com/watch?v=p7oRMB5V2kE

For Python, Fraser pointed out http://xion.io/pyqcy/ and Google tells
me there's also https://pypi.python.org/pypi/pytest-quickcheck

Gary Bernhardt's work is also worth exploring, including the
"Functional Core, Imperative Shell" model discussed in his
"Boundaries" presentation
(https://www.youtube.com/watch?v=yTkzNHF6rMs) a few years back (an
implementation of this approach is available for Python at
https://pypi.python.org/pypi/nonobvious/). His closing keynote
presentation at PyCon this year was also relevant (relating to the
differences between the assurances that testing can provide vs those
offered by powerful type systems like Idris), but unfortunately not
available online.

Andrew's recommendation to "approach via NumPy" is also a good one.
Scientific programmers tend to be much better mathematicians than
other programmers (and hence more likely to appreciate the value of
development techniques based on function composition), and the rapid
acceptance of the matrix multiplication PEP shows the scientific
Python community have also become quite skilled at making the case to
python-dev for new language level features of interest to them :)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia