Why functional Python matters

[Dave Benjamin]

In article <mailman.1050438685.19213.python-list at python.org>,
Jp Calderone wrote:
>   While they take getting used to, you can't say that list comprehensions
> aren't functional.  They're not traditional-lisp functional, but that isn't
> the only kind :)

Certainly not, especially considering where they supposedly came from. I
think that they are certainly functional, but only representative of
particular (but common) functional idioms related to list processing.

For instance, there has yet to be a dictionary comprehension. This is
something I think would be nice to add, because I currently do things like
this: 

dict([(x['key'], x['value']) for x in data])

Where it might be nice to say instead:

{x['key']: x['value'] for x in data}

But I'm getting off track here. I'm not arguing that we should have
map/filter instead of list comprehensions, or the other way around. I think
both are useful and operate on different levels, similar to the relationship
between relational algebra and relational calculus. Just because they can
both do the same things doesn't mean one should be abolished in lieu of the
other.

And in a broader sense, I'm not even talking about built-in functions or
language constructs so much as the direction of Python's evolution. I don't
necessarily think that Python needs to become another LISP, though I agree
with Paul Graham on many of his observations of language evolution. I think
that Python is just fine being Python, but I don't want to see it become
another Java either. The day Python tells me "top-level functions have been
deprecated" is the day I find a new pet language. ;)

>> [snip - my plea for lambda love]
>
>   No worries, there are enough people who would shout loudly enough were
> this to be removed that its existence is safe, at least for the near future.
> 
>   Just make sure you give a holler anytime someone suggests discarding them.

Well, I suppose that's what I'm now attempting to do. =)

>   Lambdas are an excellent tool for obfuscation, that's for sure.  You're
> right not to recommend overusing them in production code.  Generally, I try
> to limit myself to the simplest expressions, on the order of "x and y" or
> "result[index]", and translate to fully fledged functions for larger
> expressions.  It's just easier to maintain.

This has certainly crossed my mind, and I understand that this is one of the
main reasons why Guido dislikes the construct. He seems to put readability
at a very high priority, which is why Python is often so elegant. But
readability is one of many things that are important to me. Another is
mathematical conciseness. If you have represented an idea very formally and
concisely, you can always expand it so that all functions and variables are
named and traditional if/while blocks are used. Doing the reverse is often
much more difficult.

For an even greater exercise in obfuscation, see my functional types recipe.
Warning: you may lose respect for me after you read this. ;)
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/161403

I have successfully produced a custom XML-generation script that translates
a Java-based object model into XML using Jython. I began by writing only
lambdas, and almost all of them were one-liners. Then, I turned the lambdas
into proper "def"ined functions. Then, I observed the patterns in argument
passing to determine what data could be shared, and used this observation to
determine what classes to create. When I was done, I had something that
looked like it had been an object-oriented design from the start, but it was
very logical, concise, fast, and readable. On top of all of that, I had
*confidence* that my code worked properly because it was designed from
small, easily-defined parts that I could (attempt to) prove the correct
operation of. Proving the correctness of objects isn't so easy.

>   Also, note that lambdas *aren't* side-effect free, because Python isn't
> side-effect free.  You can write ones that are, but then again, you can
> write functions with "def" that are side-effect free too.

This is true. And you can write classes that are side-effect free, too. The
string object is a perfect example. It's also an example of how performance
can suffer when you enforce being side-effect free (the += operator). It's
not cut-and-dry by any means. However, because of the way Guido designed
lists and dictionaries, mutators typically return None, so it's awkward at
best to write lambdas that use them. This encourages side-effect-free lambdas.

See this page for a good laugh:
http://p-nand-q.com/lambda.htm

>   Statements are, by definition, things which cannot appear in lambdas ;)
> No worries here, I think.

Except that there are some that consider lambda "broken", and that it should
either support statements or be removed entirely. This I disagree with.

>   Luckily, map() and filter() are implementable in terms of list
> comprehensions.  So even in Python3k, when they're finally removed from the
> language, you can implement them yourself in Python, and gain the
> performance benefits of using a JIT on python bytecode ;)

So they are scheduled to be removed! I knew it! =) This is why I'm paranoid!

>   The inverse of zip is already available (and with one less character than
> unzip! ;).  x == zip(*zip(x)) (ignoring sequence types, of course).

Cool! Thanks for the tip...

>   I don't think Python is headed the way of Unlambda, don't worry :)

Hehe... it does need a COME FROM construct though. Who wants to help me
write the PEP for that one?

>   What I think is an important distinction to make here is the language vs
> the library.  Right now, there are overlapping language features.  I'd much
> prefer it if some of the things, even the useful things like map(), were
> re-implemented in terms of other Python features and moved out of the
> language and into the library.  This gives the best of both worlds - a
> language simple to learn and use that also provides an expressive set of
> features through its stdlib.

I agree with your premise, but I've been feeling lately that there isn't
such a clear distinction between language and library. For instance, when
the string object and the string module merged, the main difference was that
my sentences were rearranged and I didn't have to say "import string"
anymore. But I never really thought in terms of a string *library*, so on the
surface it was part of the language all along.

If I just have to say "from functional import *" to get an environment
friendly to FP, I will happily quiet down and mind my own business. I just
want to make sure FP is being represented. ;)

Thanks for your reply,
Dave