[Python-ideas] A tuple of various Python suggestions

[Nick Coghlan]

On 9 April 2016 at 07:15, Keith Curtis <keithcu at gmail.com> wrote:
> Hi all,
>
> I just discovered this alias and I thought I'd post a few ideas. I
> wouldn't call myself a Python master yet, but it's an amazing language
> and my biggest wish is that it was more widely used in the industry.
>
> Here are a few suggestions:
>
> 1. Decrease the bug count. I recently noticed that there are about
> 5,400 active bugs in http://bugs.python.org/. That surprised me
> because I almost never see anyone complain about bugs in Python
> (compared to the number who complain about bugs in LibreOffice,
> graphics drivers, Gnome / KDE, etc.)
>
> There are a lot of people on this list, and if the brainpower can be
> applied to practical, known, existing problems, it is a great way to
> improve Python while also considering more exotic ideas. I can also
> suggest making a pretty graph of the bug count and putting it on the
> front page of python.org for greater visibility.

Folks in institutional environments can most readily contribute to
this by obtaining Python from commercial redistributors (rather than
using the free community versions and then expecting free ongoing
support from volunteers), and then funneling bug reports through their
vendor rather than filing them directly with upstream (since donating
bug fixes to commercial organisations for free isn't something most
people consider a fun hobby).

> 2. Python is somewhat popular on servers, and there is a lot of
> potential for more. WordPress is easy to use and powerful, but lots of
> people don't want to program in PHP. Or Javascript, Java, Ruby, etc.
>
> Codebases like Whoosh full-text search
> (https://bitbucket.org/mchaput/whoosh/wiki/Home) are important, but
> have minimal dev resources as most people are using Lucene /
> ElasticSearch. The common choice is between 1.3M lines of Java:
> https://www.openhub.net/p/elasticsearch, containing 1100 todos and
> 1000 references to "deprecated", or 41K lines of Python written mostly
> by one person.
>
> Hadoop is another big Java project (1.9M lines), and there is even an
> ecosystem around it. Python interoperates with Hadoop, but it should
> be possible to build a radically simpler framework that provides the
> same functionality using Python-native functionality and without all
> the baggage. Hadoop has several interesting sister projects: a
> distributed database, scalable machine learning, a high-level data
> flow language, a coordination service, etc. I'm sure you'd build
> something smaller, cleaner, faster in many cases, more reliable, etc.

This is about business dynamics and institutional supply chain
management, not software, so wishing won't make it so. However, given
the business challenges facing the vendors behind the Java projects
you mention here, Python-based alternatives are also going to be a
tough sell to potential investors.

(That said, you may also be interested in the Apache Spark project,
where Python, Java, Scala and R are the top tier analytics development
languages)

> 3. It was a sad mistake that Google picked Java over Python for
> Android. However, there is now a great program called Kivy which
> allows people to write apps for IOS or Android with one codebase, but
> it could also use more resources, as for example it doesn't fully
> support Python 3.x yet.
>
> There are 10s of thousands of bugs in the popular Python libraries and
> I would fix those before proposing more language changes.

People work on things in their own time because they find them
enjoyable (or otherwise inherently rewarding), not because they're
interested in facilitating increased corporate adoption.

That said, yes, while it's high profile, contributing to CPython is
one of the *least* effective ways of helping to improve the overall
Python ecosystem in the near term, as it can literally take years to
roll out major changes (as Python 2.6 is still the de facto baseline
version, and there are many situations where folks are still using
Python 2.4 and earlier). The best cases are those where we can define
new APIs, protocols and idioms in ways that can also be adopted in
earlier versions of the language by way of third party libraries and
standard library backports.

> 4. I enjoy reading about the Python performance improvements, but it
> is mostly a perception problem with all the existing workarounds.
> Gnome wrote version 3 of their shell in Javascript because they didn't
> think Python would be fast enough. Lots of people write Node because
> it's compiled and "fast". I suggest taking some of the effort working
> on performance, and spend it on evangelizing to other programmers that
> Python / Cython / PyPy, etc. are already good enough! There are a lot
> of programmers out there who would be happier if they could work in
> Python.

There's nothing stopping anyone interested in this area from working
on any kind of evangelisation they want to. However, what harm does it
cause us personally if people decide to use other programming
languages? Python tautologically fits the brains of Pythonistas, but
that's nowhere near being the same thing as it being the right
language for everyone for every purpose.

It's also the case that any developer with only one language currently
in their toolbox (even when that language is Python) is a developer
with lots of learning opportunities ahead of them:
http://www.curiousefficiency.org/posts/2015/10/languages-to-improve-your-python.html

> 5. It would be great to get Python in the web browsers as an
> alternative to Javascript. There are a number of projects which
> convert Python to Javascript, but this would be more direct.
> LibreOffice ships with a Python interpreter, why can't Firefox and
> Webkit? ;-) Obviously there are interoperability issues, but it would
> be great to just side-step all the complexity of Javascript (Here is a
> server-oriented article, but it gives a flavor:
> http://notes.ericjiang.com/posts/751) This might sound like a crazy
> idea, but the engineering problems aren't that hard.

This misunderstands the nature of the relationship between JavaScript,
CSS and the HTML Domain Object Model: these are technologies that have
co-evolved for describing and dynamically updating user interfaces
that interact with remote services over a network, and they're
*really* good at it. Replicating that ecosystem in other program
languages would technically be possible, but there's little incentive
to do so given the work on WebAssembly and ongoing improvements in
transpilers.

> 6. In a few cases, there are two many codebases providing the same
> functionality, and none of them are really doing the job. For example,
> the de-facto MySQL Python interop library
> (https://pypi.python.org/pypi/MySQL-python) only supports Python 2.x
> and appears to be abandoned. There are several other libraries out
> there with different features, performance, compatibility, etc. and
> it's kind of a minefield for what should be a basic scenario. It takes
> leadership to jump in and figure out how to fix it and make one
> codebase good enough that the others can switch over.

This is why people pay open source redistributors to ensure they have
access to commercially supported components, rather than trusting that
whatever they happened to find on the internet will continue to be
maintained by anonymous benefactors.

> 7. Focus more on evolving the libraries rather than the language. I've
> recently discovered Toolz, which has a more complete set of functional
> language methods. I think some of them should be included in the
> official versions. A lot of people don't think Python is good enough
> for functional programming and this would help. These new routines add
> complexity, but a newbie doesn't need to write in a functional way, so
> it obeys the "only pay for what you use" rule.

"Some of them should be included" is not an actionable proposal.
Anyone is free to propose specific additions to functools, and make
the case for why those particular ones should be included in the
standard library rather than continuing to be accessed via version
independent 3rd party libraries like Toolz. (Advance warning: "I want
a purely functional solution to a problem that can already be readily
solved with procedural code" generally isn't accepted as a compelling
justification)

A simpler possibility might be to review the Functional Programming
HOWTO at https://docs.python.org/3/howto/functional.html and consider
ways that that might be updated to reference third part libraries, as
well as potentially made more discoverable via the functools and
itertools reference documentation.

> There are a number of under-staffed libraries and frameworks. I see
> people complain about the YAML parsing library being unmaintained, the
> default HTTP functionality being difficult and limiting, poor SOAP
> support, etc. There are a million ways to improve the Python ecosystem
> without making any language changes. You don't have a big rich company
> who can pay for thousands of full-time developers working on
> libraries, but the bug reports are a great way to prioritize.

What makes you think anyone here has the authority to tell anyone else
how to spend their time?

Folks work on community open source projects based on their personal
interests and their commercial interests. While you're right that lots
of things could stand to be improved, the best people to complain to
about underinvestment (or misdirected investment) are commercial
redistributors selling supported versions of CPython and other
community projects in the Python ecosystem, rather than the already
overcommitted volunteers contributing their own time for their own
reasons.

> 8. I've yet to find a nice simple free IDE with debugging for Python.
> I use Atom, but it has primitive debugging. I tried PyCharm but it's
> very complicated (and not free, and Java). I use Jupyter sometimes
> also but I'd prefer a rich client app with watch windows, etc.

Not the right list for that question.

> 9. It would be interesting to re-imagine the spreadsheet with a more
> native Python interface. Pandas and matplotlib are great, but it would
> be cool to have it in LibreOffice Calc that supports drag and drop,
> copy and paste, can read and write ODS, etc. (Also, LibreOffice Base
> is basically unmaintained. I think if 10 Python programmers passionate
> about databases and GUIs showed up, it could re-invigorate this dead
> codebase.)

Also not the right list.

> 10. Being simple to learn and powerful is very hard. Fortunately, you
> can break compatibility every 10 years. My only suggestion is to get
> rid of the __self__ somehow ;-)

Methods are just functions, so this is never going to happen. (When
folks understand why, they've generally made a decent step forward in
appreciating the differences between a procedural-first usage model
for a language, vs an objects-first one)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia