[Tutor] Standard Library Performance (3.4.1)

[Adam Jensen]

On 10/24/2014 08:01 AM, Stefan Behnel wrote:
> Alan Gauld schrieb am 24.10.2014 um 13:03:
>> Not all library modules are C based however so it doesn't
>> always apply. But they are usually optimised and thoroughly
>> debugged so it is still worth using them rather than building
>> your own.
> 
> It's worth stressing this point a bit more. Lots of people have been using
> this code, and some found bugs in it that are now fixed. This means that
> whatever implementation of similar functionality you can come up with
> yourself will most likely have more bugs and be less generally versatile.
> And even if the standard library code doesn't fit your needs, start by
> taking a deep look at what the Python Package Index (PyPI) offers instead
> of writing your own.

If the standard library is thoughtfully designed, tested, tuned and
released in a stable way then I agree, it's worth spending development
time to search and explore the library for potentially useful components
then examine the component's interface, decipher the documentation and
tinker with the component to discover its characteristics. There is some
cost in doing this (developer time and energy). Sometimes it seems like
there will be less expense in just writing a few lines of pure python
but a ~10x difference in performance settles that - the library route is
probably worth the effort.

> Developer time is much better spent reusing other people's code and helping
> to squash the remaining bugs in it than having everyone write their own
> buggy code over and over.

I haven't studied the PyPI engineering process yet but my limited
experience with just installing third party packages (e.g., pip install
scipy) makes me a bit wary about naively using these packages as
foundational components.

BTW - I did manage to get all of the scipy dependencies built and
installed but damn, that was a grueling process. I'm surprised it
wasn't automated and thoroughly shocked by the number of compilation
warnings. (I once heard someone describe the current state of software
engineering as being like "building a skyscraper out of bananas").