On 19 September 2014 21:59, Steven D'Aprano steve@pearwood.info wrote:
Now obviously there are "best of breed" third party libraries. I don't see many people trying to build a better BeautifulSoup. (Maybe they are, and I just don't know about them, which demonstrates the problem.) That's not necessarily a bad thing. But I think we should be careful of putting the thumb on the scales too much.
The problem is that *failing* to provide recommendations can induce analysis paralysis in newcomers, such that they decide a less mature, more centralised ecosystem with fewer choices is "easier", or even "more powerful". While the "easier" can be accurate (since there are fewer choices to make), the "more powerful" is generally. It's just that the discoverability problem is *much* easier to solve when a language community is dominated by a single use case or a single major vendor, so providing opinionated guidance becomes less controversial. This can lead to comparing one language's entire ecosystem to another (less familiar) language's standard library, rather than comparing the full strength of both ecosystems.
As an example of how a default recommendation can help address the discoverability problem, these days, if someone is completely new to Python web development, I will tell them "start with Django, and use djangopackages.com for package recommendations". However, I will also tell them that Django isn't necessarily the best fit for everything, so sometimes something like Flask or Pyramid (et al) might be a better choice.
The reason I do it that way is that Django is far more opinionated than most other Python web frameworks and makes a lot more decisions *for* you. Experts have legitimate reasons for debating several of Django's choices, but newcomers don't yet know enough to understand when and why you might want to do something differently. Recommending a framework that makes those choices on their behalf is actually helpful at that point, even if it means they'll be missing out on some other cool libraries (at least for the time being).
This approach provides an experience far closer to the modern Ruby web development model where Rails is the default choice (by a long way), and folks only later start considering the use of something less opinionated like Sinatra for problems where an all-inclusive solution like Rails is less appropriate.
Providing "pip" by default is the same way - there are actually times when it isn't the best choice for solving particular packaging related problems, but it's always enough to get you started, *and* it lets you more easily bootstrap other tools like conda and zc.buildout when they're a better fit.
The goal in providing "default recommendations" is thus to minimise the "time to awesome" for newcomers, while gently nudging them in the direction of good development practices, rather than raising barriers in front of them that say "you must first learn how to make this apparently irrelevant decision before you can continue". And if someone never needs to tackle problems that require moving beyond the default recommendations? That's fine - there are an awful lot of people happily solving problems without moving beyond what the standard library or their redistributor provide.
You also see this pattern with some of the design guidance we give newcomers like "don't use metaclasses" and "don't use monkeypatching (except as part of a test mocking library)". What we actually mean is "these are power tools, and genuinely hard to use well. By the time you're ready to wield them, you'll likely also have realised that the implied caveat on the standard advice is 'unless it's the only way to solve a specific problem'". That's a complex mouthful to inflict on someone that is still learning though, so we just oversimplify the situation instead (even though the end result is so oversimplified as to technically be a lie).
My experience is also that the scientific community appear to be *far* more pragmatic about library choices than the professional development community (or anyone that is interested in programming for its own sake, rather than what it lets us do). We're a bit more prone to looking under the hood and deciding we don't like a library because of how it's made - competing libraries may arise based on differences in opinion regarding good design and development practices, rather than fundamental limitations in what a library makes possible. Scientists and data analysts are far more likely to just grab a library because of what it lets them *do* (or communicate), without looking too closely at how its put together.
That greater level of pragmatism then seems to make it easier for category killers to arise at the library level. However, I have to admit this particular idea is a speculative hypothesis, rather than something that has been subjected to rigorous objective analysis. Anyone looking for a PhD topic in sociology? :)
Regards, Nick.
P.S. As far as I can tell, the relative ease with which dominance can be asserted is also why VCs tend to favour newer languages with more centralised ecosystems - it's easier for them to assert control, and attempt to "punch the ticket" if the language achieves future success. It's much harder to do that for a massive decentralised sprawl like the Python community.