[Python-ideas] The stdlib++ user experience (Was: Introduce `start=1` argument to `math.factorial`)

[Andrew Barnert]

On Sep 19, 2014, at 8:25, Paul Moore <p.f.moore at gmail.com> wrote:

> On 19 September 2014 16:15, Andrew Barnert <abarnert at yahoo.com> wrote:
>> There's more to it. Many of the most important packages are doing something that didn't have a category until they invented it. For example, what is BeautifulSoup exactly? Originally, IIRC, it was intended to be a more lenient HTML parser. But nowadays it doesn't even do the parsing itself; it drives a stdlib or other third-party parser. It's sort-of a better (than what?) DOM interface for HTML and XML, which is something no one realized they needed until it existed. If the Python community had the kind of less-fanciful, easier-to-find, specific names that, say, perl has, what would it be called? HTML::Parsers::TagSoupParser?
>> 
>> Also, the boring names tend to mean that, when there _is_ an obvious category, the first entrant stakes out the best name, and everyone else ends up with just less-felicitous variants of the same name. The fact that SleekXMPP and Wokkel replaced Jabberpy as the most popular XMPP libraries means that it's pretty easy for me to recommend SleekXMPP over Jabberpy. In another language, you have to tell people to use libXMPPClient or XMPPLib instead of libXMPP.
> 
> I wasn't trying to say that "boring" names were better - far from it,
> quirky names are part of the Python culture. But there isn't anything
> stopping Beautiful Soup adding keywords like "html parser scraping" or
> something like that. The trick is picking keywords that people will
> search for, which is often hard (and that's probably why the keyword
> metadata is underused). Having a registry of commonly-used keywords,
> and a user interface that means you don't have to go online and search
> when you're writing your app, might help. Think of something like
> Stack Overflow's auto-completed tags.
> 
> I don't have any good solutions here, I just think it *should* be
> possible for the author of Beautiful Soup to make it clear to
> potential users that it might be what they are looking for when they
> think of <insert relevant task here>. As somebody said earlier in this
> thread, think of it as search engine optimisation for projects.

That's a good point. BeautifulSoup, lxml, requests, Scrapy, Mechanize, and pyv8 certainly don't belong in any sane category together, but they also clearly share in common the fact that they're useful for (different but overlapping kinds of) scraping projects, and a keyword-influenced search (as opposed to keyword browsing) seems like the right way to get that across.

The problem is that search tends to work best when, even if you don't know what you're looking for, you'll easily recognize it when you see it. But if I'm searching for scraping, is Mechanize or BeautifulSoup better? That question doesn't just not have an answer, it doesn't even make sense. I pretty much have to look at what they both do to figure out which one fills the hole in my scraping project.

Which means that "search is the answer" tends to conflict with the earlier goal from this discussion of "just tell me the best answer, don't make me research it". That may just be because that goal is impossible to do well, and making it a lot easier to research by giving you pointers at a few different things that you are likely to want to look into.

Or it may mean that "find the most stable implementation of frequently-implemented simple thing X" (like a console-mode progress bar) and "find me good tools to help with complex thing Y" (like scraping web sites) are inherently different problems, and there might be a solution that's _helpful_ for both, but anything that's perfect for one will be useless for the other.