Libraries and Standardization
tl;dr: Two of us think it'd be good to collaborate even more and standardize on more things between projects. We'd like your feedback.
So Claudiu and I had some time to chat while we were at PyCon. I think much of the discussion was spurred by https://github.com/PyCQA/meta/issues/25 and I think we came to some solid agreement on things we'd like to see the PyCQA become and do.
We covered two main topics:
1. The joking name that I chose for our tiny organization is causing confusion for others. For example, we have several tools that we maintain that contribute to improving and maintaining people's code quality. However, people often see two or more of those tools as exclusive or as conflicting (e.g., pylint and flake8, pylint and pycodestyle)
2. There are a lots of things that we all do similarly but just slightly differently. It would be *fantastic* for our users if they didn't have to learn 15 ways of doing things (I'm exaggerating of course).
An Authority should be an Authority ... or at least act as something approximating a single unit
The idea here is that we should do our best to explain to users that there's no animosity between pylint and flake8. In fact, Flake8 runs Pylint everytime we run our tests. Just like flake8 runs itself against its own code base as well as Pylint, we should have a cohesive story to users. There's lots of junk on the internet about why to use one over the other.
I proposed that we might want to sell users on an approach like so: If you've never used one of these tools before, start small. Flake8 seems to be slightly easier for folks to get started with. Let's suggest that as a first step kind of like sticking your toes in the water to determine the temperature. Once you're happy with that and you want something a little more advanced and powerful, start using Pylint. Continue using both. They overlap but they don't entirely coincide.
You learned A, I learned B, they learned C, let's all learn from each other
The next step would be aggregating everyone's learnings. Pylint has been around for nearly 2 decades (if I remember correctly) and Flake8 has been around for almost a decade (maybe it's been a decade? I don't even know anymore). We have more and more tools. We all use configuration files and we all have slightly different behaviour with respect to how we handle them, find them, etc.
We could all standardize and document a standard for the organization. In my opinion, that standard should be pyproject.toml (c.f., https://snarky.ca/clarifying-pep-518/). INI + ConfigParser is terrible. TOML, while I don't like the fact that it hasn't been properly specified or standardized, is at least a bit more flexible. Further, the packaging community has chosen this and is unifying on it. The PEP even specifically suggests people being able to use sections named "tool.pylint" or "tool.flake8". It looks close enough to INI that I don't think it'd be a pain for users to migrate.
I think as we make this migration, we should also consider migrating to a simpler configuration structure - namely eliminating User configuration (either in $HOME or in $XDG_CONFIG) as well as system level configuration.
It makes sense that if we're standardizing on all of this for users, we can also build libraries for standardizing things in the org: ideas of some low hanging-fruit:
- Multiprocessing (-j/--jobs for concurrency) - Configuration file management (e.g., format, parsing, locating, and merging) - Naming error codes (e.g., standardizing on how pylint names things) - Standardizing on configurability of running versus ignored checks - And a few others
Naturally, this is just the things we discussed and we're by no means the people making final decisions here, so we'd like you to weigh in.
Ian Stapleton Cordasco