-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

Hello,

Kristin Brennan from Coverity has asked me for an interview about Python core development and how we are using Coverity Scan. Coverity is planing to have a monthly series of interviews with open source projects that use their service, for example http://www.coverity.com/company/press-releases/read/coverity-introduces-mont...

She has send me a list of questions up front. I like you to review and comment on my preliminary answers, please.

Thanks, Christian

Q: How many active developers do you have contributing to the project?

• • 174 according to http://hg.python.org/committers.txt
• • 152 according to http://bugs.python.org/user?iscommitter=1&@action=search&@sort=username&@pagesize=300
• • about 60 active according to https://www.ohloh.net/p/python/
• • about 360 contributors agreements (513 - 152) according to http://bugs.python.org/user?contrib_form=yes&%40action=search
• • about 1400 names in Misc/ACKS

Python Core Mentorship program and PyLadies have helped to attract new contributors.

Q: Why do you think you are able to achieve high levels of quality? ( than liked size commercial projects)

Python has an established and well working workflow. The majority of commits are accompanied by a ticket. Most bug fixes (except for trivial ones) and all new features are reviewed by other developers (Rietvield) before the patches are committed. Documentation updates, changelog entries and unit tests are usually part of a patch, too. Large features and modifications go through the PEP (Python Enhancement Proposal) process.

CPython core development heavily relies on automatic tests. We are using buildbot for continuous integration since at least 2006. About 40 buildbot instances to run 10k test cases on different of platforms and architectures: Linux (multiple distributions), Windows, Mac, BSD, even exotic operating systems like Solaris and AIX and hardware like PPC, MIPS, Sparc and Alpha (Snakebite).

CPython uses time based releases not feature releases. New features only land in the development branch, when they are stable and went through our review process. We are not under pressure to add "cool stuff" to increase our market share. Our goal is to provide a stable and slowly evolving foundation for our community. Revolutionary pieces of software are developed outside the core by other developers. Some of them are later integrated into the core when they are deemed mature and best practice.

Backward compatibility is also very important to us -- except when we break it deliberately with Python 3. New features are never added to patch level releases, too.

Most of Python is written in Python, too. It's much easier and less errorprone to maintain Python code than C code. The rest of Python is written in well structured ANSI C (C89) with a well designed C API and a strong focus on POSIX.

Q: What is it about the developers on your program that you think enables them to create high quality code?

All core committers are highly motivated and care deeply for Python. Although we are split up across lots of countries, cultures and time zones we are able to work together as a team very well ...

[any ideas?]

Q: What happens if you have a developer working on the project that submits code doesn't meet quality expectations?

It rarely happens as most changes go through a thorough review process before they are committed. Once in a while some issues slip through -- after all we are just humans. Since commits are tightly monitored such issues are pointed in a matter of hours, even minutes. Either the issue is sorted out as soon as possible or the commit is reverted.

Everybody is more careful in the vicinity of a new release, too.

Q: What sort of process do you follow to ensure high quality code?

Python has coding standards for C and Python code. Major chances to through the PEP process, other chances go through a review process. Stable APIs, ABIs, automated tests and continuous integration ... but also tedious bike shedding discussions on the mailing list.

[repetition of stuff I said before ...]

Q:Do you have people in formal roles to ensure the quality of the code?

In theory Python has a hierarchy:

Guido (Benevolent Dictator for Life) > release manager > expert for module or area of interested > core committer > contributor

In practice this hierarchy is never imposed upon somebody but rather used as a tool to aid the development process. Every core committer is responsible for her checkins and does her best to meet our demands in quality. She also helps contributors to improve their patches and teaches them Python's coding conventions and best practices. Experts for a module or topic are often included in the discussion to get their opinion and to benefit from their knowledge.

Q: Can you describe how development testing and the Coverity Scan service fits into your release process?

Coverity comes into play when the code base has stabilized and a new minor release is approaching its release candidate phase. Coverity is especially useful to find issues in unlikely code paths like error case that are not reached under ordinary circumstances. A stable code base makes it easier to find and fix the problematic code segments.

Recently I went through all untriaged Coverity issues and either fixed, closed or triaged them all. For the future I'm planing to fix or report issues as they are detected. Of course it depends on my free time...

Q: What tools do you use, besides Coverity, and how do they impact your ability to deliver high quality code

• • Roundup issue tracker
• • Rietveld code review
• • buildbot for CI
• • fusil for fuzzing tests and pyfailmalloc to add random malloc() failures (both created by Python core dev Victor Stinner)
• • GCC's gcov
• • clang analyzer (Brett ?)
• • instrumented Python builds (--with-pydebug) with extra checks, asserts and reference leak checks
• • ...

Most tools are written in Python

Q: What challenges do you face with regard to maintaining high quality code that are unique to open source and how do you overcome those challenges?

[Does anybody have an idea for a good answer?]

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQIcBAEBCAAGBQJR5y8AAAoJEMeIxMHUVQ1FjRYP/j9nhCk82JrBp8h3Sie9ryOx N2o5qX5xYWehjV7Y1iQXFK9s43VprB8xup6ZC09/1lwQKLjsnyk6QgLf9+RrDWin gaeG4CTwykuiASzU1xQJxdyj7YXDmGy/C5zIxpSNKVkLn7AwB/ob1+Tf7AAkeNHd z8C8xCihBYHgp8iY8k3Y7yuYpFuVr6+Qz8wtl/ubU76ys5IsPf5h0JxMfQh+oNpc +8TewXQsPnPlFvkdm9T0ecnAG7s6MCCqhEL/11kSx9LLKXEK6CtSnEEMgkWdDpmC bLeiM6UWs9hiimXKSnWEP/J3fXAFsq9M22jkmPDYSPIEtY0SoHHaHLdrb9uj0BUX jtM1c/I+gFeWO76RN6LJXqpMpoH8vGDeHnvP0jJbeMN+Ma6XSD01LD8hK0dPZ4lX H6p2VfoZCrj28XPLyClKLSryZ23kq1JHqxM6pOTYQEVUtmZLJlKkTn9GCIwZiLPd FRlzaZpmto/iCqfq2s9XSs9maddMUWbUrn+30MDvU+cl1nb+JKxNKF5nNxqc64tI xRUs+b819UmeMq8eQ82IVi4o1fkh+QMOqZO2hmZWwUR8j9S4SAYQGnbvYzHrAnrw JI9+xBm72z3e09cjHe2PZ3oG/hPCM9BargAmNEqYaI6hE0R1CTqbVenBAwGfWD2B 6Nu9Iki9skroWbo34rKX =hLXG -----END PGP SIGNATURE-----