[Python-Dev] Python profiler and other tools

[Guido van Rossum]

On Sat, Jul 24, 2010 at 8:29 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sun, Jul 25, 2010 at 3:30 AM, Guido van Rossum <guido at python.org> wrote:
>> On Fri, Jul 23, 2010 at 4:20 PM, Alexander Belopolsky
>> <alexander.belopolsky at gmail.com> wrote:
>>> There must be a good reason why traditional software development tools
>>> such as debugger, profiler and coverage are mostly neglected in
>>> python.
>>
>> Most such tools are probably better developed outside the standard
>> library. There are many reasons for that, but I would guess that it
>> mostly has to do with very different development cycles for tools.
>> Tool development usually goes in quick bursts (driven by urgent needs
>> and perhaps improving tool developer skills) unrelated to the stdlib
>> release cycle (and releasing the stdlib separately won't make much of
>> a difference). Also tools often have rough edges, and the stdlib
>> (except for the stuff added before 1995 or so :-) has a high
>> perfectionism standard.
>
> Part of me agrees with you regarding the generally different tool
> lifecycle, but another part says we need these tools in the standard
> library or we risk inadvertently breaking the hooks they would still
> rely on, even as third party projects.

The hooks need to be exercised by unittests, for sure. Hooks are APIs
with a contract and a promise of stability (though sometimes it's
complicated).

> I suspect a major factor here relates to the degree of unit test
> coverage for these components - for areas that aren't of direct
> interest to me, I'll still deal with it if something I do breaks that
> code's unit tests, but if the tests don't start failing I'll never
> even think to check for an incompatibility. (e.g. I seem to recall
> line tracing for with statements and/or generator expressions being
> broken for a while before we fixed it)

Sure, but I don't see how the presence of a tracing tool in the stdlib
would have helped -- in fact, there's pdb which does use the tracing
hooks, but that was a new use case and nobody thought of checking it.

> PEP 306 (the one about changing the grammar) and PEP 339 (which
> includes a short section on changing the bytecode definition) may also
> be missing some steps to make sure that any syntax and opcode changes
> are correctly covered by the disassembly, debugging, profiling and
> tracing tests.

That's a good idea regardless.

-- 
--Guido van Rossum (python.org/~guido)