On Feb 16, 2022, at 11:28 AM, Colin Dunklau <colin.dunklau@gmail.com> wrote:



On Feb 16, 2022, at 11:41 AM, Jean-Paul Calderone <exarkun@twistedmatrix.com> wrote:



On Fri, Feb 11, 2022 at 5:36 PM Jean-Paul Calderone <exarkun@twistedmatrix.com> wrote:
Hello all,

I think it would be lovely if trial caught up to the last decade of advances in coverage measurement technology.  I think this means integrating with coverage.py - probably the hands-down leader in Python coverage technology for at least the last 10 years, if not more - instead of the stdlib "trace" module which is ... something else.  Or maybe there's an even better option out there somewhere - it would be amazing if all of the trial-based test suites out there got whatever the best current option is - why should ever project have to figure this out for itself?

When was the last time anyone ran trial --coverage on purpose?  Did they realize they were choosing the bad option?

I know that you can hack around this situation roughly like this:

python -m coverage run -m twisted.trial ...

but this has some shortcomings.
  1. If trial --coverage exists shouldn't it be the good option?
  2. python -m coverage run -m twisted.trial -jN ... is a bad time.  How about some coverage measurement that's multi-core friendly?  It's a real drag going from a 30 second no-coverage test run using 16 cores to a 15 minute coverage-measuring run on a single core.
Presumably trial -jN uses us processes, so https://coverage.readthedocs.io/en/6.3.1/subprocess.html is worth a read. 

Best practice today is, at least in your CI:

  1. pip install coverage_enable_subprocess
  2. set COVERAGE_PROCESS_START env var to your coveragerc
  3. trial -j
  4. `coverage combine` after running

This works for trial (and also works for any other tool that you might want to support coverage).

One of the reason I think there’s been relatively little movement on this obvious flaw in Twisted is that this is fairly easy to set up. That also means that it’s fairly easy to implement within Trial though, were someone interested.

Does anyone agree that this is something short of an ideal situation?  Is anyone interested in helping address it?

This should definitely be addressed.  Ideally, trial would both supply its own `--coverage` option (largely in order to have somewhere to hang the documentation in `--help`) and also Just Work under `python -m coverage run` or `coverage run`.

Jean-Paul

Anyone?

I don’t think I can commit to actually doing this, but if I find some time it’s definitely something I’d love to have resolved.  I can definitely do some quick code reviews, if someone else wants to make the implementation happen.

_______________________________________________
Twisted mailing list -- twisted@python.org
To unsubscribe send an email to twisted-leave@python.org
https://mail.python.org/mailman3/lists/twisted.python.org/
Message archived at https://mail.python.org/archives/list/twisted@python.org/message/7IHGMNFFEJE3NDQRNY3TTMTO3PTYZUVK/
Code of Conduct: https://twisted.org/conduct
_______________________________________________
Twisted mailing list -- twisted@python.org
To unsubscribe send an email to twisted-leave@python.org
https://mail.python.org/mailman3/lists/twisted.python.org/
Message archived at https://mail.python.org/archives/list/twisted@python.org/message/T6JSDDVCGZCT62GNHKOD4MFOSG33AF6Q/
Code of Conduct: https://twisted.org/conduct