From rob.a.mcdonald at gmail.com Sun Aug 2 02:16:52 2020 From: rob.a.mcdonald at gmail.com (Rob McDonald) Date: Sat, 1 Aug 2020 23:16:52 -0700 Subject: [pytest-dev] Test discovery Message-ID: According to Google, 'How does pytest discover tests' is an oft-asked question. However most of those questions actually want to know 'What criteria does pytest use to discover tests'... I'm working on a program that needs to scan a directory structure for Python code and then execute some of that code -- much like pytest does. So, I'd like to know _how_ does pytest discover (and run) tests. I've searched around in the source for where this happens, but I haven't had any luck. Much appreciated if someone can point me to the relevant bits of code? Otherwise, any pointers on best practices and functions/methods to use for this sort of thing are appreciated. Best, Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at the-compiler.org Mon Aug 3 05:20:00 2020 From: me at the-compiler.org (Florian Bruhin) Date: Mon, 3 Aug 2020 11:20:00 +0200 Subject: [pytest-dev] Test discovery In-Reply-To: References: Message-ID: <20200803092000.5q4t65ztciv5pd73@hooch.localdomain> Hey Rob, On Sat, Aug 01, 2020 at 11:16:52PM -0700, Rob McDonald wrote: > According to Google, 'How does pytest discover tests' is an oft-asked > question. However most of those questions actually want to know 'What > criteria does pytest use to discover tests'... > > I'm working on a program that needs to scan a directory structure for > Python code and then execute some of that code -- much like pytest does. > So, I'd like to know _how_ does pytest discover (and run) tests. > > I've searched around in the source for where this happens, but I haven't > had any luck. Much appreciated if someone can point me to the relevant > bits of code? Otherwise, any pointers on best practices and > functions/methods to use for this sort of thing are appreciated. Most of it happens in pytest's internal "python" plugin: https://github.com/pytest-dev/pytest/blob/master/src/_pytest/python.py For a simpler example how pytest's collection tree works, see: https://docs.pytest.org/en/latest/reference.html#collection-hooks https://docs.pytest.org/en/latest/example/nonpython.html Finally, you might be interested in Oliver Bestwalter's talk "Abridged metaprogramming classics - this episode: pytest": https://github.com/obestwalter/abridged-meta-programming-classics There, Oliver implements a small subset of pytest's "magic" from scratch. Note, however, that there are various utility functions available in the stdlib (inspect, importlib, pkgutils) which make those kinds of tasks easier, at least with Python 3.5+. However, both pytest and Oliver's talk avoid them (because of Python 2 support and because it was a fun limitation, respectively). For a more straightforward implementation, you can roughly: - Use pkgutil.walk_packages to find modules - Use importlib.import_module to import them - Use inspect.getmembers to get e.g. functions I do the first two in my own project here: https://github.com/qutebrowser/qutebrowser/blob/master/qutebrowser/extensions/loader.py Florian -- me at the-compiler.org (Mail/XMPP) | https://www.qutebrowser.org https://bruhin.software/ | https://github.com/sponsors/The-Compiler/ GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc I love long mails! | https://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From rob.a.mcdonald at gmail.com Wed Aug 5 12:34:26 2020 From: rob.a.mcdonald at gmail.com (Rob McDonald) Date: Wed, 5 Aug 2020 09:34:26 -0700 Subject: [pytest-dev] Test discovery In-Reply-To: <20200803092000.5q4t65ztciv5pd73@hooch.localdomain> References: <20200803092000.5q4t65ztciv5pd73@hooch.localdomain> Message-ID: Thanks Florian, It took me a while to digest everything you brought up - and to get things working. However, it was just what I needed. Thanks a bunch, Rob On Mon, Aug 3, 2020 at 2:20 AM Florian Bruhin wrote: > Hey Rob, > > On Sat, Aug 01, 2020 at 11:16:52PM -0700, Rob McDonald wrote: > > According to Google, 'How does pytest discover tests' is an oft-asked > > question. However most of those questions actually want to know 'What > > criteria does pytest use to discover tests'... > > > > I'm working on a program that needs to scan a directory structure for > > Python code and then execute some of that code -- much like pytest does. > > So, I'd like to know _how_ does pytest discover (and run) tests. > > > > I've searched around in the source for where this happens, but I haven't > > had any luck. Much appreciated if someone can point me to the relevant > > bits of code? Otherwise, any pointers on best practices and > > functions/methods to use for this sort of thing are appreciated. > > Most of it happens in pytest's internal "python" plugin: > > https://github.com/pytest-dev/pytest/blob/master/src/_pytest/python.py > > For a simpler example how pytest's collection tree works, see: > > https://docs.pytest.org/en/latest/reference.html#collection-hooks > https://docs.pytest.org/en/latest/example/nonpython.html > > Finally, you might be interested in Oliver Bestwalter's talk "Abridged > metaprogramming classics - this episode: pytest": > > https://github.com/obestwalter/abridged-meta-programming-classics > > There, Oliver implements a small subset of pytest's "magic" from scratch. > > Note, however, that there are various utility functions available in the > stdlib > (inspect, importlib, pkgutils) which make those kinds of tasks easier, at > least > with Python 3.5+. However, both pytest and Oliver's talk avoid them > (because of > Python 2 support and because it was a fun limitation, respectively). > > For a more straightforward implementation, you can roughly: > > - Use pkgutil.walk_packages to find modules > - Use importlib.import_module to import them > - Use inspect.getmembers to get e.g. functions > > I do the first two in my own project here: > > https://github.com/qutebrowser/qutebrowser/blob/master/qutebrowser/extensions/loader.py > > Florian > > -- > me at the-compiler.org (Mail/XMPP) | https://www.qutebrowser.org > https://bruhin.software/ | > https://github.com/sponsors/The-Compiler/ > GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc > I love long mails! | https://email.is-not-s.ms/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at the-compiler.org Tue Aug 11 11:45:18 2020 From: me at the-compiler.org (Florian Bruhin) Date: Tue, 11 Aug 2020 17:45:18 +0200 Subject: [pytest-dev] Webinar and training about pytest basics (German / Switzerland) Message-ID: <20200811154518.bbx6w7u6rujh4xmj@hooch.localdomain> Hey, (Back in June, I've sent a mail about a pytest workshop I'm giving in September. I'm not sure if it made it through everywhere, as I accidentally used a mail address which wasn't subscribed to the lists I was posting to. Since I also have a second announcement to make, I'm taking the freedom to post another "marketing" mail - after that, things will likely be silent for a while from my side again!) Webinar ------- On August 18th (next week, if you read this mail on the same day I'm writing it), I'm giving a 1h webinar about pytest basics in German: https://mylearning.ch/kurse/online-kurse/tech-webinar/ This is in collaboration with the Institute for Networked Solutions (INS) at a local university (Hochschule Rapperswil / HSR), hence the webinar is in German. If you'd be interested in something similar in English, please let me know - if there's enough interest, I'll repeat it in English at a later point in time. Training -------- On September 8th, I teach a full-day pytest training as part of the CH Open "Workshoptage" in Rotkreuz, Switzerland: https://workshoptage.ch/workshops/2020/pytest-test-driven-development-nicht-nur-fuer-python/ Of course COVID-19 is still a concern for the physical training - the organizer allows to cancel your ticket (with a full refund) up to two weeks before the training takes place and also closesly monitors the situation in Switzerland. Would be delighted to see some of you there! :) For the webinar, there should also be a recording available later. Florian -- me at the-compiler.org (Mail/XMPP) | https://www.qutebrowser.org https://bruhin.software/ | https://github.com/sponsors/The-Compiler/ GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc I love long mails! | https://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From ssbarnea at redhat.com Sat Aug 15 13:02:27 2020 From: ssbarnea at redhat.com (Sorin Sbarnea) Date: Sat, 15 Aug 2020 18:02:27 +0100 Subject: [pytest-dev] Transfer of pytest-html and pytest-molecule under pytest-dev Message-ID: I would like to transfer two pytest plugins (MIT) that I wrote and maintain to pytest-dev organization. My main goals being: - assure their long term maintenance - adopt similar testing methods - improve cross plugin testing (also to implement pre-release testing to avoid post-release conflicts) - improve discoverability of the plugins https://pypi.org/project/pytest-plus/ https://pypi.org/project/pytest-molecule/ I think that the projects already pass the requirements from https://docs.pytest.org/en/stable/contributing.html#submitting-plugins-to-pytest-dev -- because they were creating with shared ownership from start. I may note that the list of requirements is a little bit outdated, but I think nobody bothered with minor details like replacing README.txt with README.(txt|md|rst), or mentioning that required metadata can be better stored in setup.cfg instead of setup.py, which can even be missing (at least in theory). Both plugins are very simple and they do not have any special runtime/testing requirements. They will both use github-actions (the second one is tested with zuul-ci now, but I can switch it very easy). Cheers, Sorin Sbarnea From flub at devork.be Sun Aug 16 07:28:25 2020 From: flub at devork.be (Floris Bruynooghe) Date: Sun, 16 Aug 2020 13:28:25 +0200 Subject: [pytest-dev] Transfer of pytest-html and pytest-molecule under pytest-dev In-Reply-To: References: Message-ID: <87d03qvnhi.fsf@powell.devork.be> On Sat 15 Aug 2020 at 18:02 +0100, Sorin Sbarnea wrote: > I would like to transfer two pytest plugins (MIT) that I wrote and maintain to pytest-dev organization. Cool! > I may note that the list of requirements is a little bit outdated, but > I think nobody bothered with minor details like replacing README.txt > with README.(txt|md|rst), or mentioning that required metadata can be > better stored in setup.cfg instead of setup.py, which can even be > missing (at least in theory). Feel free to submit a PR to make it a bit more recent! I guess some people could be scared off by this. From ssbarnea at redhat.com Sun Aug 16 07:59:49 2020 From: ssbarnea at redhat.com (Sorin Sbarnea) Date: Sun, 16 Aug 2020 12:59:49 +0100 Subject: [pytest-dev] Transfer of pytest-html and pytest-molecule under pytest-dev In-Reply-To: <87d03qvnhi.fsf@powell.devork.be> References: <87d03qvnhi.fsf@powell.devork.be> Message-ID: In fact I was even considering writing a generic testing job for plugins, so we could consolidate testing, making maintenance easier and also enabling us to perform integration testing. Is still a dream, so I am not sure I will manage to find spare time to make it a reality. On Sun, 16 Aug 2020 at 12:28, Floris Bruynooghe wrote: > On Sat 15 Aug 2020 at 18:02 +0100, Sorin Sbarnea wrote: > > > > > I would like to transfer two pytest plugins (MIT) that I wrote and > maintain to pytest-dev organization. > > > > Cool! > > > > > I may note that the list of requirements is a little bit outdated, but > > > I think nobody bothered with minor details like replacing README.txt > > > with README.(txt|md|rst), or mentioning that required metadata can be > > > better stored in setup.cfg instead of setup.py, which can even be > > > missing (at least in theory). > > > > Feel free to submit a PR to make it a bit more recent! I guess some > > people could be scared off by this. > > > > -- -- /sorin -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicoddemus at gmail.com Mon Aug 17 07:51:04 2020 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Mon, 17 Aug 2020 08:51:04 -0300 Subject: [pytest-dev] Transfer of pytest-html and pytest-molecule under pytest-dev In-Reply-To: References: Message-ID: Hi all, On Sat, Aug 15, 2020 at 2:05 PM Sorin Sbarnea wrote: > I would like to transfer two pytest plugins (MIT) that I wrote and > maintain to pytest-dev organization. > > My main goals being: > - assure their long term maintenance > - adopt similar testing methods > - improve cross plugin testing (also to implement pre-release testing to > avoid post-release conflicts) > - improve discoverability of the plugins > > > https://pypi.org/project/pytest-plus/ > https://pypi.org/project/pytest-molecule/ +1 from me. I'm assuming Floris also agrees, so feel free to transfer the repositories to me to conclude the transfer. Cheers, Bruno I think that the projects already pass the requirements from > https://docs.pytest.org/en/stable/contributing.html#submitting-plugins-to-pytest-dev > -- because they were creating with shared ownership from start. > > I may note that the list of requirements is a little bit outdated, but I > think nobody bothered with minor details like replacing README.txt with > README.(txt|md|rst), or mentioning that required metadata can be better > stored in setup.cfg instead of setup.py, which can even be missing (at > least in theory). > > > Both plugins are very simple and they do not have any special > runtime/testing requirements. They will both use github-actions (the second > one is tested with zuul-ci now, but I can switch it very easy). > > > Cheers, > Sorin Sbarnea > > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From flub at devork.be Mon Aug 17 15:42:39 2020 From: flub at devork.be (Floris Bruynooghe) Date: Mon, 17 Aug 2020 21:42:39 +0200 Subject: [pytest-dev] Transfer of pytest-html and pytest-molecule under pytest-dev In-Reply-To: References: Message-ID: <87zh6tt5xs.fsf@powell.devork.be> On Mon 17 Aug 2020 at 08:51 -0300, Bruno Oliveira wrote: > On Sat, Aug 15, 2020 at 2:05 PM Sorin Sbarnea wrote: >> https://pypi.org/project/pytest-plus/ >> https://pypi.org/project/pytest-molecule/ > > +1 from me. > > I'm assuming Floris also agrees, so feel free to transfer the repositories > to me to conclude the transfer. Oh, I thought I already saw the repository transfer which is why I didn't add a +1. But sure, :) From nicoddemus at gmail.com Mon Aug 17 15:54:20 2020 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Mon, 17 Aug 2020 16:54:20 -0300 Subject: [pytest-dev] Transfer of pytest-html and pytest-molecule under pytest-dev In-Reply-To: <87zh6tt5xs.fsf@powell.devork.be> References: <87zh6tt5xs.fsf@powell.devork.be> Message-ID: On Mon, Aug 17, 2020 at 4:42 PM Floris Bruynooghe wrote: > On Mon 17 Aug 2020 at 08:51 -0300, Bruno Oliveira wrote: > > On Sat, Aug 15, 2020 at 2:05 PM Sorin Sbarnea > wrote: > >> https://pypi.org/project/pytest-plus/ > >> https://pypi.org/project/pytest-molecule/ > > > > +1 from me. > > > > I'm assuming Floris also agrees, so feel free to transfer the > repositories > > to me to conclude the transfer. > > Oh, I thought I already saw the repository transfer which is why I > didn't add a +1. But sure, :) > Oh wait, my bad, completely missed the Subject which clearly states pytest-html, hehehe. Cheers, Bruno -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at the-compiler.org Mon Aug 17 16:20:16 2020 From: me at the-compiler.org (Florian Bruhin) Date: Mon, 17 Aug 2020 22:20:16 +0200 Subject: [pytest-dev] Transfer of pytest-html and pytest-molecule under pytest-dev In-Reply-To: References: <87zh6tt5xs.fsf@powell.devork.be> Message-ID: <20200817202016.xgjl5adhnvh43mu2@hooch.localdomain> On Mon, Aug 17, 2020 at 04:54:20PM -0300, Bruno Oliveira wrote: > On Mon, Aug 17, 2020 at 4:42 PM Floris Bruynooghe wrote: > > > On Mon 17 Aug 2020 at 08:51 -0300, Bruno Oliveira wrote: > > > On Sat, Aug 15, 2020 at 2:05 PM Sorin Sbarnea > > wrote: > > >> https://pypi.org/project/pytest-plus/ > > >> https://pypi.org/project/pytest-molecule/ > > > > > > +1 from me. > > > > > > I'm assuming Floris also agrees, so feel free to transfer the > > repositories > > > to me to conclude the transfer. > > > > Oh, I thought I already saw the repository transfer which is why I > > didn't add a +1. But sure, :) > > > > Oh wait, my bad, completely missed the Subject which clearly states > pytest-html, hehehe. That's confusing point 1: I don't see how this is related to pytest-html at all? Confusing point 2 is that Sorin already did transfer pytest-plus before asking (losing proper access on GitHub in the process), which is when I pointed out that there's a process for doing this kind of thing properly. I originally offered to transfer it back, but given that it looks like there's agreement about the transfer, I guess I won't need to do that anymore. Someone will still need to set up the proper teams and access rights though. Florian -- me at the-compiler.org (Mail/XMPP) | https://www.qutebrowser.org https://bruhin.software/ | https://github.com/sponsors/The-Compiler/ GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc I love long mails! | https://email.is-not-s.ms/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From flub at devork.be Wed Aug 19 08:13:46 2020 From: flub at devork.be (Floris Bruynooghe) Date: Wed, 19 Aug 2020 14:13:46 +0200 Subject: [pytest-dev] Transfer of pytest-html and pytest-molecule under pytest-dev In-Reply-To: <20200817202016.xgjl5adhnvh43mu2@hooch.localdomain> References: <87zh6tt5xs.fsf@powell.devork.be> <20200817202016.xgjl5adhnvh43mu2@hooch.localdomain> Message-ID: Agree that this was all a bit confusing. Oh well, I'll learn from it that next time I should be a bit more careful and not expect that because github did things they're correct. Thanks for clarifying Florian! On Mon, 17 Aug 2020, 22:20 Florian Bruhin, wrote: > On Mon, Aug 17, 2020 at 04:54:20PM -0300, Bruno Oliveira wrote: > > On Mon, Aug 17, 2020 at 4:42 PM Floris Bruynooghe > wrote: > > > > > On Mon 17 Aug 2020 at 08:51 -0300, Bruno Oliveira wrote: > > > > On Sat, Aug 15, 2020 at 2:05 PM Sorin Sbarnea > > > wrote: > > > >> https://pypi.org/project/pytest-plus/ > > > >> https://pypi.org/project/pytest-molecule/ > > > > > > > > +1 from me. > > > > > > > > I'm assuming Floris also agrees, so feel free to transfer the > > > repositories > > > > to me to conclude the transfer. > > > > > > Oh, I thought I already saw the repository transfer which is why I > > > didn't add a +1. But sure, :) > > > > > > > Oh wait, my bad, completely missed the Subject which clearly states > > pytest-html, hehehe. > > That's confusing point 1: I don't see how this is related to pytest-html at > all? > > Confusing point 2 is that Sorin already did transfer pytest-plus before > asking > (losing proper access on GitHub in the process), which is when I pointed > out > that there's a process for doing this kind of thing properly. > > I originally offered to transfer it back, but given that it looks like > there's > agreement about the transfer, I guess I won't need to do that anymore. > Someone > will still need to set up the proper teams and access rights though. > > Florian > > -- > me at the-compiler.org (Mail/XMPP) | https://www.qutebrowser.org > https://bruhin.software/ | > https://github.com/sponsors/The-Compiler/ > GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc > I love long mails! | https://email.is-not-s.ms/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: