Testing package generated with setuptools/distutils
Hi guys, I have a question here (sorry if my lack of practice in English). I currently working in one of the many projects about covid-19, and I just deployed a new version for a group of epidemiologists. To be fair I deployed not once but three times because of two miss-configuration of setuptools. - First, I forget to add the internal package arcoviv19.web to include the web app of the project (https://github.com/ivco19/libs/blob/master/setup.py#L89) - Second, forget to add a line into my MANIFEST.in, so the project WebClient doesn't work properly. (https://github.com/ivco19/libs/blob/master/MANIFEST.in) So, after the fixes, I starting to mock tests to check this kind of mistake. The idea was quick and dirty but works: I give a path to my entire source-tree and the the file generated with "setup.py sdist" must contain the exact same files. So the question: Some know about a tool that checks this kind of thing/problem? -- Juan B Cabral
On Thu, May 14, 2020 at 03:25:02AM -0300, Juan BC wrote:
You may find https://pypi.org/project/check-manifest/ useful. HTH, Marius Gedminas -- Theory is when you know something, but it doesn't work. Practice is when something works, but you don't know why. Programmers combine theory and practice: Nothing works and they don't know why.
На 14.05.20 г. в 9:25 ч., Juan BC написа:
Why are you not using `find_packages()` with possible excludes for the test dirs?
"exact same files" doesn't quite work for the majority of the cases. Your Python package includes a subset of all the files which are in the git repository and you can indeed try to verify that this is the case. A simple extract & diff will do the trick. However your package doesn't seem to include the databases/ directory for example. You see how this starts having exceptions very quickly hence a generic tool is very tricky and impractical to build. FTR in one of my projects we have find_packages() in setup.py (which relies on __init__.py being present) so we made a custom pylint plugin to warn about .py files which are inside directories missing __init__.py: https://github.com/kiwitcms/Kiwi/blob/master/kiwi_lint/empty.py#L30 An added nice bonus is that Django's test runner also relies of the presence of __init__.py and this helped us discover directories where we had had automated tests (test_...py) but they were not automatically picked up by the test runner. -- Alex
Thanks for the tools and the example. I just searching for something like check_manifest, because as was pointed, my configuration is "faaaar" to be generic or configurable. My dream tool was something like (just imagine a toxenv) [testenv:check-package] skip_install = True usedevelop = False deps = pkgcheck commands = python setup.py sdist pkgcheck --manifest MANIFEST.in --requirements requirements.txt --src src/ --setup setup.py dist/my_package.tar.gz And this magically checks the package vs all the configuration files. Maybe a pylint extension can make some of these functionalities. El jue., 14 may. 2020 a las 6:33, Alexander Todorov (<atodorov@mrsenko.com>) escribió:
-- Juan B Cabral
On Thu, May 14, 2020 at 03:25:02AM -0300, Juan BC wrote:
You may find https://pypi.org/project/check-manifest/ useful. HTH, Marius Gedminas -- Theory is when you know something, but it doesn't work. Practice is when something works, but you don't know why. Programmers combine theory and practice: Nothing works and they don't know why.
На 14.05.20 г. в 9:25 ч., Juan BC написа:
Why are you not using `find_packages()` with possible excludes for the test dirs?
"exact same files" doesn't quite work for the majority of the cases. Your Python package includes a subset of all the files which are in the git repository and you can indeed try to verify that this is the case. A simple extract & diff will do the trick. However your package doesn't seem to include the databases/ directory for example. You see how this starts having exceptions very quickly hence a generic tool is very tricky and impractical to build. FTR in one of my projects we have find_packages() in setup.py (which relies on __init__.py being present) so we made a custom pylint plugin to warn about .py files which are inside directories missing __init__.py: https://github.com/kiwitcms/Kiwi/blob/master/kiwi_lint/empty.py#L30 An added nice bonus is that Django's test runner also relies of the presence of __init__.py and this helped us discover directories where we had had automated tests (test_...py) but they were not automatically picked up by the test runner. -- Alex
Thanks for the tools and the example. I just searching for something like check_manifest, because as was pointed, my configuration is "faaaar" to be generic or configurable. My dream tool was something like (just imagine a toxenv) [testenv:check-package] skip_install = True usedevelop = False deps = pkgcheck commands = python setup.py sdist pkgcheck --manifest MANIFEST.in --requirements requirements.txt --src src/ --setup setup.py dist/my_package.tar.gz And this magically checks the package vs all the configuration files. Maybe a pylint extension can make some of these functionalities. El jue., 14 may. 2020 a las 6:33, Alexander Todorov (<atodorov@mrsenko.com>) escribió:
-- Juan B Cabral
participants (3)
-
Alexander Todorov -
Juan BC -
Marius Gedminas