questions about tox 3.3 plugin api and plugin development
Hello, I'm creating my first tox plugin, and I have some questions. the plugin is: https://github.com/luzfcb/tox-lambda-autodiscovery *The context:* I'm doing an experimental project (closed source) with serverless[1] framework, aws lambda[2] and django. aws lambda, allows me to have a requirements.txt file for each lambda function and for each folder with lambda function, I also have one or more test files. ( this is necessary because lambda restricts the maximum size of the lambda function package to 50MB. The package includes all python and non-python dependencies.) the directory structure looks like this ( do not be surprised with the camel case, it is restriction imposed by aws Lambda and aws CloudFormation) [image: image.png] *The problem I want to solve:* I want to avoid creating manually, the tox testenv, for each of these lambda functions (because in the future there will be hundreds of them.) I want to avoid doing this. [image: image.png] that is, I want my tox plugin to automatically analyze the project directories structure, and look for folders that contain a *requirements.txt* file and that also contain test files, named *test*.py*, and automatically create, one tox testenv for each of these folders. my plugin currently can do this. (I do not know if it makes the best way, but it works). currently, for it to work, it is only necessary to add [testenv:lambdaautodiscovery] in your tox.ini *The questions:* *1* - how create a new variables for substitution ( to work like *{toxinidir}* )? What is the correct way to create it? https://github.com/luzfcb/tox-lambda-autodiscovery/issues/1 2 - I have a custom tox testenv, with some extra variables and configurations, and I would like to use it as a base, to create the others automatically generated tox testenv. Am I doing it the right way? https://github.com/luzfcb/tox-lambda-autodiscovery/blob/master/tox_lambda_au... Do you have any suggestions on how to improve this? *3* - the only thing that still does not work is the main one: *command* is there any way to defer *command* processing, or any other way to make it work? see: https://github.com/luzfcb/tox-lambda-autodiscovery/issues/2 Thank you all for your help. Any comment is welcome. [1] https://serverless.com/ [2] https://aws.amazon.com/lambda/ -- Fabio C. Barrionuevo da Luz Palmas - Tocantins - Brasil - América do Sul
Hello,
I think you did a fairly good job on what's possible at the moment.
1. The substitution works at the moment on the config parser side (see
https://github.com/tox-dev/tox/blob/master/src/tox/config.py#L967). And
it's not possible at the moment to hook into it (other than coming up with
some monkeypatching). I believe the road ahead is to separate things (aka
config source, config expansion, config substitution). This is though
highly non-trivial and probably a task for tox 4.0.
2. No better way I suppose at the moment. Part of doing
https://github.com/tox-dev/tox/issues/828 might unravel some of the
requirements.
3. The command processing happens here (
https://github.com/tox-dev/tox/blob/master/src/tox/config.py#L1495) and no
real way to defer it. Other than maybe come up with a custom syntax that
your plugin unrolls afterwards. E.g. use <asdas> instead of {asdas} and
then unrolling it.
Does this envs need to be customize-able by the user? Why not skip the
entire config reading stuff and directly instantiate the TestEnvConfig, and
pass it into the session that?
https://github.com/tox-dev/tox/blob/master/src/tox/config.py#L844
On Fri, Oct 5, 2018 at 4:25 PM Fabio C. Barrionuevo da Luz
Hello, I'm creating my first tox plugin, and I have some questions.
the plugin is: https://github.com/luzfcb/tox-lambda-autodiscovery
*The context:*
I'm doing an experimental project (closed source) with serverless[1] framework, aws lambda[2] and django.
aws lambda, allows me to have a requirements.txt file for each lambda function and for each folder with lambda function, I also have one or more test files. ( this is necessary because lambda restricts the maximum size of the lambda function package to 50MB. The package includes all python and non-python dependencies.)
the directory structure looks like this ( do not be surprised with the camel case, it is restriction imposed by aws Lambda and aws CloudFormation)
[image: image.png]
*The problem I want to solve:*
I want to avoid creating manually, the tox testenv, for each of these lambda functions (because in the future there will be hundreds of them.)
I want to avoid doing this.
[image: image.png]
that is, I want my tox plugin to automatically analyze the project directories structure, and look for folders that contain a *requirements.txt* file and that also contain test files, named *test*.py*, and automatically create, one tox testenv for each of these folders.
my plugin currently can do this. (I do not know if it makes the best way, but it works).
currently, for it to work, it is only necessary to add [testenv:lambdaautodiscovery] in your tox.ini
*The questions:*
*1* - how create a new variables for substitution ( to work like *{toxinidir}* )? What is the correct way to create it?
https://github.com/luzfcb/tox-lambda-autodiscovery/issues/1
2 - I have a custom tox testenv, with some extra variables and configurations, and I would like to use it as a base, to create the others automatically generated tox testenv. Am I doing it the right way?
https://github.com/luzfcb/tox-lambda-autodiscovery/blob/master/tox_lambda_au...
Do you have any suggestions on how to improve this?
*3* - the only thing that still does not work is the main one: *command* is there any way to defer *command* processing, or any other way to make it work? see: https://github.com/luzfcb/tox-lambda-autodiscovery/issues/2
Thank you all for your help. Any comment is welcome.
[1] https://serverless.com/ [2] https://aws.amazon.com/lambda/
-- Fabio C. Barrionuevo da Luz Palmas - Tocantins - Brasil - América do Sul
_______________________________________________ tox-dev mailing list -- tox-dev@python.org To unsubscribe send an email to tox-dev-leave@python.org https://mail.python.org/mm3/mailman3/lists/tox-dev.python.org/
first version that works. (ok, there is probably a lot of edge cases that
have not treated or thought.)
https://pypi.org/project/tox-lambda-autodiscovery/0.1.0/
questions:
1 - in each auto-created tox testenv, how to "inherit" passenv and setenv
from [testenv:lambdaautodiscovery] ?
2 - in tox_configure, it is possible to know / differentiate if tox was
called with argument -e, and what was it ?
3 - is there any example or documentation of the "correct" way of writing
tests for tox plugins?
Em sex, 5 de out de 2018 às 12:41, Bernát Gábor
Hello,
I think you did a fairly good job on what's possible at the moment.
1. The substitution works at the moment on the config parser side (see https://github.com/tox-dev/tox/blob/master/src/tox/config.py#L967). And it's not possible at the moment to hook into it (other than coming up with some monkeypatching). I believe the road ahead is to separate things (aka config source, config expansion, config substitution). This is though highly non-trivial and probably a task for tox 4.0. 2. No better way I suppose at the moment. Part of doing https://github.com/tox-dev/tox/issues/828 might unravel some of the requirements. 3. The command processing happens here ( https://github.com/tox-dev/tox/blob/master/src/tox/config.py#L1495) and no real way to defer it. Other than maybe come up with a custom syntax that your plugin unrolls afterwards. E.g. use <asdas> instead of {asdas} and then unrolling it.
Does this envs need to be customize-able by the user? Why not skip the entire config reading stuff and directly instantiate the TestEnvConfig, and pass it into the session that? https://github.com/tox-dev/tox/blob/master/src/tox/config.py#L844
On Fri, Oct 5, 2018 at 4:25 PM Fabio C. Barrionuevo da Luz < bnafta@gmail.com> wrote:
Hello, I'm creating my first tox plugin, and I have some questions.
the plugin is: https://github.com/luzfcb/tox-lambda-autodiscovery
*The context:*
I'm doing an experimental project (closed source) with serverless[1] framework, aws lambda[2] and django.
aws lambda, allows me to have a requirements.txt file for each lambda function and for each folder with lambda function, I also have one or more test files. ( this is necessary because lambda restricts the maximum size of the lambda function package to 50MB. The package includes all python and non-python dependencies.)
the directory structure looks like this ( do not be surprised with the camel case, it is restriction imposed by aws Lambda and aws CloudFormation)
[image: image.png]
*The problem I want to solve:*
I want to avoid creating manually, the tox testenv, for each of these lambda functions (because in the future there will be hundreds of them.)
I want to avoid doing this.
[image: image.png]
that is, I want my tox plugin to automatically analyze the project directories structure, and look for folders that contain a *requirements.txt* file and that also contain test files, named *test*.py*, and automatically create, one tox testenv for each of these folders.
my plugin currently can do this. (I do not know if it makes the best way, but it works).
currently, for it to work, it is only necessary to add [testenv:lambdaautodiscovery] in your tox.ini
*The questions:*
*1* - how create a new variables for substitution ( to work like *{toxinidir}* )? What is the correct way to create it?
https://github.com/luzfcb/tox-lambda-autodiscovery/issues/1
2 - I have a custom tox testenv, with some extra variables and configurations, and I would like to use it as a base, to create the others automatically generated tox testenv. Am I doing it the right way?
https://github.com/luzfcb/tox-lambda-autodiscovery/blob/master/tox_lambda_au...
Do you have any suggestions on how to improve this?
*3* - the only thing that still does not work is the main one: *command* is there any way to defer *command* processing, or any other way to make it work? see: https://github.com/luzfcb/tox-lambda-autodiscovery/issues/2
Thank you all for your help. Any comment is welcome.
[1] https://serverless.com/ [2] https://aws.amazon.com/lambda/
-- Fabio C. Barrionuevo da Luz Palmas - Tocantins - Brasil - América do Sul
_______________________________________________ tox-dev mailing list -- tox-dev@python.org To unsubscribe send an email to tox-dev-leave@python.org https://mail.python.org/mm3/mailman3/lists/tox-dev.python.org/
-- Fábio C. Barrionuevo da Luz Palmas - Tocantins - Brasil - América do Sul http://pythonclub.com.br/ Blog colaborativo sobre Python e tecnologias Relacionadas, mantido totalmente no https://github.com/pythonclub/pythonclub.github.io . Todos são livres para publicar. É só fazer fork, escrever sua postagem e mandar o pull-request. Leia mais sobre como publicar em README.md e contributing.md. Regra básica de postagem: "Você" acha interessante? É útil para "você"? Pode ser utilizado com Python ou é útil para quem usa Python? Está esperando o que? Publica logo, que estou louco para ler...
1. envconfig.setenv is a dictionary, just read from one env and update it
into another.
2. Config.option is the content of the cli parsed argument, you can check
what was passed for -e from there
https://github.com/tox-dev/tox/blob/master/src/tox/config.py#L830
3. https://github.com/tox-dev/tox/blob/master/src/tox/_pytestplugin.py is
shipped with tox and can be used to write tests, see for example
https://github.com/tox-dev/tox-venv/blob/master/tests/test_z_cmdline.py#L253
. We really need to have better documentation of this though.
On Tue, Oct 9, 2018 at 6:57 PM Fabio C. Barrionuevo da Luz
first version that works. (ok, there is probably a lot of edge cases that have not treated or thought.)
https://pypi.org/project/tox-lambda-autodiscovery/0.1.0/
questions:
1 - in each auto-created tox testenv, how to "inherit" passenv and setenv from [testenv:lambdaautodiscovery] ?
2 - in tox_configure, it is possible to know / differentiate if tox was called with argument -e, and what was it ?
3 - is there any example or documentation of the "correct" way of writing tests for tox plugins?
Em sex, 5 de out de 2018 às 12:41, Bernát Gábor
escreveu: Hello,
I think you did a fairly good job on what's possible at the moment.
1. The substitution works at the moment on the config parser side (see https://github.com/tox-dev/tox/blob/master/src/tox/config.py#L967). And it's not possible at the moment to hook into it (other than coming up with some monkeypatching). I believe the road ahead is to separate things (aka config source, config expansion, config substitution). This is though highly non-trivial and probably a task for tox 4.0. 2. No better way I suppose at the moment. Part of doing https://github.com/tox-dev/tox/issues/828 might unravel some of the requirements. 3. The command processing happens here ( https://github.com/tox-dev/tox/blob/master/src/tox/config.py#L1495) and no real way to defer it. Other than maybe come up with a custom syntax that your plugin unrolls afterwards. E.g. use <asdas> instead of {asdas} and then unrolling it.
Does this envs need to be customize-able by the user? Why not skip the entire config reading stuff and directly instantiate the TestEnvConfig, and pass it into the session that? https://github.com/tox-dev/tox/blob/master/src/tox/config.py#L844
On Fri, Oct 5, 2018 at 4:25 PM Fabio C. Barrionuevo da Luz < bnafta@gmail.com> wrote:
Hello, I'm creating my first tox plugin, and I have some questions.
the plugin is: https://github.com/luzfcb/tox-lambda-autodiscovery
*The context:*
I'm doing an experimental project (closed source) with serverless[1] framework, aws lambda[2] and django.
aws lambda, allows me to have a requirements.txt file for each lambda function and for each folder with lambda function, I also have one or more test files. ( this is necessary because lambda restricts the maximum size of the lambda function package to 50MB. The package includes all python and non-python dependencies.)
the directory structure looks like this ( do not be surprised with the camel case, it is restriction imposed by aws Lambda and aws CloudFormation)
[image: image.png]
*The problem I want to solve:*
I want to avoid creating manually, the tox testenv, for each of these lambda functions (because in the future there will be hundreds of them.)
I want to avoid doing this.
[image: image.png]
that is, I want my tox plugin to automatically analyze the project directories structure, and look for folders that contain a *requirements.txt* file and that also contain test files, named *test*.py*, and automatically create, one tox testenv for each of these folders.
my plugin currently can do this. (I do not know if it makes the best way, but it works).
currently, for it to work, it is only necessary to add [testenv:lambdaautodiscovery] in your tox.ini
*The questions:*
*1* - how create a new variables for substitution ( to work like *{toxinidir}* )? What is the correct way to create it?
https://github.com/luzfcb/tox-lambda-autodiscovery/issues/1
2 - I have a custom tox testenv, with some extra variables and configurations, and I would like to use it as a base, to create the others automatically generated tox testenv. Am I doing it the right way?
https://github.com/luzfcb/tox-lambda-autodiscovery/blob/master/tox_lambda_au...
Do you have any suggestions on how to improve this?
*3* - the only thing that still does not work is the main one: *command* is there any way to defer *command* processing, or any other way to make it work? see: https://github.com/luzfcb/tox-lambda-autodiscovery/issues/2
Thank you all for your help. Any comment is welcome.
[1] https://serverless.com/ [2] https://aws.amazon.com/lambda/
-- Fabio C. Barrionuevo da Luz Palmas - Tocantins - Brasil - América do Sul
_______________________________________________ tox-dev mailing list -- tox-dev@python.org To unsubscribe send an email to tox-dev-leave@python.org https://mail.python.org/mm3/mailman3/lists/tox-dev.python.org/
-- Fábio C. Barrionuevo da Luz Palmas - Tocantins - Brasil - América do Sul
Blog colaborativo sobre Python e tecnologias Relacionadas, mantido totalmente no https://github.com/pythonclub/pythonclub.github.io .
Todos são livres para publicar. É só fazer fork, escrever sua postagem e mandar o pull-request. Leia mais sobre como publicar em README.md e contributing.md. Regra básica de postagem: "Você" acha interessante? É útil para "você"? Pode ser utilizado com Python ou é útil para quem usa Python? Está esperando o que? Publica logo, que estou louco para ler...
_______________________________________________ tox-dev mailing list -- tox-dev@python.org To unsubscribe send an email to tox-dev-leave@python.org https://mail.python.org/mm3/mailman3/lists/tox-dev.python.org/
participants (2)
-
Bernát Gábor
-
Fabio C. Barrionuevo da Luz