[Python-ideas] File format for automatic and manual tests

[Victor Porton]

This is an idea of a new PEP.

I propose to create a portable file format which will list command line 
options to run Python scripts with dual purpose:

1. for automatic tests executing scripts from this file (and optionally 
checking their stdout against specified values).

2. for running the scripts in a Python debugger.

The scripts may be either real command line applications written in 
Python or scripts created specifically for testing.

Currently I use PyCharm to debug my command line application.

PyCharm allows to define and store different command line options for 
Python scripts.

This has two drawbacks:

1. It is not portable and this makes me not to submit the PyCharm 
configuration into my GitHub repo.

2. I can't (at least easily) use these command line profiles for 
automated testing.

I propose to create the file format listing such test/debug cases based 
on options available in PyCharm, that is:

- Python script path

- command line parameters

- environment variables

- Python interpreter and interpreter options

- working directory

- tweaks of PYTHONPATH to include our source directory as necessary

- redirect input from

It would be nice to add "variables" (substituted with strings) to our 
file format. Automated tests should support iterations of command lines 
with different vars substituted.

I hope a future version of PyCharm will support our file format.