What's the latest best practice on Python project directory layouts?
Cameron Simpson
cs at cskk.id.au
Fri Jun 14 21:15:14 EDT 2019
On 14Jun2019 09:23, Malcolm Greene <python at bdurham.com> wrote:
>I have a collection of command line scripts that share a collection of common modules. This code collection is for internal use and will run under a single version of Python 3.6+ and a single OS. My understanding of best practice is to organize this collection of Python files into a folder structure like this:
>
># common files
>.gitignore
>readme.md
>requirements.txt
>setup.py <--- what is the advantage of this file for internally distributed code bases?
>
># app specific package folders
>app-1
> __init__.py (optional; if needed)
> __main__.py
> app-1-module-1.py
> app-1-module-2.py
> app-1-module-N.py
>
>app-2
> __init__.py (optional; if needed)
> __main__.py
> app-2-module-1.py
> app-2-module-2.py
> app-2-module-N.py
>
># modules shared across multiple apps
>common
> common-module-1.py
> common-module-2.py
> common-module-N.py
>
># tests - place at package level with sub-packages for each package -OR- underneath each app package?
>tests
> app-1
> test_app-1-module-1.py
> test_app-1-module-2.py
> test_app-1-module-N.py
> app-2
> test_app-2-module-1.py
> test_app-2-module-2.py
> test_app-2-module-N.py
>
># virtual env folder placed at same level as packages ???
>venv
> <virtual-env files go here>
>
>And execute each app via the following ...
>
>python -m app-1 <optional-parameters ...>
>
>Questions
>
>1. Does the above structure sound reasonable?
Yes. Though I like to get them out of the top directory, details below.
>2. Where to place virtual env files and what to call this folder? venv, .env, etc?
I use "venv" myself.
>3. Where to put tests (pytest)? In a tests folder or under each package?
Personally, I'd do it however you would do them if the apps and the
common modules were standalone. I use a foo_tests.py beside my foo.py
module file myself, but a naming scheme adhering to the discoverability
of your test running tool would also be a good choice.
>4. Use a src folder or not? If so, where to put above files relative to
>the src folder?
Well, yeah. Always a subdirectory, I hate littering the top level.
Here is how I lay out a project, based on my current one (nonPython bits
elided):
project/
bin/
lib/python/all-modules-here
venv/
Various points:
- I dislike using Python's "search for modules in the current
directory"; I would _always_ rather set $PYTHONPATH for specific
control.
- The lib/python depth is to accomodate lib/other-languages according to
the mix in the project.
Some context: I'm making a full stack app for a client at present. The
client's code is like this:
lib/python/clientname/appname/*.py
It happens that all the "common" code is in the appname subdirectory
because it is self contained, but were it not it would be in
lib/python/clientname/*.py
or possibly
lib/python/clientname/util/*.py
(or "common" if you prefer).
One important aspect of this is that it lets me keep the client code
away from conflicts with other library names, so the "clientname" module
path component is important for this, _and_ it better labels the purpose
of the module.
So the environment setup looks like this:
project=/path/to/project # or $(dirname "$0") if $0 is useful
PYTHONPATH=$project/lib/python
PATH=$project/bin:$project/venv/bin:$PATH
export PYTHONPATH PATH
and the app is run:
python -m clientname.appname args...
The app does relative imports for itself:
from .util import SomeAppClass
and would import common code absolutely:
from clientname.util import SomeCommonClass
In my current project I've actually got a small shell script which does
the environment setup above and invokes the python app (or various other
utility tasks like "init the database" etc). I find this approach
generally useful.
Anyway, this might inform what choices you make. Happy to elaborate on
specifics, though they get more personal and idiosyncratic the more fine
grained we get.
Cheers,
Cameron Simpson <cs at cskk.id.au>
More information about the Python-list
mailing list