[Tutor] Python project organisation

[dn]

On 23/04/2023 11.46, Phil wrote:
> I'm looking suggestions for organising my python projects.
> 
> I had, until recently, all of my python projects stored in the one 
> directory (~/Python), all 298 of them. This has become almost 
> unmanageable so I split my projects into multiple sub-directories named 
> Tkinter, Wxpython, etc. and named each project, for example those under 
> the Tkinter sub-directory, as "tkinter_project_name.py". This is, or 
> was, better but still not good.
> 
> This problem has now come to a head because because my upgraded Linux 
> distribution now requires the use of virtual environments, otherwise 
> "pip3 install" fails.
> 
> So I created a virtual environment "~/.venv/bleak/bin" named after the 
> name of the module that I needed to upgrade. It probably should be named 
> after the project instead. The code for the project is now in that 
> virtual environment directory which brings me to my question.
> 
> Should I create a virtual environment for each project, I don't think 
> so? If I do, won't that make the PYTHONPATH unwieldy with hundreds of 
> directories needing to be added to PYTHONPATH? I can see a case where a 
> project should have it's own environment and that's where a project 
> includes images that are solely used by that project. At the moment 
> images and custom classes are all stored under python's global environment.

Yes, each project SHOULD be 'insulated' from the next. This will allow 
one project to use a newer version of a library than the others, for 
example. Whereas, updating the library system-wide sh/would require you 
to re-test every project which uses same, to head-off any 'breakages' or 
other difficulties.

The mode of utilising virtual-environments is that only one (or a few 
related) project/environments will be open or "live" at a time. Part of 
making an environment active is to add it into the PYTHONPATH. 
Conversely, part of shutting-down the environment, eg when you move your 
attention to a different project, is to remove those references again, 
ie back to system-settings. Thus, the modification to PYTHONPATH is only 
temporary.

I have recently started using Poetry under PyCharm. When I decide to 
work on a particular project, the IDE takes-care of all the 'plumbing', 
eg PYTHONPATH. The combination of both tools keeps track of (only) the 
Python libraries which this particular project needs - and enables 
libraries to be separately updated, as-and-when. Poetry provides a 
mechanism to "package" the project and move it (my code and the 
Python-environment) to another machine, or wherever.

All this is documented in the PyCharm 'docs'. Presumably VS-Codium/e 
(etc) will offer something equivalent.

-- 
Regards,
=dn