* It is [currently [#PEP426JSONLD)] necessary to run setup.py with each
given destination platform, because parameters are expanded within the
scope of setup.py.
* Because of this, client side dependency resolution (with a given
platform) is currently the only viable option for something like this
...
* Build: Docker, Tox (Dox) to build package(s)
* Each assembly of packages is / could be a package with a setup.py
(and/or a requirements.txt)
* And tests:
* http://conda.pydata.org/docs/building/meta-yaml.html#test-section
* Release: DevPi
* http://doc.devpi.net/latest/
* conda env environment.yml YAML:
http://conda.pydata.org/docs/using/envs.html
* [x] conda packages
* [x] pip packages
* [ ] system packages (configuration management)
And then, really, Is there a stored version of this instance of a named
Docker image?
#reproducibility #linkedreproducibility
On Sat, Nov 7, 2015 at 8:37 AM, Thomas Güttler wrote: I wrote down a tought about Serverside Dependency Resolution and
Virtualenv Build Server What do you think? Latest version: https://github.com/guettli/virtualenv-build-server virtualenv-build-server
####################### Rough roadmap how a server to build virtualenvs for the python programming
language could be implemented. Highlevel goal
-------------- Make creating new virtual envionments for the Python programming language
easy and fast. Input: fuzzy requirements like this: django>=1.8, requests=>2.7 Output: virtualenv with packages installed. Two APIs
------------ #. Resolve fuzzy requirements to a fixed set of packages with exactly
pinned versions.
#. Read fixed set of packages. Build virtualenv according to given
platform. Steps
----- Steps: #. Client sends list of fuzzy requirements to server: * I need: django>=1.8, requests=>2.7, ... #. Server solves the fuzzy requirements to a fixed set of requirememnts:
django==1.8.2, requests==2.8.1, ... #. Client reads the fixed set of requirements. #. Optional: Client sends fixed set of requirements to the server. Telling
him the plattform * My platform: sys.version==2.7.6 and sys.platform=linux2 #. Server builds a virtualenv according to the fixed set of requirements. #. Server sends the environment to the client #. Client unpacks the data and has a usable virtualenv Benefits
-------- Speed: * There is only one round-trip from client to server. If the dependencies
get resolved on the client the client would need to download the available
version information.
* Caching: If the server gets input parameters (fuzzy requirements and
platform information) which he has seen before, he can return the cached
result from the previous request. Possible Implementations
------------------------ APIs
====
Both APIs could be implementated by a webservice/Rest interface passing
json or yaml. Serverside
========== Implementation Strategie "PostgreSQL"
..................................... Since the API is de-coupled from the internals the implementation could be
exchanged without the need for changes at the client side. I suggest using the PostgreSQL und resolving the dependcy graph using SQL
(WITH RECURSIVE). The package and version data gets stored in PostgreSQL via ORM (Django or
SQLAlchemy). The version numbers need to be normalized to ascii to allow fast
comparision. Related: https://www.python.org/dev/peps/pep-0440/ Implementation Strategie "Node.js"
.................................. I like python, but I am not married with it. Why not use a different tools
that is already working? Maybe the node package manager:
https://www.npmjs.com/ Questions
---------
Are virtualenv relocatable? AFAIK they are not. General Thoughts
---------------- * Ignore Updates. Focus on creating new virtualenvs. The server can do
caching and that's why I prefer creating virtualenvs which never get
updated. They get created and removed (immutable). I won't implement it
-------------------- This idea is in the public domain. If you are young and brave or old and
wise: Go ahead, try to implement it. Please communicate early and often.
Ask on mailing-lists or me for feedback. Good luck :-) I love feedback
--------------- Please tell me want you like or dislike: * typos and spelling stuff (I am not a native speaker)
* alternative implementation strategies.
* existing software which does this (even if implemented in a different
programming language).
* ... --
http://www.thomas-guettler.de/
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig