python package management confusion

[dcs3spp]

On Wednesday, 16 January 2019 07:07:29 UTC, dieter  wrote:
> dcs3spp via Python-list <python-list at python.org> writes:
> > ...
> > So to manage the development of private packages, e.g. wheels, I would have to use my own private repository (something like devpi or a an alternative cloud pypi subscription service) to store each private dependency that I have written.
> 
> No, you do not need something like "devpi" (or similar).
> Instead, you can set up a "virtualenv" (there is a Python package
> which can build "virtualenv"s), and use "setuptool"'s "develop"
> "setup" command to "install" links to the package sources you
> currently have under development. "develop" will automatically
> install external requirements via "pypi".
> 
> > ...
> > However, if I wanted to take a step further and run a CI build using cloud services(e.g. in a private gitlab.com repository) for a package that uses the private packages, then presumably there is no access to the devpi repository on my local system? So, alternatively when developing private Python packages I either use requirements.txt or pay subscription for a private pypi cloud repository and configure pip, setup.cfg on gitlab.com CI to reference it in config files. When the CI build completes it pushes the package to the private pypi repository. 
> 
> I assume that you will be able to build an appropriate "virtualenv"
> in a CI build setup.

Thankyou for responding and thanks for your patience with this newbie dieter....

Ahhhh, so it is possible to use a virtualenv to pull in dependencies from setup.py ....

I have the setup.py below and the pyramid_core package surrounded by * in the requires list has own setup.py and virtual environment. I currently have pip.conf and setup.cfg etc. setup to pull this dependency from devpi repository.

How do I configure python setup.py develop to pull the pyramid_core dependent packages using virtualenv?


*************** setup.py ****************** 

import os 
import sys 

from setuptools import setup, find_packages 

here = os.path.abspath(os.path.dirname(__file__)) 
with open(os.path.join(here, 'README.md')) as f: 
    README = f.read() 
with open(os.path.join(here, 'CHANGES.md')) as f: 
    CHANGES = f.read() 

requires = [ 
    'cryptography', 
    'odfpy', 
    'PyJWT', 
    'pycrypto', 
    'pyramid', 
    *****   'pyramid_core',  *******   
    requirements.txt file? 
    'pyramid_debugtoolbar', 
    'pyramid_tm', 
    'requests==2.18.4', 
    'SQLAlchemy', 
    'transaction', 
    'zope.sqlalchemy', 
    'waitress', 
    'psycopg2-binary', 
    'python-dateutil', 
    'uwsgi', 
    'marshmallow-sqlalchemy', 
    ] 

setup_requires = [ 
    'pytest-runner', 
] 

tests_require = [ 
    'boto3', 
    'lovely-pytest-docker', 
    'pytest', 
    'pytest-cov', 
    'tuspy', 
    'WebTest >= 1.3.1', 
] 

setup(name='api', 
      version='0.0', 
      description='api', 
      long_description=README + '\n\n' + CHANGES, 
      classifiers=[ 
          "Programming Language :: Python", 
          "Framework :: Pyramid", 
          "Topic :: Internet :: WWW/HTTP", 
          "Topic :: Internet :: WWW/HTTP :: WSGI :: Application", 
      ], 
      author='simon pears', 
      author_email='roughlea-music at outlook.com', 
      url='', 
      keywords='web wsgi bfg pylons pyramid', 
      packages=find_packages('src'), 
      package_dir={'': 'src'}, 
      include_package_data=True, 
      zip_safe=False, 
      extras_require={ 
          'testing': tests_require, 
      }, 
      install_requires=requires, 
      setup_requires=setup_requires, 
      tests_require=tests_require, 
      test_suite='tests', 
      entry_points="""\ 
      [paste.app_factory] 
      main = api:main 
      [console_scripts] 
      initialize_api_db = api.scripts.initializedb:main 
      """, 
      )