Mailman 3 September 2016 - Distutils-SIG

Re: [Distutils] Publicly Queryable Statistics
by Jason Antman 18 Sep '16

18 Sep '16

Donald, Thanks so much for all of your work on this. I know it's an old thread, but I've built a tool that generates some statistics from this information. It queries data for a given list of projects, or for all of a specific user's projects, caches the data locally on disk, and generates both a HTML report with a bunch of graphs, as well as download badges. For my own use, I'm running it cron'ed from my desktop once a night, and uploading the reports and badges to a public Amazon S3 bucket. The project is: https://pypi.org/project/pypi-download-stats/ Example output: http://jantman-personal-public.s3-website-us-east-1.amazonaws.com/pypi-stat… It's a bit rough around the edges, and currently doesn't have any unit tests - my hope is that this will be an interim solution until Warehouse has built-in stats, but I'd be happy to polish it up a bit as time allows if anyone finds it useful. Side note for Donald: It appears that the dataset currently contains data for 2016-01-22 to 2016-03-06 and 2016-05-22 to current. Is there any plan or possibility of backfilling either the 2016-03-07 to 2016-05-21 gap, or the older data? -Jason Antman

2 1

Module Installation Issues
by Mills, Ryan 16 Sep '16

16 Sep '16

I just recently downloaded Python 3.5 and cannot seem to install any packages like Numpy, etc. I have tried all the instructions on the website and keep getting errors: For example, when I type "python -m pip install Numpy" it returns a Syntax Error. I am completely new to Python so I must be missing something here - I haven't altered any files since installing it the other day. Do I use the Python IDLE Shell? Are there other packages I need to install first? Any help would be greatly appreciated. -Ryan Ryan Mills Quantitative Risk Analyst, Banking Officer [cid:image001.jpg@01CA3228.EF8AEBA0] Capital Analytics & Stress Testing 2000 McKinney Avenue, Suite 700 Dallas, TX 75201 214.932.6653 direct 20.6653 internal ryan.mills(a)texascapitalbank.com<mailto:%0bryan.mills@texascapitalbank.com> [cid:image001.jpg@01CA3228.EF8AEBA0] [TCB_horiz_Log_rgb] [cid:image004.jpg@01D20DA2.E1988D60] If you are not the addressee and have received this email in error, please notify me immediately. This email is confidential and may contain privileged or proprietary information that is unlawful for you to read, copy, distribute, disclose or otherwise use in any way.

13 29

Buoldout2 e setup.py: install_requires ignored
by Alessandro Dentella 14 Sep '16

14 Sep '16

Hi, I just migrated from buildout 1.7 to buildout 2.5. In this migration I stopped using a recipe that created a virtualenv as a part of buildout and I now use an external (basic) virtualenv to calll bootrap, so I can't compare the two configuration in a strict way. My problem is that now it seems that install_requires declared in setup.py are simply ignored, while all packages declared in the eggs directory are used/installed along with all the packages they depends on. I can't find what's wrong with my configuration, I report below a minimal setup that shows the problem. thanks in advance for any hints sandro *:-) sandro@bluff:/tmp/new$ cat buildout.cfg [buildout] parts = django eggs = django ipython django_extensions project-name = mytest [django] recipe = djangorecipe settings = settings eggs = ${buildout:eggs} ---------------------------------------------------- sandro@bluff:/tmp/new$ cat setup.py from setuptools import setup, find_packages setup( name = "example with buildout2", version = "0.1", author = "Sandro", author_email = "sandro(a)e-den.it", description = ("Sperem ghe la vaga ben"), license = "BSD", url = "http://hg.trepalchi.it/", #packages=['an_example_pypi_project', 'tests'], long_description="Bla bla", classifiers=[ "Development Status :: 3 - Alpha", "Topic :: Utilities", "License :: OSI Approved :: BSD License", ], install_requires=[ 'pyquery', ] ) The steps I do:: mkvirtualenv -p /usr/bin/python3 base3 /home/sandro/.virtualenvs/base3/bin/python3 bootstrap.py ./bin/buildout And I obtain bin/django where pyquery **is missing** $ cat bin/django #!/home/sandro/.virtualenvs/base3/bin/python3 import sys sys.path[0:0] = [ '/home/sandro/.buildout/eggs/Django-1.10.1-py3.5.egg', '/home/sandro/.buildout/eggs/ipython-5.0.0-py3.5.egg', '/home/sandro/.buildout/eggs/djangorecipe-2.2.1-py3.5.egg', '/home/sandro/.buildout/eggs/zc.recipe.egg-2.0.3-py3.5.egg', '/home/sandro/.buildout/eggs/zc.buildout-2.5.3-py3.5.egg', '/home/sandro/.buildout/eggs/pexpect-4.2.1-py3.5.egg', '/home/sandro/.buildout/eggs/Pygments-2.1.3-py3.5.egg', '/home/sandro/.buildout/eggs/prompt_toolkit-1.0.7-py3.5.egg', '/home/sandro/.buildout/eggs/traitlets-4.2.2-py3.5.egg', '/home/sandro/.buildout/eggs/simplegeneric-0.8.1-py3.5.egg', '/home/sandro/.buildout/eggs/pickleshare-0.7.4-py3.5.egg', '/home/sandro/.buildout/eggs/decorator-4.0.10-py3.5.egg', '/home/sandro/.buildout/eggs/setuptools-27.1.2-py3.5.egg', '/home/sandro/.buildout/eggs/ptyprocess-0.5.1-py3.5.egg', '/home/sandro/.buildout/eggs/wcwidth-0.1.7-py3.5.egg', '/home/sandro/.buildout/eggs/six-1.10.0-py3.5.egg', '/home/sandro/.buildout/eggs/ipython_genutils-0.1.0-py3.5.egg', '/tmp/new', ] import djangorecipe.binscripts if __name__ == '__main__': sys.exit(djangorecipe.binscripts.manage('project.settings')) -- Sandro Dentella *:-) http://trepalchi.it Il portale degli artisti http://www.reteisi.org Soluzioni libere per le scuole http://sqlkit.argolinux.org SQLkit home page - PyGTK/python/sqlalchemy

4 5

Code for inserting paths into system configuration?
by Matthew Brett 13 Sep '16

13 Sep '16

Hi, I was reflecting on the detail of putting pip user bin directories on the user's path, and was then thinking of making a tiny pip package like this: pip install pip_user_config python -m pip_user_config where the latter would cleverly insert the user bin directory into the user's configuration. Does anyone know of any code to do something like that? Best, Matthew

3 3

namespace_package
by KP 11 Sep '16

11 Sep '16

I'm not sure where the issue is, but when I specify a namespace_package in the setup.py file, I can indeed have multiple packages with the same base (foo.bar, foo.blah, etc...). The files all install in to the same directory. It drops the foo/__init__.py that would be doing the extend_path, and instead adds a ".pth" file that is a bit over my head. The problem is that it does not seem to traverse the entire sys.path to find multiple foo packages. If I do not specify namespace_packages and instead just use the pkgutil.extend_path, then this seems to allow the packages to be in multiple places in the sys.path. Is there something additional for the namespace_package that i need to specify in order for all of the sys.path to be checked? I'm using 18.5 setuptools....but I am not sure if this somehow ties in to wheel/pip, since I'm using that for the actual install.

4 19

[proposal] module dependency specification and overriding for packages
by Ronny Pfannschmidt 11 Sep '16

11 Sep '16

Hi, while trying to vendor packages that depend on six, i noticed a problematic implication it is impossible to do this without patching those modules, as such adding a fragile maintenance burden. in order to elevate that i propose creating a mechanism that allows to physically or virtually map sets of interdependent libraries (that are prepared for it into a sub-package of the package vendor-ing it this would elevate a few issues a) removing the need to patch b) making de-vendor-ing of exact versions trivial (just unpacked wheels and adding it to the sub-packages __path__) c) easing updates, as the vendor-ing turns into a process of fetching the wheels and unpacking them into the package folder i do have some implementation ideas, but i would like to hear the opinions of others before proposing an exact implementation -- Ronny

4 6

Improved Stats Features in Python
by Rui Sarmento 10 Sep '16

10 Sep '16

Dear Sirs, I've implemented some stats functions related to Factor Analysis in the statistics area. Specifically, the KMO test and the Bartlett test also. At this time I do not seem to find any module performing these tests. Is there any chance I could add these functions to a package in Python. What is the procedure to perform such contribution. Thank you very much in advance for the suggestions. Best Regards, Rui

3 8

Pep 440 question/clarification Version specifiers no "Or" available
by Matthias Bussonnier 08 Sep '16

08 Sep '16

Hi all, In the line of recent discussions[1] about releasing some packages only for some versions of Python. I was reading pep 440 and in particular the "Version specifier section". The following sentence caught my eye: > The comma (",") is equivalent to a logical and operator: a candidate version must match all given version clauses in order to match the specifier as a whole. Is there no way to have an "or" ? It seem to me that it might be useful for package wanting to express compatibility with `2.6`, `2.7` or `>3.3` for example, in the form `=~2.6 <or operator> =~3.3`. I completely agree that the use case is _limited_ and likely rare, I was just wondering if I'm missing something, if it was an oversight and if not if adding a section detailing that "or" was not considered, or that "or" was explicitly not included for X,Y,Z reason would be a good thing. Thanks, -- Matthias [1]:https://mail.python.org/pipermail/distutils-sig/2016-August/029604.html

4 5

Fwd: What is the official position on distutils?
by Sylvain Corlay 05 Sep '16

05 Sep '16

Hi Nick (re-sending to the list) On Sun, Sep 4, 2016 at 5:16 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > > It's important to note that one of our key goals in PyPA is to get to > a point where the API of a developer's publishing toolchain of choice > only depends on the *version of the publishing tools* they're using, > and not on the specific version of Python (as long as they're using a > supported version). Encouraging the expansion and increased use of the > distutils API runs counter to that goal in most cases (since it > encourages writing publication and distribution code that will only > work with newer versions of Python), but may sometimes make sense if > it's info that really is CPython centric, or if it's a feature > primarily aimed at tightly controlled environments that can > aggressively drop support for older versions. Today's state of things is that any project that is not a trivial pure-python module often has use of a combination of all three libraries: distutils, setuptools and pip in their setup.py and in a convoluted way. Newcomers to the toolchain look in similar projects for the right incantations that will fulfill their needs. We are pretty far from being in a state where good practices can even be defined. As a package author, I generally make sure that my need is not fulfilled by a proper usage of the standard library before adding a dependency to a third-party library, but packaging is an area where this does not work. Requiring pip only to know where distutils puts assets is pretty bad. Interestingly, on the subject of building extensions (where the has_flag method discussion applies), distutils.ccompiler is the only available tool and happens to not be monkey-patched by setuptools at the moment. Although setuptools re-defines buildext (and tries to get a version from cython first!) So it seems that there no real "right place" within setuptools to monkey-patch distutils' ccompiler. Sylvain

2 1

What is the official position on distutils?
by Brett Cannon 04 Sep '16

04 Sep '16

The discussion of Sylvain's proposed changes to distutils suggests that there isn't a clear-cut agreement or position of this SIG -- and thus Python -- on changes to distutils, its future, etc. Is there an official position I'm not aware of? If not, could we get one so we know how to handle any more proposed changes to distutils? For me personally (and to start this conversation if necessary), I view distutils as the package in the stdlib that handles building extension modules in the stdlib for Python. That view means that if something doesn't service that explicit goal then it shouldn't go into distutils. Now that we are moving down the road towards making the build step configurable I'm fine with saying that distutils is not expected to work for people other than Python itself and others can use setuptools, enscons, or other projects while we continue to improve the situation to where the build system is just something you specify in pyproject.toml to generate your wheel. IOW distutils is only exposed publicly because Python doesn't hide anything, but making it useful for the general case of direct usage by people is a non-goal of the package.

11 22