Mailman 3 May 2016 - devpi-dev

Serve devpi on a location other than /, nginx
by Feischi 23 May '16

23 May '16

Hi, I am trying to configure nginx to serve devpi on https://<myhost>/devpi/. The generated nginx configuration works fine on https://<myhost>. But since we are in a restricted network environment, all services have to share one nginx server instance. My current simplified configuration looks as follows: location /devpi/ { root /data/devpi/; proxy_pass http://localhost:4040; proxy_set_header X-outside-url $scheme://$host:$server_port; proxy_set_header X-Real-IP $remote_addr; expires -1; # no-cache } But I get infinite redirection from this configuration. Can you help me? Thanks! Best regards, Feischi

2 4

Can't start devpi-server on Amazon Linux 2015.09
by jblack...＠gmail.com 19 May '16

19 May '16

I'm trying to install devpi-server on Amazon Linux 2015.09 but when I start it, I'm receiving the following error: $ cat /etc/system-release Amazon Linux AMI release 2015.09 $ devpi-server --version 3.0.2 $ devpi-server --start 2016-03-22 21:53:12,909 INFO NOCTX Loading node info from /opt/devpi/.devpi/server/.nodeinfo 2016-03-22 21:53:12,910 INFO NOCTX generated uuid: 89e20f8f78fc4f2f8d8b96e121115993 fatal: The backend 'sqlite' can't be found, is the plugin not installed? I am able to import sqlite3 from Python. $ python Python 2.7.10 (default, Dec 8 2015, 18:25:23) [GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sqlite3 >>> What am I missing?

4 7

devpi-server-4.0: fixing the pip-8.1.2 issue / pep 503 compliance
by holger krekel 13 May '16

13 May '16

devpi-server-4.0: fixing the pip-8.1.2 problem / PEP503 compliance ============================================================================ We've made available critically important releases of the devpi private packaging available. If you are not using "devpi" yet then you can may just read http://doc.devpi.net and forget about the rest of this announcement. This is for the many who experienced the "pip doesn't install packages anymore with devpi" problem. First of all, you may temporarily pin "pip" to avoid the problem on the client side: pip install pip==8.1.1 This is obviously a crutch but gives you some time to perform the export/import cycle required for hosting private packages via devpi-server-4.0 and being compatible with pip-8.1.2. If you are using devpi-server as a pure pypi.python.org cache you don't need to perform export/import and can just delete your server directory ($HOME/.devpi/server by default) before you install and start up devpi-server-4.0. If you are hosting private packages on devpi you will need to perform an export/import cycle of your server state in order to run devpi-server-4.0. The "4.0" in this case only signals this export/import need -- no other big changes are coming with 4.0. At the end of this announcement we explain some details of why we needed to go for a 4.0 and not just a micro bugfix release. To export from devpi-server-3.X -------------------------------- upgrade to the new devpi-server-3.1.2 before you export, like this: pip install "devpi-server<4.0" Now stop your server and run: devpi-server --export EXPORTDIR --serverdir SERVERDIR where EXPORTDIR should be a fresh new directory and SERVERDIR should be the server state directory ($HOME/.devpi/server by default). To export from devpi-server-2.X -------------------------------- Upgrade to the latest devpi-server-2.X release: pip install "devpi-server<3.0" devpi-common>=2.0.10 Here we force the devpi-common dependency to not accidentally be "devpi-common==2.0.9" which could lead to problems. Now stop your server and run: devpi-server --export EXPORTDIR --serverdir SERVERDIR where EXPORTDIR should be a fresh new directory and SERVERDIR should be the server state directory ($HOME/.devpi/server by default). to import state into devpi-server-4.0 ---------------------------------------- Upgrade to the latest devpi-server-4.X release: pip install "devpi-server<5.0" devpi-web If you don't use "devpi-web" you can leave it out from the pip command. Check you have the right version: devpi-server --version Now import from your previously created EXPORTDIR: devpi-server --serverdir SERVERDIR_NEW --import EXPORTDIR This will take a while if you have many indexes or lots of documentation -- devpi-web will create a search index over all of it during import. You are now good to go -- pip works again! devpi-client also has a 2.6.3 -------------------------------- We also published a minor bugfix "devpi-client-2.6.3" release which should work with both devpi-server-2.6 and devpi-server-4.0 as we are generally trying to keep devpi-client forward/backward compatible. You only need to install devpi-client-2.6.3 if you also install devpi-server into the same virtual environment. Otherwise using devpi-client-2.6.2 with both devpi-server-2.6 and devpi-server-4.0 probably works fine as well. background on the pip/devpi bug for the curious ----------------------------------------------- Besides devpi, also artifactory and other private index servers have experienced failures with pip-8.1.2. The change from 8.1.1 was that pip now asks for PEP503-normalized names when requesting the simple page from an index. Previously "-" and "." would be allowed but with the new normalization "." is substituted with "-". Now "pip install zope.interface" triggers a request to "+simple/zope-interface" and devpi in turns asks pypi.python.org/simple/zope-interface and gets an answer with lots of "zope.interface-*.tar.gz" release links. But those are not matched because without PEP503 "zope.interface" and "zope-interface" are different things. Moreover, pypi.python.org used to redirect to the "true" name but does not do this anymore which contributed to the eventual problem. We decided to go for 4.0 because since 3.0 we base database keys on normalized project names -- and this normalization is used in like 20-30 code places across the devpi system and plugins. Trying to be clever and avoid the export/import and trick "pip-8.1.2" into working looked like a can of worms. Now with devpi-server-4.0 we are using proper PEP503 specified normalization so should be safe. best, holger and florian P.S.: we offer support contracts btw and thank in particular Dolby Laboratories, YouGov Inc and BlueYonder GmbH who funded a lot of the last year's devpi work and now agreed to be named in public - and no, we didn't get around to make a flashy web site yet. For now, just mail holger at merlinux to discuss support and training options.

1 0

Fix for pip 8.1.2
by Florian Schulze 13 May '16

13 May '16

Hi! I released devpi-common 2.0.9 and devpi-server 3.1.1 to fix https://github.com/pypa/pip/issues/3666 For devpi-server 2.6.1 the update to devpi-common 2.0.9 is enough. I only released devpi-server 3.1.1 because of a change in the import code in 3.1.0 which was also affected by this. I tested devpi-common 2.0.9 with devpi-server/web 2.6.x and devpi-server/web 3.1.x. Regards, Florian Schulze

6 12

announcement draft / upgrade instructions
by holger krekel 13 May '16

13 May '16

While we are still doing and asking for tests I've written a draft release announcement 4.0, see here: http://merlinux.eu/~hpk/d4/announce/server-4.0.html and see it also attached as a RST file. The releases are not available yet at pypi.python.org but if you do: export PIP_INDEX_URL=https://devpi.net/fschulze/devpi400 or use the "-i" option with PIP you can use the instructions from the draft announcement verbatim. Maybe someone is still left who hasn't tried upgrading and could test the draft release announcement? I'll also happily receive patches for the attached RST source file. best, holger

2 1

Re: [devpi-dev] problem with devpi test
by Alexis Roda 13 May '16

13 May '16

Upgrading to devpi 4 fixes my two issues. I'll report back if I find any issue with this release. Thanks agains.

1 0

Running devpi on a mounted yas3fs -- try harder, or fail fast?
by Russ Ferriday 11 May '16

11 May '16

Hi devpi experts, I've been trying to use yas3fs under devpi. Why? Because I've been beguiled by Codeship's new Docker CI/CD and the option to run up test instances automatically, test a github commit, and register package updates with devpi on success. All of this needs to run on cloud services, not on a VPN setting, so having transient devpi services connected to a global data file makes some sense. I've made a devpi service that puts S3 bucket under it <https://github.com/topiaruss/docker-devpi>self. this is based on some work by bubenkoff <https://github.com/topiaruss/docker-devpi/commit/223dc282d9a254f2ba17ed00b7…> . In my codeship configuration, I run the devpi service just to pull/push packages via devpi to my S3 bucket, then the container is discarded, leaving the packages in S3 for future tests and deployment. It partially works, at the moment, so that after I upload a package I can list it. The next instantiation of the image connects to the bucket, but never lists existing packages. I see the bucket filling with packages, but devpi list does not list them. I found that I needed to use the recheck flag, or sometimes only a few of the devpi files would be written. I'm guessing that the process closed before all the locally cached files were purged through yas3fs. yas3fs --recheck-s3 s3://$S3_PATH /mnt I'm going to play more with the yas3fs settings. Has anyone played with this? Had success? Know any immediate gotchas? --r.

2 1

don't install devpi-common 2.0.9, working on fix
by Florian Schulze 11 May '16

11 May '16

Hi! Unfortunately the fix wasn't as simple and causes further issues. If you haven't upgraded yet, don't. Use pip < 8.1.2. If you use devpi-server 2.x and already updated: Downgrade to devpi-common 2.0.8. Any packages with dots in their name uploaded while devpi-common 2.0.9 was installed will be hidden. If you can't upload them again for whatever reason, then you can recover the release files via an export using devpi-common 2.0.9, but in that export previously uploaded releases will be missing, so only use it to recover the newly uploaded files. Do a proper export with devpi-common 2.0.8 and use pip < 8.1.2 in the meantime. For devpi-server 3.x we are currently working on a fix. Regards, Florian Schulze

1 1

problem with devpi test
by Alexis Roda 11 May '16

11 May '16

Hi all, I'm having trouble running 'devpi test', it complains about: No matching distribution found for dja.dni (from hera_conveni_core->bt2-conveni-core==0.1.20) But the package "dja.dni" is there: $ devpi list dja.dni https://.../devpi/alex/dev/+f/e62/6a34488d42f5a/dja.dni-0.1.3.tar.gz https://.../devpi/alex/dev/+f/d5a/dd76fae3638ed/dja.dni-0.1.2.tar.gz https://.../devpi/alex/prod/+f/553/617872ee9bcb8/dja.dni-0.1.1.tar.gz Running the tests with "tox" is fine. Looking at the logs it seems that devpi is requesting the package "dja-dni" (translates the dot into a dash) while tox requests "dja.dni". Tested with devpi-server/client 2.6.0/2.5.0 and 3.1.0/2.6.2. Is there any workaround?

2 1

Format of index_url in pip.conf
by Dan Langer 06 May '16

06 May '16

Hello, We interact with devpi using the pip-compatible interface, by putting a value for index_url in ~/.pip/pip.conf. Looking at the documentation, some places use https://devpi_address.com/user/index/ and some places use https://devpi_address.com/user/index/+simple/ - which should we be using, and what's the difference? Thanks, Dan

2 2