Hi,
I am trying to configure nginx to serve devpi on https://<myhost>/devpi/.
The generated nginx configuration works fine on https://<myhost>. But since
we are in a restricted network environment, all services have to share one
nginx server instance.
My current simplified configuration looks as follows:
location /devpi/ {
root /data/devpi/;
proxy_pass http://localhost:4040;
proxy_set_header X-outside-url $scheme://$host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
expires -1; # no-cache
}
But I get infinite redirection from this configuration.
Can you help me? Thanks!
Best regards,
Feischi
I'm trying to install devpi-server on Amazon Linux 2015.09 but when I start
it, I'm receiving the following error:
$ cat /etc/system-release
Amazon Linux AMI release 2015.09
$ devpi-server --version
3.0.2
$ devpi-server --start
2016-03-22 21:53:12,909 INFO NOCTX Loading node info from
/opt/devpi/.devpi/server/.nodeinfo
2016-03-22 21:53:12,910 INFO NOCTX generated uuid:
89e20f8f78fc4f2f8d8b96e121115993
fatal: The backend 'sqlite' can't be found, is the plugin not installed?
I am able to import sqlite3 from Python.
$ python
Python 2.7.10 (default, Dec 8 2015, 18:25:23)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>>
What am I missing?
devpi-server-4.0: fixing the pip-8.1.2 problem / PEP503 compliance
============================================================================
We've made available critically important releases of the devpi private packaging
available. If you are not using "devpi" yet then you can may just read
http://doc.devpi.net and forget about the rest of this announcement.
This is for the many who experienced the "pip doesn't install packages
anymore with devpi" problem. First of all, you may temporarily pin "pip"
to avoid the problem on the client side:
pip install pip==8.1.1
This is obviously a crutch but gives you some time to perform the
export/import cycle required for hosting private packages via
devpi-server-4.0 and being compatible with pip-8.1.2.
If you are using devpi-server as a pure pypi.python.org cache you don't
need to perform export/import and can just delete your server directory
($HOME/.devpi/server by default) before you install and start up
devpi-server-4.0.
If you are hosting private packages on devpi you will need to perform an
export/import cycle of your server state in order to run devpi-server-4.0.
The "4.0" in this case only signals this export/import need -- no other
big changes are coming with 4.0. At the end of this announcement we explain
some details of why we needed to go for a 4.0 and not just a micro bugfix release.
To export from devpi-server-3.X
--------------------------------
upgrade to the new devpi-server-3.1.2 before you export, like this:
pip install "devpi-server<4.0"
Now stop your server and run:
devpi-server --export EXPORTDIR --serverdir SERVERDIR
where EXPORTDIR should be a fresh new directory and SERVERDIR
should be the server state directory ($HOME/.devpi/server by default).
To export from devpi-server-2.X
--------------------------------
Upgrade to the latest devpi-server-2.X release:
pip install "devpi-server<3.0" devpi-common>=2.0.10
Here we force the devpi-common dependency to not accidentally
be "devpi-common==2.0.9" which could lead to problems.
Now stop your server and run:
devpi-server --export EXPORTDIR --serverdir SERVERDIR
where EXPORTDIR should be a fresh new directory and SERVERDIR
should be the server state directory ($HOME/.devpi/server by default).
to import state into devpi-server-4.0
----------------------------------------
Upgrade to the latest devpi-server-4.X release:
pip install "devpi-server<5.0" devpi-web
If you don't use "devpi-web" you can leave it out from the pip command.
Check you have the right version:
devpi-server --version
Now import from your previously created EXPORTDIR:
devpi-server --serverdir SERVERDIR_NEW --import EXPORTDIR
This will take a while if you have many indexes or lots of documentation --
devpi-web will create a search index over all of it during import.
You are now good to go -- pip works again!
devpi-client also has a 2.6.3
--------------------------------
We also published a minor bugfix "devpi-client-2.6.3" release which
should work with both devpi-server-2.6 and devpi-server-4.0 as we
are generally trying to keep devpi-client forward/backward compatible.
You only need to install devpi-client-2.6.3 if you also install
devpi-server into the same virtual environment. Otherwise using
devpi-client-2.6.2 with both devpi-server-2.6 and devpi-server-4.0
probably works fine as well.
background on the pip/devpi bug for the curious
-----------------------------------------------
Besides devpi, also artifactory and other private index servers
have experienced failures with pip-8.1.2. The change from 8.1.1
was that pip now asks for PEP503-normalized names when requesting
the simple page from an index. Previously "-" and "." would be
allowed but with the new normalization "." is substituted with "-".
Now "pip install zope.interface" triggers a request to
"+simple/zope-interface" and devpi in turns asks
pypi.python.org/simple/zope-interface and gets an answer
with lots of "zope.interface-*.tar.gz" release links. But those
are not matched because without PEP503 "zope.interface" and "zope-interface"
are different things. Moreover, pypi.python.org used to redirect
to the "true" name but does not do this anymore which contributed
to the eventual problem.
We decided to go for 4.0 because since 3.0 we base database
keys on normalized project names -- and this normalization is
used in like 20-30 code places across the devpi system and plugins.
Trying to be clever and avoid the export/import and trick "pip-8.1.2"
into working looked like a can of worms. Now with devpi-server-4.0
we are using proper PEP503 specified normalization so should be safe.
best,
holger and florian
P.S.: we offer support contracts btw and thank in particular
Dolby Laboratories, YouGov Inc and BlueYonder GmbH who funded a lot of
the last year's devpi work and now agreed to be named in public - and
no, we didn't get around to make a flashy web site yet. For now,
just mail holger at merlinux to discuss support and training options.
Hi!
I released devpi-common 2.0.9 and devpi-server 3.1.1 to fix
https://github.com/pypa/pip/issues/3666
For devpi-server 2.6.1 the update to devpi-common 2.0.9 is enough.
I only released devpi-server 3.1.1 because of a change in the import
code in 3.1.0 which was also affected by this.
I tested devpi-common 2.0.9 with devpi-server/web 2.6.x and
devpi-server/web 3.1.x.
Regards,
Florian Schulze
While we are still doing and asking for tests I've written a draft
release announcement 4.0, see here:
http://merlinux.eu/~hpk/d4/announce/server-4.0.html
and see it also attached as a RST file. The releases are not
available yet at pypi.python.org but if you do:
export PIP_INDEX_URL=https://devpi.net/fschulze/devpi400
or use the "-i" option with PIP you can use the instructions
from the draft announcement verbatim.
Maybe someone is still left who hasn't tried upgrading and could
test the draft release announcement? I'll also happily receive
patches for the attached RST source file.
best,
holger
Hi devpi experts,
I've been trying to use yas3fs under devpi. Why? Because I've been
beguiled by Codeship's new Docker CI/CD and the option to run up test
instances automatically, test a github commit, and register package updates
with devpi on success. All of this needs to run on cloud services, not on a
VPN setting, so having transient devpi services connected to a global data
file makes some sense.
I've made a devpi service that puts S3 bucket under it
<https://github.com/topiaruss/docker-devpi>self. this is based on some work
by bubenkoff
<https://github.com/topiaruss/docker-devpi/commit/223dc282d9a254f2ba17ed00b7…>
.
In my codeship configuration, I run the devpi service just to pull/push
packages via devpi to my S3 bucket, then the container is discarded,
leaving the packages in S3 for future tests and deployment.
It partially works, at the moment, so that after I upload a package I can
list it. The next instantiation of the image connects to the bucket, but
never lists existing packages. I see the bucket filling with packages, but
devpi list does not list them.
I found that I needed to use the recheck flag, or sometimes only a few of
the devpi files would be written. I'm guessing that the process closed
before all the locally cached files were purged through yas3fs.
yas3fs --recheck-s3 s3://$S3_PATH /mnt
I'm going to play more with the yas3fs settings.
Has anyone played with this? Had success? Know any immediate gotchas?
--r.
Hi!
Unfortunately the fix wasn't as simple and causes further issues.
If you haven't upgraded yet, don't. Use pip < 8.1.2.
If you use devpi-server 2.x and already updated:
Downgrade to devpi-common 2.0.8.
Any packages with dots in their name uploaded while devpi-common 2.0.9
was installed will be hidden.
If you can't upload them again for whatever reason, then you can recover
the release files via an export using devpi-common 2.0.9, but in that
export previously uploaded releases will be missing, so only use it to
recover the newly uploaded files.
Do a proper export with devpi-common 2.0.8 and use pip < 8.1.2 in the
meantime.
For devpi-server 3.x we are currently working on a fix.
Regards,
Florian Schulze
Hi all,
I'm having trouble running 'devpi test', it complains about:
No matching distribution found for dja.dni (from
hera_conveni_core->bt2-conveni-core==0.1.20)
But the package "dja.dni" is there:
$ devpi list dja.dni
https://.../devpi/alex/dev/+f/e62/6a34488d42f5a/dja.dni-0.1.3.tar.gz
https://.../devpi/alex/dev/+f/d5a/dd76fae3638ed/dja.dni-0.1.2.tar.gz
https://.../devpi/alex/prod/+f/553/617872ee9bcb8/dja.dni-0.1.1.tar.gz
Running the tests with "tox" is fine.
Looking at the logs it seems that devpi is requesting the package "dja-dni"
(translates the dot into a dash) while tox requests "dja.dni".
Tested with devpi-server/client 2.6.0/2.5.0 and 3.1.0/2.6.2.
Is there any workaround?
Hello,
We interact with devpi using the pip-compatible interface, by putting a
value for index_url in ~/.pip/pip.conf.
Looking at the documentation, some places
use https://devpi_address.com/user/index/ and some places use
https://devpi_address.com/user/index/+simple/ - which should we be using,
and what's the difference?
Thanks,
Dan
