devpi-dev March 2016

devpi-dev@python.org

8 participants
7 discussions

Can't start devpi-server on Amazon Linux 2015.09
by jblack...＠gmail.com 18 May '16

18 May '16

I'm trying to install devpi-server on Amazon Linux 2015.09 but when I start it, I'm receiving the following error: $ cat /etc/system-release Amazon Linux AMI release 2015.09 $ devpi-server --version 3.0.2 $ devpi-server --start 2016-03-22 21:53:12,909 INFO NOCTX Loading node info from /opt/devpi/.devpi/server/.nodeinfo 2016-03-22 21:53:12,910 INFO NOCTX generated uuid: 89e20f8f78fc4f2f8d8b96e121115993 fatal: The backend 'sqlite' can't be found, is the plugin not installed? I am able to import sqlite3 from Python. $ python Python 2.7.10 (default, Dec 8 2015, 18:25:23) [GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sqlite3 >>> What am I missing?

4 7

devpi web themes
by Andrzej Ostrowski 15 Apr '16

15 Apr '16

Hi, I'm playing around with devpi themes. My goal is to create a nice theme for my company. I'd like to add extra information to some pages, like the index page or the main page. I added a snippet of code which produces an "index inheritance diagram" on each index page display. Because the company I work for is not that small, these diagrams can get strange. A sample diagram: index inheritance diagram <https://lh3.googleusercontent.com/--9yr5WSohOc/VZ0BL6xN3YI/AAAAAAAAAAs/8wZq…>. Each box is an index, I intentionally removed all user and index names, except for root/pypi. It's not my indices, please don't judge me for the design :) I'm concerned about performance here. First of all, currently I generate those on display action, which doesn't seem the way to go. Could devpi provide a hook so that I'm able to regenerate a diagram on an index update event? Additionally, I'd like to add more information to an index page and the root pages. So the question is: Could functions in devpi_web.views expose more data in get_index() and root()? Potentially other functions? I know I can always get the data with python snippets in the templates, but it all takes precious time. For instance, in my company, some indices are regular user indices, some of them are project indices. I'd like to sort the list of indices on the root page: display the project ones first, and user indices below. The way I would see this is the custom_data property of an index. The bottom line: I think people could easily create fancier themes if the devpi web views functions exposed more data to the templates, even if that data is not used in the default theme. It would be great to get the full indexconfig json structure for each index in both get_index() and root() functions from devpi_web.views module. Now I'm not sure how much more data others might want to get there. I hope these things make sense. Any thoughts?

4 6

Optimized import
by Florian Schulze 25 Mar '16

25 Mar '16

Hi! I worked on optimizing import. As a side effect this also reduces replication and event handling (devpi-web indexing) times, because fewer serials are generated. A development release is available here: https://devpi.net/fschulze/dev/+f/fc1/506ba44a0f4b9/devpi-server-3.1.0.dev0… The PR with the changes is here: https://bitbucket.org/hpk42/devpi/pull-requests/347/optimize-import/diff Regards, Florian Schulze

2 1

Devpi web shows a 502 bad gateway
by Radha Krishna Srimanthula 17 Mar '16

17 Mar '16

Hi, I've followed the installation procedure exactly as mentioned in the documentation 1. Installed devpi-server and devpi-web 2. Installed supervisor and nginx 3. Generated config files 4. Copied nginx and supervisord conf files to appropriate locations 5. Started nginx and supervisor The devpi-web shows up properly but when I browse through to root/pypi, the page shows up an error "An error has occurred: 502 Bad Gateway URL u'https://pypi.python.org/simple/' returned -1. When I run the devpi-server with the same command line as in supervisor conf file along with --host 0.0.0.0, the error does not show up. Any idea how to fix this? Regards, Radha.

2 1

Devpi Usage Report
by Andrew Smith 08 Mar '16

08 Mar '16

Hi It was suggested on IRC that reporting our use of devpi here may be useful for others, so that's what I'm doing. We use devpi at my company. Actively using it is around 50 developers, 100~ python projects across 5~ branches. This amounts to 500~ releases in devpi. Many of our builds are automated and a change to a high up dependency can lead to al 100 projects being rebuilt. Each release also has documentation with it. With 2.6 we would quite often see timeouts when trying to look up packages on the devpi server. Especially when many builds were or had been occurring. We upgraded from 2.6 to 3.0.2 - I had hoped the simple index optimisation would remove these timeouts but they didn't seem to have an impact on it. We still got a lot of timeouts. I also noticed that a lot of time is spent indexing documentation. To deal with this we now run 3 --requests-only instances of devpi and 1 regular instance. We have nginx <http://doc.devpi.net/latest/quickstart-server.html> in front - I have set this up to only load balance /+status to regular and all other requests to the 3 --requests only. This has totally solved our timeout issue - the regular instance is mostly doing documentation indexing (which takes a long time when you upload 500 new releases) and the requests only instances deal with the upload/index requests admirably. Hope this might help someone and thanks for the great project. -- Andy Smith http://andrewmichaelsmith.com | @bingleybeep

2 2

devpi-server releases
by Florian Schulze 03 Mar '16

03 Mar '16

I have released devpi-server 2.6.1 and 3.0.2. The 2.6.1 release fixes an issue in exported data, that could lead to checksum mismatches on import. This often happens with custom versions like 1.0.whatever that are not PEP440 conform. The 3.0.2 release contains the same changes as 2.6.1 as well as fixes for ``mirror_whitelist`` and an issue on replicas if a mirror index returns 404. Thanks to Henk Kahlfuss for the ``mirror_whitelist`` bug report and improvement idea. Regards, Florian Schulze 2.6.1 (2016-03-03) ------------------ - add more info when importing data. Thanks Marc Abramowitz for the PR. - include version in file paths in exported data to avoid possible name conflicts. 3.0.2 (2016-03-03) ------------------ - fix setting of ``mirror_whitelist``. - normalize names when setting ``mirror_whitelist``. - fix handling of 404 in mirror indexes on replicas. - include version in file paths in exported data to avoid possible name conflicts.

1 0

Possible bug on setting whitelisted packages
by Henk Kahlfuss 02 Mar '16

02 Mar '16

Hello devpi-Team, we were using devpi-client to set the formerly pypi_whitelist and everything was working as expected. Recently we updated to devpi-server 3.0.1 (running export/import cycle), devpi-web 3.0.0, and devpi-client 2.5.0. From that moment on we switched from setting pypi_whitelist to mirror_whitelist. Unfortunately the result is not as expected: We are executing: devpi index /descript_cli/stable "mirror_whitelist=django-cms,Django,django-devserver,django-exclusivebooleanfield,South,pywin32" But as a result devpi web shows the following: whitelisted packages , , , , , - - - 2 3 D S a a a a a b c c d d d d d e e e e e e e f g g g g h i i i j j j j l l l m n n n n n n o o o o o o o p r r s s s t u u v v v w x y The command: devpi index /descript_cli/stable Results in: type=stage bases=root/pypi volatile=False acl_upload=descript_cli pypi_whitelist= mirror_whitelist=django-cms,Django,django-devserver,django-exclusivebooleanfield,South,pywin32 The whitelist setting is not working. To circumvent this issue we are using mirror_whitelist=* in the meantime to circumvent the bug. Using the asterisk works like expected. Can please someone support us on that issue? Regards, Henk Kahlfuss

2 5

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

devpi-dev March 2016