Mailman 3 May 2013 - Distutils-SIG

504 timeouts on pypi.python.org
by Tres Seaver 18 May '13

18 May '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Both in the web interface and in registration / upload of releases. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver(a)palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlGWr2IACgkQ+gerLs4ltQ5bAwCffg+Zn0MwHPaxrALbCSbHXeRP 1LEAoKMyu/z9nb1+nBSktEE0spAWYQ3W =QIvr -----END PGP SIGNATURE-----

3 3

problem
by Taranjeet Singh 18 May '13

18 May '13

neither distribute nor pip is installing on my laptop (windows). installing distribute gives an error of Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.39.tar.gz Extracting in c:\users\jatin\appdata\local\temp\tmpec1sv7 Now working in c:\users\jatin\appdata\local\temp\tmpec1sv7\distribute-0.6.39 Installing Distribute Something went wrong during the installation. See the error message above. Traceback (most recent call last): File "C:\Python27\distribute_setup.py", line 546, in <module> sys.exit(main()) SystemExit: 2 please help me out

2 1

Continuous Deployment Style Build System for Python (Requesting Feedback)
by James Carpenter 17 May '13

17 May '13

Please take a look and let us know what you think. Please include feedback on any confusion or errors in the docs too, so we can fix them. =========================================== Defend Against Fruit is focused on providing a pragmatic, continuous deployment style build system for Python. Current Python build systems do not properly account for the needs of effective continuous deployment. This package extends the Python tooling to add the missing pieces. With an eye to agile development principles and fast-feedback, we want a build system which satisfies the following goals: * Every SCM change-set committed should result in a potentially shippable release candidate. * When a defect is introduced, we want to immediately detect and isolate the offending SCM change-set. This is true even if the defect was introduced into a library we depend upon. * Library management should be so easy as to never impede code changes, even in multi-component architecture. More details available at: http://teamfruit.github.io/defend_against_fruit/ License: Apache Public License v2 Authors: James Carpenter jcarpenter621 at yahoo.com LinkedIn: http://www.linkedin.com/in/jamescarpenter1 Matthew Tardiff mattrix at gmail.com LinkedIn: http://www.linkedin.com/in/matthewtardiff

4 3

Continuous Deployment Style Build System for Python
by James Carpenter 16 May '13

16 May '13

Defend Against Fruit is focused on providing a pragmatic, continuous deployment style build system for Python. Current Python build systems do not properly account for the needs of effective continuous deployment. This package extends the Python tooling to add the missing pieces. With an eye to agile development principles and fast-feedback, we want a build system which satisfies the following goals: * Every SCM change-set committed should result in a potentially shippable release candidate. * When a defect is introduced, we want to immediately detect and isolate the offending SCM change-set. This is true even if the defect was introduced into a library we depend upon. * Library management should be so easy as to never impede code changes, even in multi-component architecture. More details available at: http://teamfruit.github.io/defend_against_fruit/ License: Apache Public License v2 Authors: James Carpenter jcarpenter621(a)yahoo.com LinkedIn: http://www.linkedin.com/in/jamescarpenter1 Matthew Tardiff mattrix(a)gmail.com LinkedIn: http://www.linkedin.com/in/matthewtardiff

1 0

My rst README is not formatted on pypi.python.org
by Jeremy Gillick 16 May '13

16 May '13

I wrote a valid (as far as I can tell/test) reStructuredText README file, but on pypi it's just shown as plain text. I have run the file through the rst2html.py tool and it works without errors and appears to be formatted correctly. Github is also able to display it correctly. Can someone tell me what I am doing wrong? My project is at: https://pypi.python.org/pypi/lcinvestor Are there any restrictions on pypi to file length of header hierarchy? Thanks, Jeremy

3 4

Overriding dependency versions
by John Anderson 16 May '13

16 May '13

If I have a shared dependency between 2 packages but they have pinned different versions of a 3rd package, is there a way to override this? Most of the time when people pin a version they are doing it because they have tested up to that version, but it doesn't always mean they don't work with a later version. For example: P1: sqlalchemy==0.7.6 P2: sqlalchemy P3: sqlalchemy==0.8 If I want to to override this for all packages and tell them its fine to just use 0.8.1 and ignore whatever they were pinned at, can I?

7 19

Proposal: Restrict the characters in a project name
by Donald Stufft 16 May '13

16 May '13

Currently PyPI allows a project name to contain basically any character except for a /. However most of the installation tooling doesn't not work with this wide of a namespace. It also opens up several avenues for spoofing attack where you trick people into copy and pasting an install command that looks like you're installing one package but you are really installing a different one. So I propose that moving forward that all projects/distributions are required to have names using only urlsafe characters. Specifically letters, decimal digits, hyphen, period, and underscore. Doing this would allow a better experience for people attempting to install packages, it would allow tool authors to test and make sure they can install all valid packages etc. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

9 34

Pypi Openid login problem
by Dario Bertini 15 May '13

15 May '13

Hi, I tried to login on Pypi with openid, but I keep getting back this error: on Chrome: OpenID response has been tampered with:NotAuthenticated('Replay attack detected', 9) on Firefox, strangely there's a little difference: Login failed:NotAuthenticated('Replay attack detected', 9) I tried to login from different pages, both this: https://pypi.python.org/pypi?:action=openid (which seems the plain login which I'm looking for) and this: https://pypi.python.org/pypi?:action=login&provider=Launchpad (launchpad is my Openid), after returning from launchpad, I land on a openid_return page that lets me supply some more information, (username, email and optional GPG key) with a Register button (upon clicking, I get the same error as above) Is this a known problem? thank you

1 0

Fixing Cheese Shop search
by David Wilson 14 May '13

14 May '13

Hi there, In a fit of madness caused by another 30 seconds-long PyPI search I decided to investigate the code, in the hopes of perhaps finding something simple that would alleviate the extremely long search time. I discovered what appears to be a function that makes 6 SQL queries for each term provided by the search user, in turn those queries expand to what appear to be %SUBSTRING% table scans across the releases table, which appears to contain upwards of half a gigabyte of strings. Now the root cause has been located, what to do about it? I looked at hacking on the code, but it seems webui.py is already too massive for its own good, and in any case PostgreSQL's options for efficient search are quite limited. It might be all-round good if the size of that module started to drop.. I wrote a crawler to pull a reasonable facsimile of the releases table on to my machine via the XML-RPC API, then arranged for Xapian to index only the newest releases for each package. The resulting full-text index weighs in at a very reasonable 334mb, and searches complete almost immediately, even on my lowly Intel Atom colocated server. I wrote a quick hack Flask app around it, which you can see here: http://5.39.91.176:5000/ The indexer takes as input a database produced by the crawler, which is smart enough to know how to use PyPI's exposed 'changelog' serial numbers. Basically it is quite trivial and efficient to run this setup in an incremental indexing mode. As you can see from the results, even my lowly colo is trouncing what is currently on PyPI, and so my thoughts tend toward making an arrangement like this more permanent. The crawler code weighs in at 150 lines, the indexer a meagre 113 lines, and the Flask example app is 74 lines. Implementing an exact replica of PyPI's existing scoring function is already partially done at indexing time, and the rest is quite easy to complete (mostly cutpasting code). Updating the Flask example to provide an XML-RPC API (or similar), then *initially augmenting* the old search facility seems like a good start, with a view to removing the old feature entirely. Integrating indexing directly would be pointless, the PyPI code really doesn't need anything more added to it until it gets at least reorganized a little. So for the cost of 334mb of disk, a cron job, and a lowly VPS with even just 700MB RAM, PyPI's search pains might be solved permanently. Naturally I'm writing this mail because it bothers me enough to volunteer help. :) Prototype code is here: https://bitbucket.org/dmw/pypi-search (note: relies on a pre-alpha quality DB library I'm hacking on) Thoughts?

2 3

RFC: Egg cache fro self-contained buildout
by Jim Fulton 14 May '13

14 May '13

Problem ======= For (stage and production) deployment purposed, we, ZC, use RPMs. It's considered good hygene to produce source RPMs as well as binary RPMS. This led me to create zc.sourcerelease, which automates creation of self-contained source tar balls, that, among other benefits, provide input for making source RPMs, which feed into a process for creating binary RPMs. We're moving toward a continuous deployment pipeline, where binaries are produced early in the development cycle and tested in a controlled environment that matches production. It no-longer (never did actually) makes sense to produce source RPMs that could be deployed in alternate (untested) environments. In general, our existing build process is grotesquely slow: - we run a buildout to produce a source release. - we run it again to build a source (and binary) rpm from the source release. Both of these run in such a way that all of the eggs have to be rebuilt. (But sources don't have to be downloaded.) We'd like to move toward a model where we construct a build environment for each controlled deployment environment. In this build environment, we never want to build a given distribution more than once. We need to produce application binaries that are self contained. Buildout allows you to use a shared eggs directory. This can greatly speed buildouts, because already-built distributions can be found and used locally. However, buildouts that use shared eggs directories aren't self contained. They depend on the shared eggs directory. I'd like to be able to reuse previously-built eggs, but have eggs installed in my local buildout, so it's self contained. Proposal: egg-cache =================== If egg-cache is set to a directory, then when buildout builds an egg, it will copy it to the egg cache. When looking for distributions, it will look in the egg cache and, if it finds a matching egg there, it will copy the egg to the buildout eggs directory. The end result will be that an egg cache will have the same economy as the current shared eggs directory, as far as building is concerned, but it won't have the disk-space saving of a shared eggs directiory. It will lead to buildouts that are self contained (at least wrt eggs) and that can be copied to a deployment environment directly. Thoughts? Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton

4 3