From pje at telecommunity.com Mon Aug 1 17:41:47 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 01 Aug 2005 11:41:47 -0400
Subject: [Catalog-sig] PyPI update broke setuptools
In-Reply-To: <5.1.1.6.0.20050605180352.02208810@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050801113700.026e1cf8@mail.telecommunity.com>
The new version of PyPI changed the third assumption below, breaking
setuptools' ability to do fallback searches for a request whose
upper/lowercase didn't precisely match the cases used by the package
author. Is there any chance someone could revert this, since this breaks a
ton of software in the field?
At 06:22 PM 6/5/2005 -0400, Phillip J. Eby wrote:
>For the benefit of the PyPI maintainers, here is a summary of EasyInstall's
>fairly minimal assumptions about PyPI's current behavior, assuming that
>"base-url" is the root URL of the package index:
>
> * Going to "base-url/SomePackage" produces an HTML page that either has
>a title containing "Index of Packages" and links to zero or more pages for
>specific versions, or else it is a single-version package page.
>
> * Single-version package pages may have a home page and download URL
>link, each of which occurs after '<th>Home Page' and '<th>Download URL'
>respectively, if present.
>
> * Going to "base-url/" (note trailing '/') produces an HTML page
>containing links to all active versions of registered packages
>
> * Links to package pages always have URLs of the form
>"base-url/SomePackage/itsVersion" - i.e., exactly two path parts following
>the base URL, with no query strings, parameters, fragments, etc.
>
>EasyInstall should continue to work with PyPI if these assumptions continue
>to hold.
From richardjones at optushome.com.au Tue Aug 2 00:00:11 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Tue, 2 Aug 2005 08:00:11 +1000
Subject: [Catalog-sig] PyPI update broke setuptools
In-Reply-To: <5.1.1.6.0.20050801113700.026e1cf8@mail.telecommunity.com>
References: <5.1.1.6.0.20050801113700.026e1cf8@mail.telecommunity.com>
Message-ID: <200508020800.11913.richardjones@optushome.com.au>
On Tue, 2 Aug 2005 01:41 am, Phillip J. Eby wrote:
> > * Going to "base-url/" (note trailing '/') produces an HTML page
> >containing links to all active versions of registered packages
Sorry, this didn't register when you first posted. You were relying on a
side-effect that I removed while we were trying to remove the "pypi" part of
the URL. It seemed wrong to me that PyPI would return totally different
results for:
http://cheeseshop.python.org/pypi
http://cheeseshop.python.org/pypi/
Is there any chance the software could be changed to use the more correct
"?:action=search" which is what the "/" side-effect was effectively doing?
Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050802/45443e22/attachment.pgp
From richardjones at optushome.com.au Tue Aug 2 00:24:39 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Tue, 2 Aug 2005 08:24:39 +1000
Subject: [Catalog-sig] PyPI update broke setuptools
In-Reply-To: <200508020800.11913.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050801113700.026e1cf8@mail.telecommunity.com>
<200508020800.11913.richardjones@optushome.com.au>
Message-ID: <200508020824.39254.richardjones@optushome.com.au>
On Tue, 2 Aug 2005 08:00 am, Richard Jones wrote:
> On Tue, 2 Aug 2005 01:41 am, Phillip J. Eby wrote:
> > > * Going to "base-url/" (note trailing '/') produces an HTML page
> > >containing links to all active versions of registered packages
>
> Sorry, this didn't register when you first posted. You were relying on a
> side-effect that I removed while we were trying to remove the "pypi" part
> of the URL. It seemed wrong to me that PyPI would return totally different
> results for:
>
> http://cheeseshop.python.org/pypi
> http://cheeseshop.python.org/pypi/
I can't revert the change as currently the web server redirects
http://cheeseshop.python.org to the latter, so people going directly to that
URL will get the not-so-useful list of packages, rather than the home page. I
don't have root access to fix it either, but we'll work on it.
Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050802/33d640fa/attachment.pgp
From pje at telecommunity.com Tue Aug 2 02:42:17 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 01 Aug 2005 20:42:17 -0400
Subject: [Catalog-sig] PyPI update broke setuptools
Message-ID: <5.1.1.6.0.20050801202252.02591430@mail.telecommunity.com>
>Sorry, this didn't register when you first posted. You were relying on a
>side-effect that I removed while we were trying to remove the "pypi" part of
>the URL. It seemed wrong to me that PyPI would return totally different
>results for:
>
> http://cheeseshop.python.org/pypi
> http://cheeseshop.python.org/pypi/
Well, it seemed quite *right* to me, with "pypi" being the front page, and
pypi/ being the directory listing of packages. It fit perfectly with the
rest of the URL scheme, and so I assumed it was an intentional feature, not
just a desirable one. :)
>Is there any chance the software could be changed to use the more correct
>"?:action=search" which is what the "/" side-effect was effectively doing?
Certainly it could be changed, but because the PyPI change breaks the
versions that are already in the field, it would require a new release,
which would be rather awkward at the moment, because I don't want to force
people to upgrade to the heavily refactored 0.6 series from the relatively
stable 0.5 series. I was planning to make a release of 0.6a1 this coming
weekend, but not upgrade ez_setup, so that people would need to make a
conscious decision to upgrade. Thus, I'm somewhat loathe to put an
upgraded version into wide release in order to fix an unrelated problem.
Also, the reason I didn't use the uglier search API in the first place was
because the more REST-like interface using a / seemed more compatible with
manually-created catalogs. i.e., somebody can simulate a catalog of their
own just by using static files and directories, whereas the :action=search
thing is pretty heavily tied to a single protocol version of PyPI and
requires the index to be implemented using some kind of dynamic
technology. I deliberately wanted to make EasyInstall's protocol
expectations modest enough that a wide variety of technologies could
fulfill them, so as to allow for small private or specialty indexes to
exist without needing to set up databases and such running the same
software as PyPI.
PyPI's existing URL-based API (including the '/' trick) seemed *perfect*
for this, and both python.org/pypi and cheeseshop.python.org/ seemed to
obey it.
If the software has to change, it has to change. But I'd much prefer it if
we could restore service to existing versions of the software, and it would
also be nice if I could continue to allow people to create interoperable
indexes based on static files, without EasyInstall having to do more
sophisticated (and therefore more fragile) sniffing to tell what version of
PyPI (or not) it's running against, or having to introduce command line
options to tell EasyInstall what PyPI protocol version to use, or whatever.
From richardjones at optushome.com.au Fri Aug 5 04:07:57 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Fri, 5 Aug 2005 12:07:57 +1000
Subject: [Catalog-sig] PyPI enhancement doc
In-Reply-To: <42941CE4.7070206@colorstudy.com>
References: <42941CE4.7070206@colorstudy.com>
Message-ID: <200508051207.57610.richardjones@optushome.com.au>
On Wed, 25 May 2005 04:36 pm, Ian Bicking wrote:
> This is an initial list of proposed methods for PyPI to support:
> search(field_specifiers, [operator='and']):
> package_releases(package_name):
> package_stable_version(package_name):
> package_urls(package_name, version):
> package_data(package_name, version):
I have implemented as per your spec except as follows:
> All keys are required. None/NULL is translated to ''.
Missing values are marked "UNKNOWN" at the moment. This is a hysterical oddity
introduced by the PKG-INFO format, IIRC.
> Open issues: will emails be obscured? Is keywords turned into a list?
No and no at the moment.
> I'm a little soft on these, since I don't know if specifiers and the
> necessary metadata is really ready:
>
> providing_packages(specifier):
> A list of (name, version) from release_provides. E.g.,
> providing_packages('PageTemplate>=1.0') == [('zpt', '1.0')]. This
> will only return non-hidden packages.
>
> requiring_packages(specifier):
> A list of (name, version) from release_requires.
I haven't implemented these since we don't have code that handles dependency
lookup. I believe the data and implementation spec are all ready to go
though.
Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050805/7a379c44/attachment.pgp
From amk at amk.ca Fri Aug 5 14:28:30 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 5 Aug 2005 08:28:30 -0400
Subject: [Catalog-sig] PyPI enhancement doc
In-Reply-To: <200508051207.57610.richardjones@optushome.com.au>
References: <42941CE4.7070206@colorstudy.com>
<200508051207.57610.richardjones@optushome.com.au>
Message-ID: <20050805122830.GB8968@rogue.amk.ca>
On Fri, Aug 05, 2005 at 12:07:57PM +1000, Richard Jones wrote:
> > This is an initial list of proposed methods for PyPI to support:
How should we document this API? DocXMLRPCserver.py could be used to
present a simple web page that used pydoc on the RPC methods, but we
should also have a few web pages describing it. Should it be a wiki
page, or an "official" page inside the pydotorg SVN tree?
(Describing the API would make a good oreillynet.com article. If no
one else wants to write one, I will...)
--amk
From pje at telecommunity.com Mon Aug 15 01:06:38 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 14 Aug 2005 19:06:38 -0400
Subject: [Catalog-sig] File integrity checking and host blocking for
EasyInstall
Message-ID: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
After thinking over the last week's distutils-sig discussion about
security, signatures, etc., I think I have a plan for handling basic file
integrity checking and (non-cryptographic) trust management for
EasyInstall. It is not a high-security end-to-end solution, but I think it
will allow security-conscious persons to take a more "locked down" approach
if they want to, while providing everyone else with some baseline
protection against corrupted files.
The first part of the plan is to add md5 digest checking to
EasyInstall. Because one of EasyInstall's design goals is to make it easy
for anybody to publish links to packages, we need to be able to include the
md5 signature in a package's URL. I'm thinking we could achieve this via
an '#md5=...' fragment identifier. For example, a setuptools source
archive URL might be:
http://www.python.org/packages/source/s/setuptools/setuptools-0.5a13.zip#md5=91f31a9058330174640a867cf5d4de57
The advantage of this approach is that it allows anyone to assert what the
md5 of the targeted file is, and it can be asserted in any web page, just
by pointing an HREF at the file. EasyInstall could detect the '#md5='
marker, and then use this to verify the file during download.
The disadvantage, of course, is that PyPI doesn't currently support this;
it creates a separate link to a page that displays the md5, and that URL
doesn't contain anything that connects it back to the distribution file it
refers to. I could probably create some kind of parsing hack to fix that
for PyPI, but it seems it might be worth adding the #md5 trick to PyPI to
support this.
EasyInstall would also need to grow a --require-md5 option, which would
refuse to install anything from a Subversion checkout or a distribution
without a known md5 signature.
In addition to md5 support in EasyInstall, I propose to also add it to
ez_setup; there, however, the md5 values for various distributions will be
hardcoded into ez_setup.py itself. (I'll make my "release" script append
the md5 signatures for new distributions to the end of ez_setup.py.) In
this way, the bootstrap installation of setuptools can also be reasonably
secured, as long as you trust a particular version of ez_setup.py.
The next part of the plan would be to add an --allow-hosts option to
EasyInstall. This would be a list of host wildcards that EasyInstall would
be allowed to contact. For example, --allow-hosts=*.python.org would let
EasyInstall download or scan pages from PyPI or www.python.org, but not
anywhere else. The default, if not specified, would be '*', meaning that
any host may be accessed. If EasyInstall finds itself about to download a
page or distribution from a host that isn't allowed, it will abort with a
message explaining the problem.
This would allow folks like Paul Moore to configure a default --allow-hosts
list in their pydistutils.cfg, to prevent EasyInstall from downloading
things from just any old place on the Internet. Once he's verified that he
trusts a particular site, he can edit pydistutils.cfg and add it, or else
manually download the blocked URL, publish it on a trusted intranet host, etc.
So, this is not a complete security solution, as it doesn't deal with
end-to-end file integrity, and could easily be subverted by taking over a
site somewhere in the middle (e.g. python.org). But until we have more of
the cryptographic infrastructure in place, I think this plan could provide
us with a good starting point. Comments, anyone?
From richardjones at optushome.com.au Mon Aug 15 01:37:31 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Mon, 15 Aug 2005 09:37:31 +1000
Subject: [Catalog-sig] File integrity checking and host blocking for
EasyInstall
In-Reply-To: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
References: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
Message-ID: <200508150937.35777.richardjones@optushome.com.au>
On Mon, 15 Aug 2005 09:06 am, Phillip J. Eby wrote:
> After thinking over the last week's distutils-sig discussion about
> security, signatures, etc., I think I have a plan for handling basic file
> integrity checking and (non-cryptographic) trust management for
> EasyInstall.
I'm not on the distutils-sig any more (not sure why - I thought I was and
there just wasn't any discussion going on ;) so I've missed that discussion.
> The first part of the plan is to add md5 digest checking to
> EasyInstall. Because one of EasyInstall's design goals is to make it easy
> for anybody to publish links to packages, we need to be able to include the
> md5 signature in a package's URL. I'm thinking we could achieve this via
> an '#md5=...' fragment identifier. For example, a setuptools source
> archive URL might be:
>
> http://www.python.org/packages/source/s/setuptools/setuptools-0.5a13.zip#md
>5=91f31a9058330174640a867cf5d4de57
Any idea what various "normal" browsers do when they encounter something like
this?
> The advantage of this approach is that it allows anyone to assert what the
> md5 of the targeted file is, and it can be asserted in any web page, just
> by pointing an HREF at the file. EasyInstall could detect the '#md5='
> marker, and then use this to verify the file during download.
>
> The disadvantage, of course, is that PyPI doesn't currently support this;
As with all things that "PyPI doesn't currently support" I'm happy to add
features, and even happier for others to add features :)
> So, this is not a complete security solution, as it doesn't deal with
> end-to-end file integrity, and could easily be subverted by taking over a
> site somewhere in the middle (e.g. python.org). But until we have more of
> the cryptographic infrastructure in place, I think this plan could provide
> us with a good starting point. Comments, anyone?
I presume the distutils discussion included the signing of packages that PyPI
supports?
Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050815/5c3133f4/attachment.pgp
From ehs at pobox.com Mon Aug 15 03:30:37 2005
From: ehs at pobox.com (Edward Summers)
Date: Sun, 14 Aug 2005 20:30:37 -0500
Subject: [Catalog-sig] File integrity checking and host blocking for
EasyInstall
In-Reply-To: <200508150937.35777.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
<200508150937.35777.richardjones@optushome.com.au>
Message-ID: <FE2650C0-B6BC-4F5E-8980-43C928DC04A8@pobox.com>
On Aug 14, 2005, at 6:37 PM, Richard Jones wrote:
> As with all things that "PyPI doesn't currently support" I'm happy
> to add
> features, and even happier for others to add features :)
Speaking of adding features, is the PyPI CheeseShop source code
available in svn somewhere? How about the leatest db snapshot? I've
been playing around with the xmlrpc interface a bit and wondered how
a few things were working. Consider this volunteering to help out if
any hands are needed :-)
//Ed
From richardjones at optushome.com.au Mon Aug 15 03:40:36 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Mon, 15 Aug 2005 11:40:36 +1000
Subject: [Catalog-sig] File integrity checking and host blocking for
EasyInstall
In-Reply-To: <FE2650C0-B6BC-4F5E-8980-43C928DC04A8@pobox.com>
References: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
<200508150937.35777.richardjones@optushome.com.au>
<FE2650C0-B6BC-4F5E-8980-43C928DC04A8@pobox.com>
Message-ID: <200508151140.43818.richardjones@optushome.com.au>
On Mon, 15 Aug 2005 11:30 am, Edward Summers wrote:
> On Aug 14, 2005, at 6:37 PM, Richard Jones wrote:
> > As with all things that "PyPI doesn't currently support" I'm happy
> > to add
> > features, and even happier for others to add features :)
>
> Speaking of adding features, is the PyPI CheeseShop source code
> available in svn somewhere?
The code is in the pydotorg svn repos:
https://svn.python.org/packages/trunk/pypi
> How about the leatest db snapshot?
We shouldn't really make db snapshots available as the db contains contact
info for all the submitters. If you'd like to write up a script that
generates dummy packages for testing, that'd be cool :)
Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050815/c160b0d7/attachment.pgp
From ehs at pobox.com Thu Aug 18 17:43:32 2005
From: ehs at pobox.com (Edward Summers)
Date: Thu, 18 Aug 2005 10:43:32 -0500
Subject: [Catalog-sig] File integrity checking and host blocking for
EasyInstall
In-Reply-To: <200508151140.43818.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050814183910.02704078@mail.telecommunity.com>
<200508150937.35777.richardjones@optushome.com.au>
<FE2650C0-B6BC-4F5E-8980-43C928DC04A8@pobox.com>
<200508151140.43818.richardjones@optushome.com.au>
Message-ID: <83560D0C-3BF2-431D-9C34-FFD7B80549CD@pobox.com>
> We shouldn't really make db snapshots available as the db contains
> contact
> info for all the submitters. If you'd like to write up a script that
> generates dummy packages for testing, that'd be cool :)
That sounds like a good idea, and I'd be willing to take a stab at
it. Perhaps it would be easier to dump the existing db, and write a
program to obliterate the passwords in the snapshot? Are you worried
about people grabbing email addresses? Aren't those already available
via HTTP now?
It should be fairly trivial to write a program that copies the latest
text pgdump, obscures what needs to be obscured, and makes it
available for download no?
//Ed
From amk at amk.ca Fri Aug 19 00:22:18 2005
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 18 Aug 2005 18:22:18 -0400
Subject: [Catalog-sig] 'Hidden' field display bug
Message-ID: <20050818222218.GA23547@rogue.amk.ca>
For my packages on PyPI, the package editing page shows them as all
hidden; the 'Hide?' column is all "Yes". I went into the database and
toggled the hidden field for some of my packages, but that didn't make
any difference to what was displayed
Here's the relevant bit of the template. I don't know ZPT; is the
following excerpt correct?
<select tal:attributes="name string:hid_${release/version}">
<option value="0" tal:attributes="selected release/_pypi_hidden">No</option
>
<option value="1" tal:attributes="selected not:release/_pypi_hidden">Yes</o
ption>
</select>
Things like the version field are displayed correctly, so the variable
'release' seems to be defined. Is that attribute syntax correct?
--amk
From pje at telecommunity.com Fri Aug 19 01:08:23 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 18 Aug 2005 19:08:23 -0400
Subject: [Catalog-sig] File integrity checking and host blocking for
EasyInstall
Message-ID: <5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
On Mon, 15 Aug 2005, Richard Jones wrote:
>On Mon, 15 Aug 2005 09:06 am, Phillip J. Eby wrote:
> > The first part of the plan is to add md5 digest checking to
> > EasyInstall. Because one of EasyInstall's design goals is to make it easy
> > for anybody to publish links to packages, we need to be able to include the
> > md5 signature in a package's URL. I'm thinking we could achieve this via
> > an '#md5=...' fragment identifier. For example, a setuptools source
> > archive URL might be:
> >
> > http://www.python.org/packages/source/s/setuptools/setuptools-0.5a13.zip#md
> >5=91f31a9058330174640a867cf5d4de57
>
>Any idea what various "normal" browsers do when they encounter something like
>this?
Well, Firefox works fine. I haven't tried IE or Mozilla or anything
else. That is a valid link above, so if you still have the original email
you can click on it and see what happens. :)
> > So, this is not a complete security solution, as it doesn't deal with
> > end-to-end file integrity, and could easily be subverted by taking over a
> > site somewhere in the middle (e.g. python.org). But until we have more of
> > the cryptographic infrastructure in place, I think this plan could provide
> > us with a good starting point. Comments, anyone?
>
>I presume the distutils discussion included the signing of packages that PyPI
>supports?
Yes; I have yet to encounter a signed package yet, though, and I also don't
want to try to implement signature checking in the 0.6 series of
setuptools. I'll first need to learn to use GPG and get a real
understanding of how it deals with trust chains, and understand what
EasyInstall should do with them. Unless of course somebody already knows
what the design should be and can explain it well enough for me to
implement. :)
Even then though, I'd like to get 0.6a1 out the door first. I'm not 100%
sure 0.6a1 will even do any of the md5 stuff I proposed.
From richardjones at optushome.com.au Fri Aug 19 01:15:29 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Fri, 19 Aug 2005 09:15:29 +1000
Subject: [Catalog-sig] File integrity checking and host blocking for
EasyInstall
In-Reply-To: <5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
References: <5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
Message-ID: <200508190915.33852.richardjones@optushome.com.au>
On Fri, 19 Aug 2005 09:08 am, Phillip J. Eby wrote:
> On Mon, 15 Aug 2005, Richard Jones wrote:
> > On Mon, 15 Aug 2005 09:06 am, Phillip J. Eby wrote:
> > > http://www.python.org/packages/source/s/setuptools/setuptools-0.5a13.zi
> > >p#md 5=91f31a9058330174640a867cf5d4de57
> >
> >Any idea what various "normal" browsers do when they encounter something
> > like this?
>
> Well, Firefox works fine. I haven't tried IE or Mozilla or anything
> else. That is a valid link above, so if you still have the original email
> you can click on it and see what happens. :)
I'll be wanting to hear feedback from someone who has IE before I make the
jump. I just worry too much, probably :)
> > > So, this is not a complete security solution, as it doesn't deal with
> > > end-to-end file integrity, and could easily be subverted by taking over
> > > a site somewhere in the middle (e.g. python.org). But until we have
> > > more of the cryptographic infrastructure in place, I think this plan
> > > could provide us with a good starting point. Comments, anyone?
> >
> >I presume the distutils discussion included the signing of packages that
> > PyPI supports?
>
> Yes; I have yet to encounter a signed package yet
http://www.python.org/pypi/roundup :)
Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050819/57b9350b/attachment.pgp
From pje at telecommunity.com Fri Aug 19 02:12:35 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 18 Aug 2005 20:12:35 -0400
Subject: [Catalog-sig] File integrity checking and host blocking for
EasyInstall
In-Reply-To: <200508190915.33852.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
<5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050818201200.01b34878@mail.telecommunity.com>
At 09:15 AM 8/19/2005 +1000, Richard Jones wrote:
>On Fri, 19 Aug 2005 09:08 am, Phillip J. Eby wrote:
> > Yes; I have yet to encounter a signed package yet
>
>http://www.python.org/pypi/roundup :)
FYI, the source package's signature file is empty.
From pje at telecommunity.com Fri Aug 19 03:11:21 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 18 Aug 2005 21:11:21 -0400
Subject: [Catalog-sig] File integrity checking and host blocking for
EasyInstall
In-Reply-To: <200508191022.52333.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050818201200.01b34878@mail.telecommunity.com>
<5.1.1.6.0.20050818190358.01b3eeb0@mail.telecommunity.com>
<5.1.1.6.0.20050818201200.01b34878@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050818210951.01b349b0@mail.telecommunity.com>
At 10:22 AM 8/19/2005 +1000, Richard Jones wrote:
>On Fri, 19 Aug 2005 10:12 am, you wrote:
> > At 09:15 AM 8/19/2005 +1000, Richard Jones wrote:
> > >On Fri, 19 Aug 2005 09:08 am, Phillip J. Eby wrote:
> > > > Yes; I have yet to encounter a signed package yet
> > >
> > >http://www.python.org/pypi/roundup :)
> >
> > FYI, the source package's signature file is empty.
>
>Not for me it isn't.
How odd; my Firefox shows it consistently empty, but not for the binary
package, and if I try to access the file from a different machine, it's not
empty.
<sigh> Browsers. Can't live with 'em, can't live without 'em.
From westside_indie at yahoo.com Sun Aug 21 18:27:32 2005
From: westside_indie at yahoo.com (John Walton)
Date: Sun, 21 Aug 2005 09:27:32 -0700 (PDT)
Subject: [Catalog-sig] networking programming
Message-ID: <20050821162732.30206.qmail@web31007.mail.mud.yahoo.com>
Hello, everyone. I just began school, and they
already assigned us science fair. Since I'm in 8th
grade, I get to do demonstrations for our projects.
I'm probably going to demonstrate Python's networking
capabilities by writing a simple instant messenger
program. I only have a few problems:
1. I know squat about Python network Programming
2. I know nothing about networks
So if any of you know of a good Python Networking
Tutorial or a website with lots of information on
networks and networking, please reply. Thanks!
__________________________________
Yahoo! Mail for Mobile
Take Yahoo! Mail with you! Check email on your mobile phone.
http://mobile.yahoo.com/learn/mail
From ianb at colorstudy.com Sun Aug 21 21:20:39 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 21 Aug 2005 14:20:39 -0500
Subject: [Catalog-sig] networking programming
In-Reply-To: <20050821162732.30206.qmail@web31007.mail.mud.yahoo.com>
References: <20050821162732.30206.qmail@web31007.mail.mud.yahoo.com>
Message-ID: <4308D407.9080200@colorstudy.com>
John Walton wrote:
> Hello, everyone. I just began school, and they
> already assigned us science fair. Since I'm in 8th
> grade, I get to do demonstrations for our projects.
> I'm probably going to demonstrate Python's networking
> capabilities by writing a simple instant messenger
> program. I only have a few problems:
>
> 1. I know squat about Python network Programming
>
> 2. I know nothing about networks
>
> So if any of you know of a good Python Networking
> Tutorial or a website with lots of information on
> networks and networking, please reply. Thanks!
It would probably be better to ask on the tutor or general python lists:
http://mail.python.org/mailman/listinfo/tutor
http://mail.python.org/mailman/listinfo/python-list
--
Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org
From westside_indie at yahoo.com Mon Aug 22 05:49:51 2005
From: westside_indie at yahoo.com (John Walton)
Date: Sun, 21 Aug 2005 20:49:51 -0700 (PDT)
Subject: [Catalog-sig] Networking Information
Message-ID: <20050822034951.38527.qmail@web31011.mail.mud.yahoo.com>
Hello. It's me again. Thanks for all the help with
the Python Networking Resources, but does anyone know
what I'll need to know to write a paper on Network
Programming and Python. Like terminology and all
that. Maybe I'll have a section on socketets, TCP,
Clients (half of the stuff I don't even know what it
means). So, does anyone know any good websites with
Network Programming information. Thanks!
John
____________________________________________________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs
From pje at telecommunity.com Mon Aug 22 06:03:10 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 00:03:10 -0400
Subject: [Catalog-sig] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
Message-ID: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
This thing's been in limbo too long, so I've made a prerelease
snapshot. You can get it with:
easy_install -f http://peak.telecommunity.com/snapshots/ setuptools
I have fixed all reported bugs, and implemented quite a few new features
like MD5 checking (which works even with PyPI's md5 data) and an enhanced
"build_ext --inplace" that lets you switch between different Python
interpreters without having to recompile your extensions. (The extensions
are built in build/, then copied to the source tree whenever you
"build_ext" using a different interpreter than the last.)
If you have any projects using setuptools, I encourage you to give this
snapshot a try, so I can find out if there are still any lurking bugs. If
there are no bug reports by next weekend, this will get pushed out as 0.6a1.
If you want to look at the documentation, use CVS or browse the
EasyInstall.txt, setuptools.txt, and pkg_resources.txt files found at:
http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/setuptools/
(I just made some changes to these, however, that may take a few hours to
update in SourceForge's anonymous CVS.)
From richardjones at optushome.com.au Mon Aug 22 06:30:09 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Mon, 22 Aug 2005 14:30:09 +1000
Subject: [Catalog-sig] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
In-Reply-To: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
References: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
Message-ID: <200508221430.10167.richardjones@optushome.com.au>
On Mon, 22 Aug 2005 02:03 pm, Phillip J. Eby wrote:
> I have fixed all reported bugs, and implemented quite a few new features
> like MD5 checking (which works even with PyPI's md5 data)
Glad you could get things going in spite of PyPI :)
I'm going to be busy with PyWeek for the next few weeks. Once that's out of
the way, I can do some PyPI stuff if need be (of course, anyone else is free
to work on PyPI too... :)
Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050822/bd2ba436/attachment.pgp
From pje at telecommunity.com Mon Aug 22 14:59:41 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 08:59:41 -0400
Subject: [Catalog-sig] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
In-Reply-To: <200508221430.10167.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
<5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050822023224.01b22720@mail.telecommunity.com>
At 02:30 PM 8/22/2005 +1000, Richard Jones wrote:
>On Mon, 22 Aug 2005 02:03 pm, Phillip J. Eby wrote:
> > I have fixed all reported bugs, and implemented quite a few new features
> > like MD5 checking (which works even with PyPI's md5 data)
>
>Glad you could get things going in spite of PyPI :)
I didn't mean it *that* way. In fact, one of the other changes in this
release is that I took out the workarounds for .egg uploads not being
supported.
From ianb at colorstudy.com Mon Aug 22 20:26:27 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 22 Aug 2005 13:26:27 -0500
Subject: [Catalog-sig] setuptools or PyPI problem...?
Message-ID: <430A18D3.5040700@colorstudy.com>
I don't know which side this belongs to, but I had a problem when I
tried to create a package with a "-" in it ("Paste-Deploy"). setup.py
register worked fine, and created a "Paste-Deploy" project; however,
when I did an upload it created a "Paste_Deploy-0.1.tar.gz" file, and
PyPI wouldn't accept it, I believe because it thought it belonged to the
(nonexistant) Paste_Deploy project. In the end I just smushed the words
together, but I figure this really should work. I don't know if it's
setuptools (or distutils) that is uploading to PyPI incorrectly, or PyPI
that is mismatching projects.
--
Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org
From pje at telecommunity.com Mon Aug 22 20:43:44 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 14:43:44 -0400
Subject: [Catalog-sig] [Distutils] setuptools or PyPI problem...?
In-Reply-To: <430A18D3.5040700@colorstudy.com>
Message-ID: <5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>
At 01:26 PM 8/22/2005 -0500, Ian Bicking wrote:
>I don't know which side this belongs to, but I had a problem when I
>tried to create a package with a "-" in it ("Paste-Deploy"). setup.py
>register worked fine, and created a "Paste-Deploy" project; however,
>when I did an upload it created a "Paste_Deploy-0.1.tar.gz" file, and
>PyPI wouldn't accept it, I believe because it thought it belonged to the
>(nonexistant) Paste_Deploy project. In the end I just smushed the words
>together, but I figure this really should work. I don't know if it's
>setuptools (or distutils) that is uploading to PyPI incorrectly, or PyPI
>that is mismatching projects.
Try the latest setuptools; you shouldn't have this problem as I changed it
not to create anything other than eggs with escaped '-' characters.
However, it would be very nice to have PyPI support escaped
nonalphanumerics in filenames, as they are a bitch to deal with
otherwise. Eggs absolutely have to have an unambiguously parseable
filename, and the only way to do that is by escaping '-' to '_'. This
means that you can't upload eggs for a project named e.g. 'Paste-Deploy',
if PyPI rejects a Paste_Deploy-whatever.egg file.
On the broader scope of things, I'd like to see PyPI smash all
non-alphanumeric runs in project names to a single '-', and use
case-insensitive project name comparison. I'd attempt to try my hand at
PyPI patches but at the moment don't have any obvious way to test them.
From martin at v.loewis.de Tue Aug 23 00:01:00 2005
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 23 Aug 2005 00:01:00 +0200
Subject: [Catalog-sig] setuptools or PyPI problem...?
In-Reply-To: <430A18D3.5040700@colorstudy.com>
References: <430A18D3.5040700@colorstudy.com>
Message-ID: <430A4B1C.8020900@v.loewis.de>
Ian Bicking wrote:
> I don't know which side this belongs to, but I had a problem when I
> tried to create a package with a "-" in it ("Paste-Deploy"). setup.py
> register worked fine, and created a "Paste-Deploy" project; however,
> when I did an upload it created a "Paste_Deploy-0.1.tar.gz" file, and
> PyPI wouldn't accept it, I believe because it thought it belonged to the
> (nonexistant) Paste_Deploy project. In the end I just smushed the words
> together, but I figure this really should work. I don't know if it's
> setuptools (or distutils) that is uploading to PyPI incorrectly, or PyPI
> that is mismatching projects.
Sounds like a PyPI problem to me.
Regards,
Martin
From pje at telecommunity.com Tue Aug 23 00:03:53 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 18:03:53 -0400
Subject: [Catalog-sig] setuptools/EasyInstall 0.6a1 candidate 1 snapshot
In-Reply-To: <200508230752.25321.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050822023224.01b22720@mail.telecommunity.com>
<5.1.1.6.0.20050821235458.01afafd8@mail.telecommunity.com>
<5.1.1.6.0.20050822023224.01b22720@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050822180055.01b22d78@mail.telecommunity.com>
At 07:52 AM 8/23/2005 +1000, Richard Jones wrote:
>On Mon, 22 Aug 2005 10:59 pm, you wrote:
> > At 02:30 PM 8/22/2005 +1000, Richard Jones wrote:
> > >On Mon, 22 Aug 2005 02:03 pm, Phillip J. Eby wrote:
> > > > I have fixed all reported bugs, and implemented quite a few new
> > > > features like MD5 checking (which works even with PyPI's md5 data)
> > >
> > >Glad you could get things going in spite of PyPI :)
> >
> > I didn't mean it *that* way.
>
>I was being silly, mostly :)
>
>
> > In fact, one of the other changes in this
> > release is that I took out the workarounds for .egg uploads not being
> > supported.
>
>I thought they were supported now....
They are! That's why I was able to remove the
workarounds! :) Previously, setuptools' "upload" command renamed egg
uploads to .egg.zip and pretended they were source distributions. I've
removed those hacks, because PyPI supports egg uploads just fine now.
From richardjones at optushome.com.au Tue Aug 23 00:29:31 2005
From: richardjones at optushome.com.au (Richard Jones)
Date: Tue, 23 Aug 2005 08:29:31 +1000
Subject: [Catalog-sig] [Distutils] setuptools or PyPI problem...?
In-Reply-To: <5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>
References: <5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>
Message-ID: <200508230829.31592.richardjones@optushome.com.au>
On Tue, 23 Aug 2005 04:43 am, Phillip J. Eby wrote:
> On the broader scope of things, I'd like to see PyPI smash all
> non-alphanumeric runs in project names to a single '-', and use
> case-insensitive project name comparison. I'd attempt to try my hand at
> PyPI patches but at the moment don't have any obvious way to test them.
I'm not sure I'm comfortable with this. The only key of uniqueness for
packages is <package name>/<version> ... if we start messing with the package
name...
Richard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20050823/0bc65d7b/attachment.pgp
From pje at telecommunity.com Tue Aug 23 01:19:09 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 22 Aug 2005 19:19:09 -0400
Subject: [Catalog-sig] [Distutils] setuptools or PyPI problem...?
In-Reply-To: <200508230829.31592.richardjones@optushome.com.au>
References: <5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>
<5.1.1.6.0.20050822143617.01b224a0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20050822190414.01b19d98@mail.telecommunity.com>
At 08:29 AM 8/23/2005 +1000, Richard Jones wrote:
>On Tue, 23 Aug 2005 04:43 am, Phillip J. Eby wrote:
> > On the broader scope of things, I'd like to see PyPI smash all
> > non-alphanumeric runs in project names to a single '-', and use
> > case-insensitive project name comparison. I'd attempt to try my hand at
> > PyPI patches but at the moment don't have any obvious way to test them.
>
>I'm not sure I'm comfortable with this. The only key of uniqueness for
>packages is <package name>/<version> ... if we start messing with the package
>name...
I'm just pointing out that these changes make the packaging system *more*
robust, not less -- as long as it's done relatively soon.
Right now, we have people creating packages with names like "Python Super
Whatchamacallit!" because they're not using the distutils and therefore
don't notice that it's impossible to build a sane distribution with that
name. At the opposite extreme, we have packages with VERYMixedCase that
nobody will ever remember how to type correctly except the package author. :)
Neither of these is a big problem when EasyInstall isn't a part of the mix,
because if the data's crap, who cares? However, for EasyInstall it's not
practical to try to type "easy_install 'Python Super Whatchamacallit!'" or
"easy_install VERYMixedCase" and get it right. It would be better if these
names were canonicalized (internal to PyPI) as
"python_super_whatchamacallit_" and "verymixedcase", with those values
being used as database keys. This would prevent having both a
"VeryMixedCase" and a "VERYMixedCase" at the same time (e.g. 'sqlobject'
and 'SQLObject').
Case-insensitivity is needed because some important platforms (Windows, Mac
OS X) have case-insensitive filesystems. Ergo, 'SQLObject-1.0-py2.3.egg'
and 'sqlobject-1.0-py2.3.egg' are the *same file* on such systems, which is
just not going to work.
Similarly, escaping non-alphanumerics as '_' (in both the project name and
version) is necessary to ensure that egg filenames are unambiguously
machine-readable. Currently the distutils happily produces distributions
with names like 'Paste-Deploy-1.2-prerelease.tar.gz', which are
ambiguous. A program can't tell with any certainty whether this is package
"Paste", version "Deploy-1.2-prerelease", or package "Paste-Deploy-1.2",
version "prerelease", or whatever else. Any rule you can think of that
would sort out the ambiguity, I can probably find an *actual*
counterexample for you in the PyPI database! (For example, there are
packages with numbers in their names, so you can't use that to figure out
where the version starts.)
Currently, I work around case-sensitivity and ambiguity on PyPI by doing a
full index search if there's no correct case match. I also build an index
that contains all possible interpretations(!) of ambiguous filenames, so
that I can match against what a user asked for. Luckily, the odds of
somebody mistakenly requesting version "Deploy-1.2-prerelease" of "Paste"
and thereby getting Paste-Deploy version 1.2-prerelease instead are fairly
low. :)
However, while I can do that workaround for indexing PyPI's contents, it
doesn't work for installed eggs; the names *must* be unambiguous in order
to avoid the overhead of opening or extracting PKG-INFO in order to find
out the distribution's name. So egg files at least need to turn '-' to '_'
in their filenames.