From rea at swing.tm.fr Wed Nov 1 18:54:15 2006
From: rea at swing.tm.fr (Joe Owens)
Date: Wed, 1 Nov 2006 19:54:15 +0200
Subject: [Catalog-sig] martyrdom spontaneity
Message-ID: <4548DF47.1010606@ffgc.org>
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/catalog-sig/attachments/20061101/443cff55/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lapse.gif
Type: image/gif
Size: 9227 bytes
Desc: not available
Url : http://mail.python.org/pipermail/catalog-sig/attachments/20061101/443cff55/attachment.gif
From timcera at earthlink.net Mon Nov 6 02:47:30 2006
From: timcera at earthlink.net (Tim Cera)
Date: Sun, 5 Nov 2006 20:47:30 -0500
Subject: [Catalog-sig] How to get best match URL to find source?
Message-ID: <200611052047.32287.timcera@earthlink.net>
I am trying to get the best match URL that points to source. How should
I do that?
I can't use easy_install since as near as I can figure, I can't force it
to collect only source.
I have tried something like below:
pkgindex = PackageIndex()
output = pkgindex.fetch_distribution(Requirement.parse(pkg), 'sandbox',
force_scan=True, source=True)
It does download the package, but I can't find the URL anywhere. Am I
missing it? Is it possible to get to?
Kindest regards,
Tim Cera
From pje at telecommunity.com Mon Nov 6 03:46:47 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 05 Nov 2006 21:46:47 -0500
Subject: [Catalog-sig] How to get best match URL to find source?
In-Reply-To: <200611052047.32287.timcera@earthlink.net>
Message-ID: <5.1.1.6.0.20061105214154.01f21d68@sparrow.telecommunity.com>
At 08:47 PM 11/5/2006 -0500, Tim Cera wrote:
>I am trying to get the best match URL that points to source. How should
>I do that?
>
>I can't use easy_install since as near as I can figure, I can't force it
>to collect only source.
See the --editable and --build-directory arguments. Simple usage:
easy_install -eb. ProjectName
Will download projectname's source (or check it out of Subversion), and
unpack it to a directory called './projectname' (all lowercase). (The '.'
is actually whatever directory you give to the -b or --build-directory option.)
>I have tried something like below:
>
> pkgindex = PackageIndex()
> output = pkgindex.fetch_distribution(Requirement.parse(pkg), 'sandbox',
>force_scan=True, source=True)
>
>It does download the package, but I can't find the URL anywhere. Am I
>missing it? Is it possible to get to?
Not using fetch_distribution(), no. You would have to do the same steps
that fetch_distribution() does to find it.
From jim at zope.com Thu Nov 16 22:44:08 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 16 Nov 2006 16:44:08 -0500
Subject: [Catalog-sig] PyPi down again
Message-ID: <455CDBA8.5050806@zope.com>
Any idea why now?
Jim
--
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
From jim at zope.com Thu Nov 16 22:48:28 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 16 Nov 2006 16:48:28 -0500
Subject: [Catalog-sig] PyPi down again
In-Reply-To: <455CDBA8.5050806@zope.com>
References: <455CDBA8.5050806@zope.com>
Message-ID: <455CDCAC.1000509@zope.com>
Jim Fulton wrote:
> Any idea why now?
BTW, if someone will give me needed access and tell me how,
I'd be happy to help out with recovery efforts, since I seem
to notice lack of service most often.
Jim
--
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
From richardjones at optushome.com.au Thu Nov 16 22:59:24 2006
From: richardjones at optushome.com.au (Richard Jones)
Date: Fri, 17 Nov 2006 08:59:24 +1100
Subject: [Catalog-sig] PyPi down again
In-Reply-To: <455CDBA8.5050806@zope.com>
References: <455CDBA8.5050806@zope.com>
Message-ID: <200611170859.24187.richardjones@optushome.com.au>
On Friday 17 November 2006 08:44, Jim Fulton wrote:
> Any idea why now?
Just logged in:
23:00:48 up 30 days, 7:59, 2 users, load average: 34.41, 38.20, 39.90
There's a flood of apache processes, so I assume something's hitting the web
server hard.
I don't have access to the logs, so I can't say anythin beyond that.
Richard
From jim at zope.com Thu Nov 16 23:01:24 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 16 Nov 2006 17:01:24 -0500
Subject: [Catalog-sig] PyPi down again
In-Reply-To: <455CDBA8.5050806@zope.com>
References: <455CDBA8.5050806@zope.com>
Message-ID: <455CDFB4.7080701@zope.com>
Jim Fulton wrote:
> Any idea why now?
FWIW, it seems to have been very slow/overloaded rather than down.
It seems to recover for a bit and then become slow again.
Jim
--
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
From jim at zope.com Thu Nov 16 23:04:01 2006
From: jim at zope.com (Jim Fulton)
Date: Thu, 16 Nov 2006 17:04:01 -0500
Subject: [Catalog-sig] PyPi down again
In-Reply-To: <200611170859.24187.richardjones@optushome.com.au>
References: <455CDBA8.5050806@zope.com>
<200611170859.24187.richardjones@optushome.com.au>
Message-ID: <455CE051.3050302@zope.com>
Richard Jones wrote:
> On Friday 17 November 2006 08:44, Jim Fulton wrote:
>> Any idea why now?
>
> Just logged in:
>
> 23:00:48 up 30 days, 7:59, 2 users, load average: 34.41, 38.20, 39.90
>
> There's a flood of apache processes, so I assume something's hitting the web
> server hard.
>
> I don't have access to the logs, so I can't say anythin beyond that.
Thanks for looking.
Is there someone who can help? Can I help somehow?
What can we do to protect ourselves from this in the future?
(I still haven't looked at the developer info, so I'm not
in a position to recommend anything, dang it.)
Jim
--
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
From richardjones at optushome.com.au Thu Nov 16 23:11:35 2006
From: richardjones at optushome.com.au (Richard Jones)
Date: Fri, 17 Nov 2006 09:11:35 +1100
Subject: [Catalog-sig] PyPi down again
In-Reply-To: <455CE051.3050302@zope.com>
References: <455CDBA8.5050806@zope.com>
<200611170859.24187.richardjones@optushome.com.au>
<455CE051.3050302@zope.com>
Message-ID: <200611170911.35893.richardjones@optushome.com.au>
On Friday 17 November 2006 09:04, Jim Fulton wrote:
> Richard Jones wrote:
> > On Friday 17 November 2006 08:44, Jim Fulton wrote:
> >> Any idea why now?
> >
> > Just logged in:
> >
> > 23:00:48 up 30 days, 7:59, 2 users, load average: 34.41, 38.20, 39.90
> >
> > There's a flood of apache processes, so I assume something's hitting the
> > web server hard.
> >
> > I don't have access to the logs, so I can't say anythin beyond that.
>
> Thanks for looking.
>
> Is there someone who can help? Can I help somehow?
We need someone with root access to look at the logs.
I'm posting to pydotorg to see if we can get someone to look.
Without seeing the logs I'm going to guess we've got a badly-behaved web
spider hitting the browse interface.
> What can we do to protect ourselves from this in the future?
Assuming my guess about it being a spider hitting the browser interface, we
already attempt to block spiders off from the browse interface both through
the robots.txt file and by detecting their user-agent in the pypi code.
Spiders don't *have* to read robots.txt, and my RE string for user-agent
detection might be missing someone.
We won't know until we can get someone to look at the logs.
Richard
From richardjones at optushome.com.au Thu Nov 16 23:30:29 2006
From: richardjones at optushome.com.au (Richard Jones)
Date: Fri, 17 Nov 2006 09:30:29 +1100
Subject: [Catalog-sig] PyPi down again
In-Reply-To: <200611170911.35893.richardjones@optushome.com.au>
References: <455CDBA8.5050806@zope.com> <455CE051.3050302@zope.com>
<200611170911.35893.richardjones@optushome.com.au>
Message-ID: <200611170930.29273.richardjones@optushome.com.au>
On Friday 17 November 2006 09:11, Richard Jones wrote:
> On Friday 17 November 2006 09:04, Jim Fulton wrote:
> > Richard Jones wrote:
> > > 23:00:48 up 30 days, 7:59, 2 users, load average: 34.41, 38.20,
> > > 39.90
> > >
> > > There's a flood of apache processes, so I assume something's hitting
> > > the web server hard.
> > >
> > > I don't have access to the logs, so I can't say anythin beyond that.
FWIW:
23:33:45 up 30 days, 8:32, 2 users, load average: 0.31, 1.67, 12.07
Richard
From constant.beta at gmail.com Tue Nov 21 02:59:16 2006
From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=)
Date: Tue, 21 Nov 2006 02:59:16 +0100
Subject: [Catalog-sig] PyPi down again
In-Reply-To: <200611170930.29273.richardjones@optushome.com.au>
References: <455CDBA8.5050806@zope.com> <455CE051.3050302@zope.com>
<200611170911.35893.richardjones@optushome.com.au>
<200611170930.29273.richardjones@optushome.com.au>
Message-ID: <5e8b0f6b0611201759t4bdc6d3q233121cd28aa6318@mail.gmail.com>
On 11/16/06, Richard Jones <richardjones at optushome.com.au> wrote:
> > > > There's a flood of apache processes, so I assume something's hitting
> > > > the web server hard.
> > > >
> > > > I don't have access to the logs, so I can't say anythin beyond that.
>
> FWIW:
>
> 23:33:45 up 30 days, 8:32, 2 users, load average: 0.31, 1.67, 12.07
Any news on the cause of last PyPI slowdown?
Cheers,
mk
--
. o . >> http://joker.linuxstuff.pl <<
. . o It's easier to get forgiveness for being wrong
o o o than forgiveness for being right.
From richardjones at optushome.com.au Tue Nov 21 09:36:54 2006
From: richardjones at optushome.com.au (Richard Jones)
Date: Tue, 21 Nov 2006 19:36:54 +1100
Subject: [Catalog-sig] PyPi down again
In-Reply-To: <5e8b0f6b0611201759t4bdc6d3q233121cd28aa6318@mail.gmail.com>
References: <455CDBA8.5050806@zope.com>
<200611170930.29273.richardjones@optushome.com.au>
<5e8b0f6b0611201759t4bdc6d3q233121cd28aa6318@mail.gmail.com>
Message-ID: <200611211936.54121.richardjones@optushome.com.au>
On Tuesday 21 November 2006 12:59, Micha? Kwiatkowski wrote:
> On 11/16/06, Richard Jones <richardjones at optushome.com.au> wrote:
> > > > > There's a flood of apache processes, so I assume something's
> > > > > hitting the web server hard.
> > > > >
> > > > > I don't have access to the logs, so I can't say anythin beyond
> > > > > that.
> >
> > FWIW:
> >
> > 23:33:45 up 30 days, 8:32, 2 users, load average: 0.31, 1.67, 12.07
>
> Any news on the cause of last PyPI slowdown?
I've just put in a second request to pydotorg people.
Richard