from the source, IIUC, warehouse wraps celery for Tasks: https://github.com/pypa/warehouse/blob/master/warehouse/celery.py

route to view function mappings are in https://github.com/pypa/warehouse/blob/master/warehouse/routes.py

packaging models are in https://github.com/pypa/warehouse/blob/master/warehouse/packaging/models.py

warehouse elasticsearch config is in https://github.com/pypa/warehouse/blob/master/warehouse/search.py

package search functionality (cp from sqlalchemy to elasticsearch) is in https://github.com/pypa/warehouse/blob/master/warehouse/packaging/search.py https://github.com/pypa/warehouse/blob/master/tests/unit/packaging/test_sear...

You can/could re-index the whole collection for search w/ the CLI: https://github.com/pypa/warehouse/blob/master/warehouse/cli/search/reindex.p...

Fastly downstream HTTP cache invalidation (an async Celery task) is in: https://github.com/pypa/warehouse/blob/master/warehouse/cache/origin/fastly.... https://github.com/pypa/warehouse/blob/master/tests/unit/cache/origin/test_f...

IIUC, @Donald @dstufft, there should be a call to ProjectDocType.from_db (either as a celery task or a sqla hook) somewhere that I'm unable to find with GitHub search for 'from_db' On Mar 16, 2016 11:58 AM, "Brett Cannon" brett@python.org wrote:

On Tue, 15 Mar 2016 at 09:42 Chris Barker chris.barker@noaa.gov wrote:

...

Probably not the right list but:

There are a number of folks having issue siwth new PyPi pacakges not being found by search:

https://bitbucket.org/pypa/pypi/issues/412/my-package-doesnt-show-up-in-sear...

clearly issues have been posted to the project, but has been going on a for a while, with seemingly no action, or response from PyPi developers/maintainers.

Donald can correct me where I'm wrong, but from my understanding ...

...

a) Is there somewhere else we can ping the right fols?

This is as good as any.

...

b) Is there a serious lack of folks available to address such issues?

Yes.

...

if (b), should we make a concerted effort to recruit new folks to assist PyPi maintenance / development???

There are been a couple of calls for help with Warehouse which will replace PyPI as soon as it's finished: https://github.com/pypa/warehouse (the current PyPI code base is really antiquated and from my understanding a real burden to work with at this point).

-Brett

...

Thanks,

-CHB

--

Christopher Barker, Ph.D. Oceanographer

Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig

---

Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig

b) Is there a serious lack of folks available to address such issues?

This is the correct answer. I can't even keep on top of the request from people wanting to take over packages.

if (b), should we make a concerted effort to recruit new folks to assist

PyPi maintenance / development???

That would be nice, but would be pointless until Warehouse goes live.

When do we expect that? A lot of people rely on the current system, we really need to find a way to maintain it 'till it's replaced.

CHB

On Fri, Mar 18, 2016 at 9:16 AM, Brett Cannon brett@python.org wrote:

When do we expect that? A lot of people rely on the current system, we

...

really need to find a way to maintain it 'till it's replaced.

Do realize that this is in regards to the search box, not the ability for PyPI to keep standing to serve files. Obviously if something happened to file serving or some critical aspect of PyPI then that would get addressed ASAP. But fixing a search box is not critical infrastructure so I don't think it warrants a "all hands on deck" call for help

well, it's not just the search box, it's also "pip search" from the command line.

And sure -- still not an all hands on deck, but maybe a note on the issue that the team is aware of the issue, but too understaffed and focused on the new and better to go in and fix it. just today, another note on the issue:

" Is the issue even acknowledged?"

https://bitbucket.org/pypa/pypi/issues/412/my-package-doesnt-show-up-in-sear...

This really gives the impression of a non-maintained system.

I'm not trying to be critical of a volunteer effort here -- but maybe we really do need a call for help out there. Honestly, I, for one, am poorly qualified, but it's likely there is someone out there with the requisite skills tha might want to help.

-CHB

over trying to get the replacement code done instead (I get people use the search box, but there are well-known alternatives for searching PyPI).

like pip search? also apparently broken?

In all honesty, if people want a functioning search box on PyPI I would say they should submit a patch to use https://cse.google.com/cse/ .

Maybe -- actually, it turns out a plain old google search for "PyPi package name" seems to work, once you've given Google a chance to index it.

So in the meantime maybe the search box should be removed in favor of a note suggesting a google search -- or a link that does a google search for you. A broken search is a really bad thing to have on a web site. I'm sure I'm not the only one that thought that the fact that search didn't find it meant that that it hadn't been uploaded properly.

-CHB

On Mar 18, 2016, at 1:29 PM, Donald Stufft donald@stufft.io wrote:

...

On Mar 17, 2016, at 8:51 PM, Chris Barker - NOAA Federal chris.barker@noaa.gov wrote:

When do we expect that? A lot of people rely on the current system, we really need to find a way to maintain it 'till it's replaced.

Practically speaking I’m the only person maintaining the actual code behind legacy PyPI right now. Due to some personal stuff I haven’t had the mental bandwidth to dig through debugging the ball of twine that is legacy PyPI. It’s been sort of falling apart a bit and fixing some of these issues require a substantial amount of effort. I don’t know what’s wrong with search, it’s probably not one of those kinds of issues, but the hope is that we can just squash a bunch of these at once with the new code base.

As far as “When do we expect that”, the answer is basically “when it’s ready”. You can see what’s done so far at warehouse.python.org (backed by the same live database) and the current work is being done at github.com/pypa/warehouse.

Just to make it explicit, if anyone wants to submit a patch with a cogent explanation of what was wrong I'm sure it would get fixed ASAP. Such is the eternal rallying cry of FOSS :-)

--Noah