[Distutils] PyPI v2 (Was: PyPI pull request #7)

[anatoly techtonik]

On Thu, Oct 31, 2013 at 2:38 PM, Donald Stufft <donald at stufft.io> wrote:
>
> On Oct 31, 2013, at 7:32 AM, anatoly techtonik <techtonik at gmail.com> wrote:
>
>> Why I am skeptical that new site will replace old one soon? Just
>> because I don't believe in rewrites by one man army. When you develop
>> public resource, you need to rely on external feedback.
>
> Warehouse is live at https://preview-pypi.python.org/ (No landing page yet)

Not very impressive - I don't enjoy huge box for one word query -
guess Google is small for a reason. =)

Updated https://wiki.python.org/moin/CheeseShopDev

It still needs to fill up few historical details for historic preservation:
1. How many PyPI versions was there?
2. When the first one was deployed? (web.archive.org 1st reference is
dated 2007)
3. Where it was hosted, who were authors?
4. Other interesting historic facts

I like the initiative - there is something to learn from (if you're
already proficient to read advanced web code). Still I am not yet
ready to help with something I can't use. I guess I need to wait until
beta, or until there is some two-week spring-planning process, where I
can see if there is something worthy to add.

Still I couldn't resist to ask about opinion on things that don't
match my idealistic universe.


First some background questions:
1. Everything for core development is in HG. Why Git now? Why
Mercurial suxx (three major personal annoyances will do)? Why
BitBucket suxx?
2. Why Apache 2.0? Is it because everybody is using that? Why not try
CC0/CC-BY/MIT/ISC - code under these licenses is easier to copy/paste,
and is a better investment for your time as a coder studying the stuff
and filling up the space of your brain.

3. Why the raw SQL? It cripples the ability to scale, to host package
indexes on GAE or OpenStack. And it is right at the core. Why not to
"port NDB" to SQL or use HyperDB from Roundup?
4. Why not Django/TG/... framework? (+pinax, +openid, +other_stuff)

>> You also need
>> some designer guy in a team.
>
> Have one who’ll have time in a few weeks to go over everything and make it really great :)

That's cool. Will it be an iterative process or just one shot? Do you
care about public feedback? I mean of course you do, but is there a
public channel to allow users set status if they liked the previous
version more, and what should be changed for them to change their
opinion?

>> You also need a backlog for
>> collaboration. My ETA for new PyPI is no earlier than PyCon 2014 if
>> Donald and Richard will be working on it full time.
>
> Possibly! I’m unsure of how long it will take, it’s primarily Richard and I but we’ve a
> few domain experts in particular pieces who have offered to help out as well when
> we’re ready for their pieces.

PostgreSQL is killing all the fun. It takes a few hours to get
acquainted with its way of doing things - groups as users, per table
grants and insufficient DB permissions. There should be an option to
run on SQlite for development, like in Trac, Roundup etc.

>> So, instead of
>> all-or-nothing scenario I can try to find some help with incremental
>> approach.
>
> Mostly the problem with improving the current base is every change is particularly
> dangerous. The code base is large, it’s untested, and it’s not very nice. It’s extremely
> easy to break things by accident with seemingly unrelated change. Richard and I
> have both done this multiple times.

This smells like a bad spaghetti. Do you think it will be maintainable
if it is already so fragile on the early stages of development?

> So every PR we accept has a danger of breaking
> things. This incurs a high cognitive overload for accepting a PR because it means we
> have to spin up a copy of the site and manually go through and test any feature we
> think *might* be affected which typically catches most things but not always. It’s a time
> and labor intensive process which none of us enjoy and which we’ve decided not to
> prioritize unless the pay offs are large.

If you're not doing this only for yourself and your own pleasure, then
dedicating more time to improve the process and making it more
inclusive will definitely pay off for the rest of pydotorg projects
and active part of Python community, who is still here.


Do you have a diagram of the system what are you trying to build, or
is it just an experiment? It may worth to put IDE aside and try to
de-couple some things first on the whiteboard. I am afraid that I will
never be ready to help with reinventing Django. At least not without
some research and art coverage.


Most of this work about best practices, collaboration experiments and
people communication should be made by PSF. When I donate to PSF I
hope it thinks about and develops ways to encourage collaboration,
help people grow in Python, share experience and learn from each
other. Sprints, pydotorg projects, public discussions and code reviews
- they should all work together. There are good examples of open
source collaboration in Qt, Chromium and Android projects. The process
can be streamlined and fun, but nobody can even meet each other to
work on commonly interested things. Only random people who once a year
can afford time and budget to visit PyCon. I dislike "motivational"
blog posts about initiatives, without real visible research and
actions with weekly retrospectives. For that I'd propose PSF to change
to post-factum writing style from motivational, especially when the
future mechanism of the good intent is unknown or unreliable. And the
first thing it needs to do - is to start helping people inside
pydotorg communicate with outside, removing the barriers and telling
what help is needed, who is needed and where.
-- 
anatoly t.