[Distutils] PyPI v2 (Was: PyPI pull request #7)

[anatoly techtonik]

First things first, thanks for the detailed response.

On Fri, Nov 8, 2013 at 3:41 PM, Donald Stufft <donald at stufft.io> wrote:
>
>> First some background questions:
>> 1. Everything for core development is in HG. Why Git now? Why
>> Mercurial suxx (three major personal annoyances will do)? Why
>> BitBucket suxx?
>
> I’m most familiar with git, so I used git when I started it.

Seems fair.

>> 2. Why Apache 2.0? Is it because everybody is using that? Why not try
>> CC0/CC-BY/MIT/ISC - code under these licenses is easier to copy/paste,
>> and is a better investment for your time as a coder studying the stuff
>> and filling up the space of your brain.
>
> Apache 2.0 is a good license, it is similar to MIT/ISC except it has a
> patent clause.

Now everybody is also afraid of patents. Are there any other *short* licenses
with patent grant? Why patents started to work by default? I thought you
should explicitly mention if you work is patented or pending for them to work.

Regarding licenses, I'd still prefer to learn codebase that I can freely
copy/paste to my own projects without additional obligations like some that
imposed by Apache 2.0. I can understand crediting, but everything else just
keeps people away.

If not sure if CC-BY applies to software projects, but I'd choose something
as simple as this not only for code, but also for pydotorg content.

>> 3. Why the raw SQL? It cripples the ability to scale, to host package
>> indexes on GAE or OpenStack. And it is right at the core. Why not to
>> "port NDB" to SQL or use HyperDB from Roundup?
>
> It was just recently switched to raw sql from using the SQL Alchemy
> SQL expression layer. The reason for the switch was we were not
> using the portability features and it was more complicated to understand
> the expression layer than just plain SQL.

Ok. SQL Alchemy is too big and complicated. Why NoSQL didn't fit?
MongoDB is all over the net nowadays.

>> 4. Why not Django/TG/... framework? (+pinax, +openid, +other_stuff)
>
> I’ll be adding a section to the documentation on this eventually, but
> basically.
>
> Django’s ORM wasn’t powerful enough to represent the existing Schema. When
> I had to throw out the ORM I found it difficult to integrate other pieces without
> instating more global state. Ultimately I felt that a Fraken-Django that used Django
> on the surface but with pieces replaced was going to require as much or more
> of  a learning curve, even for Django developers, that it didn’t make sense to
> do it.

Maybe a good partitioning of application with big diagram could bring the best
from both world? Bare SQL and getting to the basics seems to radical to me.

> At that point I was a bit frustrated and fed up and turned to using just Werkzeug
> as a library. This enabled a rapid increase in pace (I got more down over the
> initial weekend then I had in a month previously).

Isn't it just a WSGI library? What about reusable components that are already
written? I wouldn't risk rewriting OAuth support from scratch.

> Feedback is good, but it’ll be tempered to prevent http://theoatmeal.com/comics/design_hell

That's a nice argument. =)

>>>> You also need a backlog for
>>>> collaboration. My ETA for new PyPI is no earlier than PyCon 2014 if
>>>> Donald and Richard will be working on it full time.
>>>
>>> Possibly! I’m unsure of how long it will take, it’s primarily Richard and I but we’ve a
>>> few domain experts in particular pieces who have offered to help out as well when
>>> we’re ready for their pieces.
>>
>> PostgreSQL is killing all the fun. It takes a few hours to get
>> acquainted with its way of doing things - groups as users, per table
>> grants and insufficient DB permissions. There should be an option to
>> run on SQlite for development, like in Trac, Roundup etc.
>
> Warehouse only runs on PostgreSQL. It’s not that hard to have a local copy of
> PostgreSQL and It’ll enable using the advanced feature set of PostgreSQL
> instead of being limited to the lowest common denominator.

What feature sets you need? Does they really worth killing the fun of working,
hack-a-toning and for local development?

>>>> So, instead of
>>>> all-or-nothing scenario I can try to find some help with incremental
>>>> approach.
>>>
>>> Mostly the problem with improving the current base is every change is particularly
>>> dangerous. The code base is large, it’s untested, and it’s not very nice. It’s extremely
>>> easy to break things by accident with seemingly unrelated change. Richard and I
>>> have both done this multiple times.
>>
>> This smells like a bad spaghetti. Do you think it will be maintainable
>> if it is already so fragile on the early stages of development?
>
> This is talking about the current code base, not about Warehouse. Warehouse has
> and requires 100% unit test coverage and will be gaining a functional test suite
> on top of that to test high level user stories.

At least somebody who is not getting mad using "user story" buzzwords.
A pleasure. =)
Sorry I missed the point that you broke old PyPI code, now the new
one. Still, 100% of
test coverage makes it hard to make changes, sometimes more hard than necessary.
Perhaps I should take a look at this.

BTW, why didn't you just copy crate.io?

> Warehouse
> is (hopefully) easy to iterate on, has what I believe is a well laid out
> code base, and has an extensive unit test suite.

Ok. I should take a closer look.

>> Do you have a diagram of the system what are you trying to build, or
>> is it just an experiment? It may worth to put IDE aside and try to
>> de-couple some things first on the whiteboard. I am afraid that I will
>> never be ready to help with reinventing Django. At least not without
>> some research and art coverage.
>
> It’s not a reinvention of Django. The “framework” parts of Warehouse
> are very small and are mostly glue that holds together pieces like
> Werkzeug and SQLAlchemy.

Still, is there a diagram of Warehouse architecture? Blueprint,
CAD model of a building?