On Tue, May 29, 2012 at 2:02 AM, Nick Coghlan wrote:

If you don't like our ecosystem (it's hard to tell whether or not you do: everything you post is about how utterly awful and unusable everything is, yet you're still here years later).

I understand the discouragement with regard to repeating yourself over and over again. But, let's keep the discussion friendly here, okay? This is Python-ideas: crazy proposals are fine. We can simply ignore those and move on.

On Tue, May 29, 2012 at 9:02 AM, Nick Coghlan wrote:

Once again, you're completely ignoring all existing knowledge and expertise on open collaboration and trying to reinvent the world. It's *not going to happen*.

It's too boring to live in a world of existing knowledge and expertise, and yes, I am not aware of any open collaboration stuff expertise. Any reading recommendations with concentrated knowledge that can fit my brain?

The standard library is just the curated core, and *yes*, it's damn hard to get anything added to it (deliberately so). There's a place where anyone can post anything they want, and see if others find it useful: PyPI.

The major drawbacks of remote packages in general is that it bring back project compilation from the old days. The biggest Python advantage at all times was "copy and run" ability. The drawbacks of PyPI for this proposal are: 1. every function you need will require a separate upload to PyPI 2. you can't upload function with the same stdlib name, but slightly different implementation as it is used in different projects 3. you can't find functions that people recommend to be included into stdlib 4. it is hard (impossible) to gather feedback on the quality of these proposals

The standard library provides tools to upload to PyPI, and, as of 3.3, will even include tools to download and install from it.

I am glad 3.3 is giving virtualenv and bootstrap stuff. It would really rock, if the new feature won't be settled in stone right after release and will gain a few UX iterations with allowed break-ability. As for PyPI, the major drawback of it is security - DNS attack for a couple of minutes, and one of your automatically deployed nodes is trojan ready. I remember PyPI password are stored in clear-text on developer's machine, but I don't remember if anyone turned off HTTP basic authorization on PyPI to protect passwords travelling to PyPI with every upload from intercepting. It would be an interesting exercise to sniff PyPI passwords over WiFi during next conference (i.e. https://ep2012.europython.eu/) and match those to the developer's accounts on *.python.org ;)

If you don't like our ecosystem (it's hard to tell whether or not you do: everything you post is about how utterly awful and unusable everything is, yet you're still here years later).

You're absolutely right - I like the Python ecosystem, otherwise I wouldn't stick there. It is like a vintage car - awesome, nice looking, and there is even this new twisted pyusion engine inside, but.. well - it's not for youngsters.

If you think the PyPI UI is awful or inadequate, follow the example of crate.io or pythonpackage.com and *create your own*. There's far more to the Python universe than just core development, stop trying to shoehorn everything into a place where it doesn't belong.

I have absolutely no idea how aforementioned post touches PyPI UI. Speaking about PyPI enhancements and ecosystem, instead of reinventing bicycles I'd rather patch existing one. The only problem is that patches are not accepted. https://bitbucket.org/loewis/pypi/pull-request/1/fix-imports-add-logging-to-...

On Sat, Jun 2, 2012 at 8:24 PM, Calvin Spealman wrote:

On Fri, Jun 1, 2012 at 11:08 AM, anatoly techtonik wrote:

...

On Tue, May 29, 2012 at 9:02 AM, Nick Coghlan wrote:

...

Once again, you're completely ignoring all existing knowledge and expertise on open collaboration and trying to reinvent the world. It's *not going to happen*.

It's too boring to live in a world of existing knowledge and expertise,

Frankly, this one fragment is enough to stop me reading further. Who wants to learn from the vast and broad experience when you could simply randomize the rules of reality through ignorance and stubbornness?

If everybody would think like this, the world will never learn about anti-patterns, and the software craftmanship collapsed in astonishing agony some years ago. If it doesn't make it clear - it is not randomizing - it is putting beliefs to the test asking for the current status.

I sound fickle, because I am.

It doesn't matter how do you sound, what matters is that you spoiled the fun to discuss the technical part no matter how long ago it was invented. If you have a lot of people who ask the same question - create a FAQ. That's not a vast and broad experience - that's just a time proven practice from usenet times. Common guys, what's wrong with you? It is just an idea, not a proposal or scientific paper. And I am not a scientist - I just want to discuss the idea, and I am not sending mails to python-dev anymore, because you asked to. I've spent some time trying to make the idea interesting. It is fine If you know a scientific paper about the matter, can explain it in a few words and send a link for more details. But the replies like "you're stubborn and ignorant, and nobody should help you" doesn't make you a better person. I am criticizing, because I lack time, motivation and fantasy to write stuff about good and bright sides in my life that I just don't see. I write because I see bad things that can be better, and I am still open to discuss if it is real or not. -- anatoly t.

On Tue, May 29, 2012 at 8:05 AM, anatoly techtonik wrote:

The problem with stdlib - it is all damn subjective. There is no process to add functions and modules if you're not well-behaved and skilled in public debates and don't have really a lot of time to be a champion of your module/function. In other words - it is hard (if not impossible for 80% of Python Earth population). So, many people and projects decide to opt-out. Take a look at Twisted - a lot of useful stuff, but not in Python stdlib. So..

Provide a way for people to opt-out from core stuff, but still allow to share the changes and update code if necessary.

This will require: - a local stdlib Python path convention - snippet normalization function and AST hash dumper - web site with stats - source code crawler

How it works: 1. Every project maintains its own stdlib directory with functions that they feel are good to have in standard library 2. Functions are placed so that they are imported as if from standard library, but this time with stdlib prefix 3. The license for this directory is public domain to remove all legal barriers (credits are welcome, but optional) 4. Crawler (probably PyPI) scans this stdlib dir, finds functions, normalizes them, calculates hash and submits to web site 4.1 Normalization is required to find the shared function copy/pasted across different projects with different indentation level, docstrings, parameters/variable names etc. 4.2 Hash is calculated upon AST. There are at least three hashes for each entry: 4.2.1 Full hash - all docstrings and variable names are preserved, whitespace normalized 4.2.2 Stripped hash - docstrings are stripped, variable names are normalized 4.2.3 Signature hash - a mark placed in a comment above function name, either calculated from function signature or generated randomly, used for manual tracking of copy/paste e.g. pd:ac546df6b8340a92 5. Web site maintains usage and popularity staff, accepts votes on inclusion of snippets

User stories: 1. "I want to find if there is a better/updated version of my function available" 1.1 I enter hash into web site search form 1.2 Site gives me a link to my snippet 1.3 I can see what people proposed to replace this function with 1.4 I can choose the function with most votes 1.5 I can flag the functions I may find irrelevant or 1.5 I can tag the functions that divert in different direction than I need to filter them

2. "I want to reuse code snippets without additional dependencies on 3rd party projects" 1.1 Just place them into my own stdlib directory

3. "I want to update code snippets when there is an update for them" 1.1 I run scanner, it extracts signature hashes, stripped hashes and looks if web-site version of signature matches normalized hash

4. "I want to see what people want to include in the next Python version" 1.1 A call for proposals is made 1.2 People place wannabe's into their stdlib dirs 1.3 Crawl generates new functions on a web site 1.4 Functions are categorized 1.5 Optionally included / declined with a short one-liner reason - why 1.6 Optionally provided with more detailed info why

--- feature creep cut --- 5. "I want to see what functions are popular in other languages" 1.1 A separate crawler for Ruby, PHP etc. stdlib converts their AST into compatible format where possible 1.2 Submit to site stats

6. "I want to download the function in Ruby format" 1.1 AST converter tries to do the job automatically where possible 1.2 If it fails - you are encouraged to fix the converter rules or write the replacement for this signature manually

Just an idea. -- anatoly t.

I think having a separate site "anatloy's std-lib" which somehow implemented an easy install of the top 10-100 most useful/popular/selected packages on pypi could be nice. I considered making such a bundle myself a while ago. I don't think it really needs to be python.org sanctioned. Yuval PS I like how candid the replies you got were, and indeed getting a reply is better than the sound of crickets. Though some of these replies carried the scent of excrement poredom - the author's need to import niceness.

I write because I see bad things that can be better, and I am still open to discuss if it is real or not.

There's nothing wrong with that in its place. But python-ideas is a place for ideas where the poster is pretty sure it's real *and* has a concrete proposal (the "idea" in python-ideas) to make it better *and* has the will to follow up themselves if nobody else grabs the ball. There's some room for blue-sky ideas (lacking concrete proposals or personal commitment), but if all you ever offer is blue-sky ideas that get no uptake, you're just wasting time, yours as well as everybody else's.