Mailman 3 March 2013 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists … [View More]now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich [View Less]

10 15

Specify number of items to allocate for array.array() constructor
by Sven Rahmann Feb. 22, 2020

Feb. 22, 2020

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int … [View More]

14 20

while conditional in list comprehension ??
by Wolfgang Maier Feb. 21, 2014

Feb. 21, 2014

Dear all, I guess this is so obvious that someone must have suggested it before: in list comprehensions you can currently exclude items based on the if conditional, e.g.: [n for n in range(1,1000) if n % 4 == 0] Why not extend this filtering by allowing a while statement in addition to if, as in: [n for n in range(1,1000) while n < 400] Trivial effect, I agree, in this example since you could achieve the same by using range(1,400), but I hope you get the point. This intuitively … [View More]

19 70

format specifier for "not bytes"
by Daniel Holth July 5, 2013

July 5, 2013

While I was implementing JSON-JWS (JSON web signatures), a format which in Python 3 has to go from bytes > unicode > bytes > unicode several times in its construction, I notice I wrote a lot of bugs: "sha256=b'abcdef1234'" When I meant to say: "sha256=abcdef1234" Everything worked perfectly on Python 3 because the verifying code also generated the sha256=b'abcdef1234' as a comparison. I would have never noticed at all unless I had tried to verify the Python 3 output with Python 2. … [View More]

8 17

frozenset literals
by Hua Lu May 21, 2013

May 21, 2013

This is similar to another proposal: http://mail.python.org/pipermail/python-3000/2008-January/011798.html Anyway, I was using ast.literal_eval and attempted to use frozenset({...}) as a key in a dictionary, which failed, because frozenset isn't a literal (though putting frozenset in the environment would be a security risk). I am currently working around this with tuples, but I'd like a literal for representing frozensets as well. I also use frozensets elsewhere in the code in ways similar to … [View More]

15 31

hexdump
by anatoly techtonik April 29, 2013

April 29, 2013

Just an idea of usability fix for Python 3. hexdump module (function or bytes method is better) as simple, easy and intuitive way for dumping binary data when writing programs in Python. hexdump(bytes) - produce human readable dump of binary data, byte-by-byte representation, separated by space, 16-byte rows Rationale: 1. Debug. Generic binary data can't be output to console. A separate helper is needed to print, log or store its value in human readable format in database. This takes … [View More]time. 2. Usability. binascii is ugly: name is not intuitive any more, there are a lot of functions, and it is not clear how it relates to unicode. 3. Serialization. It is convenient to have format that can be displayed in a text editor. Simple tools encourage people to use them. Practical example: >>> print(b) � � � �� >>> b '\xe6\xb0\x08\x04\xe7\x9e\x08\x04\xe7\xbc\x08\x04\xe7\xd5\x08\x04\xe7\xe4\x08\x04\xe6\xb0\x08\x04\xe7\xf0\x08\x04\xe7\xff\x08\x04\xe8\x0b\x08\x04\xe8\x1a\x08\x04\xe6\xb0\x08\x04\xe6\xb0\x08\x04' >>> print(binascii.hexlify(data)) e6b00804e79e0804e7bc0804e7d50804e7e40804e6b00804e7f00804e7ff0804e80b0804e81a0804e6b00804e6b00804 >>> >>> data = hexdump(b) >>> print(data) E6 B0 08 04 E7 9E 08 04 E7 BC 08 04 E7 D5 08 04 E7 E4 08 04 E6 B0 08 04 E7 F0 08 04 E7 FF 08 04 E8 0B 08 04 E8 1A 08 04 E6 B0 08 04 E6 B0 08 04 >>> >>> # achieving the same output with binascii is overcomplicated >>> data_lines = [binascii.hexlify(b)[i:min(i+32, len(binascii.hexlify(b)))] for i in xrange(0, len(binascii.hexlify(b)), 32)] >>> data_lines = [' '.join(l[i:min(i+2, len(l))] for i in xrange(0, len(l), 2)).upper() for l in data_lines] >>> print('\n'.join(data_lines)) E6 B0 08 04 E7 9E 08 04 E7 BC 08 04 E7 D5 08 04 E7 E4 08 04 E6 B0 08 04 E7 F0 08 04 E7 FF 08 04 E8 0B 08 04 E8 1A 08 04 E6 B0 08 04 E6 B0 08 04 On the other side, getting rather useless binascii output from hexdump() is quite trivial: >>> data.replace(' ','').replace('\n','').lower() 'e6b00804e79e0804e7bc0804e7d50804e7e40804e6b00804e7f00804e7ff0804e80b0804e81a0804e6b00804e6b00804' But more practical, for example, would be counting offset from hexdump: >>> print( ''.join( '%05x: %s\n' % (i*16,l) for i,l in enumerate(hexdump(b).split('\n')))) Etc. Conclusion: By providing better building blocks on basic level Python will become a better tool for more useful tasks. References: [1] http://stackoverflow.com/questions/2340319/python-3-1-1-string-to-hex [2] http://en.wikipedia.org/wiki/Hex_dump -- anatoly t. [View Less]

8 10

Poll about -h,--help options
by anatoly techtonik April 14, 2013

April 14, 2013

Hi, Is it interesting to know if people expect -h to work as a --help equivalent by default? As it directly affects best practices of using Python in command line, can we run such poll on python.org? I thought that this stuff is obvious, but it appears that it is not, so I'd like to see this usability study - it will be interesting for the Python community to know itself better. -- anatoly t.

7 7

list…pushed, or something
by Shane Green March 30, 2013

March 30, 2013

I'm not sure if there's anything inherently wrong with this idea, and I am well aware how incredibly easy it is to implement as an extension of the built-ins, but, I find it very useful to have variations of list().append(obj) and set().add(obj) that return, obj. I've never come up with perfect method names; sometimes I go with past-tense versions, "added" and "appended" (or "pushed"). I often use these methods in places where performance is a factor, such as using a set() to filter repeats … [View More]

8 15

type.__signature__ (and Argument Clinic)
by Eric Snow March 29, 2013

March 29, 2013

We now have a useful function signature abstraction in 3.3 with PEP 362. An outgrowth of this has been Larry Hasting's efforts on Argument Clinic. One open question is (was?) how to give a docstring to the C versions of __new__ and __init__. Though in the morning I may wonder what I was thinking, here's a possibility that popped into my head: let classes have a __signature__ attribute that is the signature for meta.__call__, as well as cls.__new__ and cls.__init__. This makes sense since … [View More]

2 2

re.compile_lazy - on first use compiled regexes
by Ronny Pfannschmidt March 28, 2013

March 28, 2013

Hi, while reviewing urllib.parse i noticed a pretty ugly pattern many functions had an attached global and in their own code they would compile an regex on first use and assign it to that global its clear that compiling a regex is expensive, so having them be compiled later at first use would be of some benefit but instead of all that reptetive code there should be an alternative to re.compile that waits with compilation for the first use -- Ronny

17 45

argparse - add support for environment variables
by Miki Tebeka March 27, 2013

March 27, 2013

Greetings, The usual way of resolving configuration is command line -> environment -> default. Currently argparse supports only command line -> default, I'd like to suggest an optional "env" keyword to add_argument that will also resolve from environment. (And also optional env dictionary to the ArgumentParser __init__ method [or to parse_args], which will default to os.environ). Example: [spam.py] parser = ArgumentParser() parser.add_argument('--spam', env='SPAM',… [View More]

3 4

Google Summer of Code - Organization Deadline Approaching - March 29
by Brian Curtin March 26, 2013

March 26, 2013

Just an FYI that there are under 3 days to apply to Google Summer of Code for mentoring organizations: http://www.google-melange.com/gsoc/homepage/google/gsoc2013. The student application deadline is later on in May. If you run a project that is interested in applying under the Python umbrella organization, contact Terri Oda at terri(a)zone12.com

1 0

Asking for feedback about issue 17552 (socket.sendfile())
by Giampaolo Rodolà March 26, 2013

March 26, 2013

http://bugs.python.org/issue17552 I hope it's alright to bother python-ideas asking for feedback but considering the design decisions involved I thought it would have been legitimate. Please feel free to comment on the issue rather than here. Thanks in advance, - Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/

1 0

rebooting the lazy regex discussion as something stricty more narrow for lazy expensively computed globals
by Ronny Pfannschmidt March 25, 2013

March 25, 2013

Hi, since the previous discussions raised lots of bikeshedding in maintenance-pain directions about pick/marshal in sourcecode, i'd like to reboot the discussion in a more narrow scope the pattern is that the stdlib has lazy-computed private globals in various modules i would like to propose an alternative way of doing those instead of code such as _hostprog = None def splithost(url): """splithost('//host[:port]/path') --> 'host[:port]', '/path'.""" global _hostprog … [View More]

1 0

Message passing syntax for objects
by Mark Janssen March 22, 2013

March 22, 2013

Hello, I just posted an answers on quora.com about OOP (http://qr.ae/TM1Vb) and wanted to engage the python community on the subject. Alan Kay's idea of message-passing in Smalltalk are interesting, and like the questioner says, never took off. My answer was that Alan Kay's abstraction of "Everything is an object" fails because you can't have message-passing, an I/O task, working in the same space as your objects -- they are two very different functionalities and they have to be preserved **… [View More]

19 43

Re: [Python-ideas] Message passing syntax for objects
by Mark Janssen March 20, 2013

March 20, 2013

On Tue, Mar 19, 2013 at 1:09 PM, Terry Reedy <tjreedy(a)udel.edu> wrote: > On 3/18/2013 11:31 PM, Andrew Barnert wrote: > >> The idea that message passing is fundamentally different from method >> calling also turned out to be one of those strange ideas, since it >> only took a couple years to prove that they are theoretically >> completely isomorphic—and, > > Since the isomorphism is so obvious, I somehow missed that Kay actually > thought that they … [View More]

1 0

Re: [Python-ideas] Message passing syntax for objects
by Mark Janssen March 19, 2013

March 19, 2013

On Mon, Mar 18, 2013 at 2:51 PM, Andrew Barnert <abarnert(a)yahoo.com> wrote: > Have you even looked at a message-passing language? > > A Smalltalk "message" is a selector and a sequence of arguments. That's what you send around. Newer dynamic-typed message-passing OO and actor languages are basically the same as Smalltalk. Yes, but you have to understand that Alan Kays came with strange ideas of some future computer-human symbiosis. So his language design and other similar … [View More]

2 1

Separate group for discussing PEP 3156 and Tulip
by Guido van Rossum March 18, 2013

March 18, 2013

I've created a Google Group for PEP 3156 and Tulip. If you're interested in this topic at all, please join the group. The group is here: https://groups.google.com/forum/?fromgroups#!forum/python-tulip Email is python-tulip(a)googlegroups.com; only members can post (open Google Groups attract too much spam). -- --Guido van Rossum (python.org/~guido)

3 2

Thoughts on the future of concurrency in Python: PEP 3148
by Steven Hazel March 18, 2013

March 18, 2013

Hi, I'm one of the authors of monocle (https://github.com/saucelabs/monocle), an async Python programming framework with a blocking look-alike syntax based on generators. We've been using monocle at my startup Sauce Labs, and I used its predecessor on a small team at BitTorrent. I think it's met with some amazing success vs. similar projects using threads. In about five years of working with teams with some members who haven't always understood monocle very well, we've never written a deadlock,… [View More] for example. I've been keeping up with PEPs and skimming python-ideas discussions about concurrency-related features in Python, and feeling unsure about how best to jump into the conversation. Today it occurred to me that sooner is better and I should just get all my various opinions out there in case they're helpful. First of all, I want to reiterate: while I have mainly criticism to offer, it is only intended to be helpful. I don't have a lot of time to devote to this myself, so I'm trying to do what I can to be useful to those who do. I think I'll take this one PEP at a time, and start with the easiest one. PEP 3148 - Futures First, I like the idea of this PEP. I think it'll improve interoperability between async frameworks in Python. I believe the name "future" is a poor choice. After many years of explaining this concept to programmers, I've found that its various abstract-conceptual names do a lot to confuse people. Twisted's "deferred" is probably the worst offender here, but both "future" and "promise" also make this very simple construct sound complicated, and it's a crime. There are two ways I've found to explain this that seem to keep people from getting confused: The first is, call it a "callback registry". Many programmers are familiar with the idea of passing callbacks into a function. You just explain that here, instead of taking the callbacks as parameters, we return a callback registry object and you can add your callbacks there. (If the operation is already done they'll just get called when you add them.) The second is a variant on that: say that this function, instead of taking a callback as a parameter, returns its callback. That sounds useless, but fortunately the callback it returns is a callable object, and it also has an "add" method that lets you add functions that it will call when called back. So it's a callback that you can manipulate to do what you want. (This is what we do in monocle: https://github.com/saucelabs/monocle/blob/master/monocle/callback.py) These approaches, by focusing on the simple and familiar idea of callbacks, do a lot to eliminate confusing ideas about communicating with the future of the process. Finally, I'm not sure I get the point of the Executor class in PEP 3148. It seems to me that any real implementation of that class needs a call_later method for scheduling operations, and a way of connecting up to event-based IO APIs. I don't really understand what Executor would be good for. Thanks for reading. -- Steven Hazel Sauce Labs Cofounder and VP Product http://saucelabs.com/ work: sah(a)saucelabs.com other: sah(a)awesame.org [View Less]

5 8

Message passing syntax for objects
by Mark Janssen March 18, 2013

March 18, 2013

> Ian Cordasco wrote: >> >> On Sun, Mar 17, 2013 at 11:53 PM, Mark Janssen >> <dreamingforward(a)gmail.com> wrote: >> >>> Hello, >>> >>> I just posted an answers on quora.com about OOP (http://qr.ae/TM1Vb) >>> and wanted to engage the python community on the subject. > > > My answer to that question would be that it *did* > catch on, it's just that we changed the terminology. > Instead of message passing, we … [View More]

2 2

Message passing for objects
by Mark Janssen March 18, 2013

March 18, 2013

Just the thing I'm talking about. So I'm not the only one trying out these ideas..... :) (via the p2p-foundation mailing list) ---------- Forwarded message ---------- From: ProjectParadigm-ICT-Program <metadataportals(a)yahoo.com> Date: Sat, Mar 16, 2013 at 3:12 PM Subject: Re: A Distributed Economy -- A blog involving Linked Data To: Brent Shambaugh <brent.shambaugh(a)gmail.com>, Michel Bauwens <michel(a)p2pfoundation.net> Cc: Samuel Rose <samuel.rose(a)gmail.com>, "… [View More]

2 1

new PEP submission: transitioning to hosting release files on PYPI
by holger krekel March 15, 2013

March 15, 2013

Hi all, FYI i just submitted the below PEP draft to the PEP editors to receive a proper version number. The topics and the draft saw extensive discussions (four versions) on catalog-sig(a)python.org. We'd like to keep the discussion there if possible. best, holger krekel PEP: XXX Title: Transitioning to release-file hosting on PyPI Version: $Revision$ Last-Modified: $Date$ Author: Holger Krekel <holger(a)merlinux.eu>, Carl Meyer <carl(a)oddbird.net> Discussions-To: catalog-… [View More]sig(a)python.org Status: Draft (PRE-submit V4) Type: Process Content-Type: text/x-rst Created: 10-Mar-2013 Post-History: Abstract ======== This PEP proposes a backward-compatible two-phase transition process to speed up, simplify and robustify installing from the pypi.python.org (PyPI) package index. To ease the transition and minimize client-side friction, **no changes to distutils or existing installation tools are required in order to benefit from the first transition phase, which will result in faster, more reliable installs for most existing packages**. The first transition phase implements an easy and explicit means for a package maintainer to control which release file links are served to present-day installation tools. The first phase also includes the implementation of analysis tools for present-day packages, to support communication with package maintainers and the automated setting of default modes for controlling release file links. The first phase also will default newly-registered projects on PyPI to only serve links to release files which were uploaded to PyPI. The second transition phase concerns end-user installation tools, which shall default to only install release files that are hosted on PyPI and tell the user if external release files exist, offering a choice to automatically use those external files. Rationale ========= .. _history: History and motivations for external hosting -------------------------------------------- When PyPI went online, it offered release registration but had no facility to host release files itself. When hosting was added, no automated downloading tool existed yet. When Philip Eby implemented automated downloading (through setuptools), he made the choice to allow people to use download hosts of their choice. The finding of externally-hosted packages was implemented as follows: #. The PyPI ``simple/`` index for a package contains all links found by scraping them from that package's long_description metadata for any release. Links in the "Download-URL" and "Home-page" metadata fields are given ``rel=download`` and ``rel=homepage`` attributes, respectively. #. Any of these links whose target is a file whose name appears to be in the form of an installable source or binary distribution, with name in the form "packagename-version.ARCHIVEEXT", is considered a potential installation candidate by installation tools. #. Similarly, any links suffixed with an "#egg=packagename-version" fragment are considered an installation candidate. #. Additionally, the ``rel=homepage`` and ``rel=download`` links are crawled by installation tools and, if HTML, are themselves scraped for release-file links in the above formats. Today, most packages released on PyPI host their release files on PyPI, but a small percentage (XXX need updated data) rely on external hosting. There are many reasons [2]_ why people have chosen external hosting. To cite just a few: - release processes and scripts have been developed already and upload to external sites - it takes too long to upload large files from some places in the world - export restrictions e.g. for crypto-related software - company policies which require offering open source packages through own sites - problems with integrating uploading to PyPI into one's release process (because of release policies) - desiring download statistics different from those maintained by PyPI - perceived bad reliability of PyPI - not aware that PyPI offers file-hosting Irrespective of the present-day validity of these reasons, there clearly is a history why people choose to host files externally and it even was for some time the only way you could do things. This PEP takes the position that there are at least some valid reasons for external hosting. Problem ------- **Today, python package installers (pip, easy_install, buildout, and others) often need to query many non-PyPI URLs even if there are no externally hosted files**. Apart from querying pypi.python.org's simple index pages, also all homepages and download pages ever specified with any release of a package are crawled by an installer. The need for installers to crawl external sites slows down installation and makes for a brittle and unreliable installation process. Those sites and packages also don't take part in the :pep:`381` mirroring infrastructure, further decreasing reliability and speed of automated installation processes around the world. Most packages are hosted directly on pypi.python.org [1]_. Even for these packages, installers still crawl their homepage and download-url, if specified. Many package uploaders are not aware that specifying the "homepage" or "download-url" in their package metadata will needlessly slow down the installation process for all users. Relying on third party sites also opens up more attack vectors for injecting malicious packages into sites using automated installs. A simple attack might just involve getting hold of an old now-unused homepage domain and placing malicious packages there. Moreover, performing a Man-in-The-Middle (MITM) attack between an installation site and any of the download sites can inject malicious packages on the installation site. As many homepages and download locations are using HTTP and not HTTPS, such attacks are not hard to launch. Such MITM attacks can easily happen even for packages which never intended to host files externally as their homepages are contacted by installers anyway. There is currently no way for package maintainers to avoid external-link crawling, other than removing all homepage/download url metadata for all historic releases. While a script [3]_ has been written to perform this action, it is not a good general solution because it removes useful metadata from PyPI releases. Even if the sites referenced by "Homepage" and "Download-URL" links were not scraped for further links, there is no obvious way under the current system for a package owner to link to an installable file from a long_description metadata field (which is shown as package documentation on ``/pypi/PKG``) without installation tools automatically considering that file a candidate for installation. Conversely, there is no way to explicitly register multiple external release files without putting them in metadata fields. Goals ----- These are the goals to be achieved by implementation of this PEP: * Package owners should be able to explicitly control which files are presented by PyPI to installer tools as installation candidates. Installation should not be slowed and made less reliable by extensive and unnecessary crawling of links that package owners did not explicitly nominate as installation files. * It should remain possible for package owners to choose to host their release files on their own hosting, external to PyPI. It should be easy for a user to request the installation of such releases using automated installer tools. * Automated installer tools should not install externally-hosted packages **by default**, but only when explicitly authorized to do so by the user. When tools refuse to install such a package by default, they should tell the user exactly which external link(s) they would need to follow, and what option(s) the user can provide to authorize the tool to follow those links. PyPI should provide all necessary metadata for installer tools to implement this easily and within a single request/reply interaction. * Migration from the status quo to the above points should be gradual and minimize breakage. This includes tooling that makes it easy for package owners with an existing release process that uploads to non-PyPI hosting to also upload those release files to PyPI. Solution / two transition phases ================================ The first transition phase introduces a "hosting-mode" field for each project on PyPI, allowing package owners explicit control of which release file links are served to present-day installation tools in the machine-readable ``simple/`` index. The first transition will, after successful hosting-mode manipulations by individual early-adopters, set a default hosting mode for existing packages, based on automated analysis. **Maintainers will be notified one month ahead of any such automated change**. At completion of the first transition phase, **all present-day existing release and installation processes and tools are expected to continue working**. Any remaining errors or problems are expected to only relate to installation of individual packages and can be easily corrected by package maintainers or PyPI admins if maintainers are not reachable. Also in the first phase, each link served in the ``simple/`` index will be explicitly marked as ``rel="internal"`` (hosted by the index itself) or ``rel="external"`` (linking to an external site that is not part of the index). In the second transition phase, PyPI client installation tools shall be updated to default to only install ``rel="internal"`` packages unless a user specifies option(s) to permit installing from external links. Maintainers of packages which currently host release files on non-PyPI sites shall receive instructions and tools to ease "re-hosting" of their historic and future package release files. This re-hosting tool MUST be available before automated hosting-mode changes are announced to package maintainers. Implementation ============== Hosting modes ------------- The foundation of the first transition phase is the introduction of three "modes" of PyPI hosting for a package, affecting which links are generated for the ``simple/`` index. These modes are implemented without requiring changes to installation tools via changes to the algorithm for generating the machine-readable ``simple/`` index. The modes are: - ``pypi-scrape-crawl``: no change from the current situation of generating machine-readable links for installation tools, as outlined in the history_. - ``pypi-scrape``: for a package in this mode, links to be added to the ``simple/`` index are still scraped from package metadata. However, the "Home-page" and "Download-url" links are given ``rel=ext-homepage`` and ``rel=ext-download`` attributes instead of ``rel=homepage`` and ``rel=download``. The effect of this (with no change in installation tools necessary) is that these links will not be followed and scraped for further candidate links by present-day installation tools: only installable files directly hosted from PyPI or linked directly from PyPI metadata will be considered for installation. Installation tools MAY evolve to offer an option to use the new rel-attribution to crawl external pages but MUST NOT default to it. - ``pypi-explicit``: for a package in this mode, only links to release files uploaded to PyPI, and external links to release files explicitly nominated by the package owner (via a new interface exposed by PyPI) will be added to the ``simple/`` index. Thus the hope is that eventually all projects on PyPI can be migrated to the ``pypi-explicit`` mode, while preserving the ability to install release files hosted externally via installer tools. Deprecation of hosting modes to eventually only allow the ``pypi-explicit`` mode is NOT REGULATED by this PEP but is expected to become feasible some time after successful implementation of the transition phases described in this PEP. It is expected that deprecation requires **a new process to deal with abandoned packages** because of unreachable maintainers for still popular packages. First transition phase (PyPI) ----------------------------- The proposed solution consists of multiple implementation and communication steps: #. Implement in PyPI the three modes described above, with an interface for package owners to select the mode for each package and register explicit external file URLs. #. For packages in all modes, label all links in the ``simple/`` index with ``rel="internal"`` or ``rel="external"``, to make it easier for client tools to distinguish the types of links in the second transition phase. #. Default all newly-registered packages to ``pypi-explicit`` mode (package owners can still switch to the other modes as desired). #. Determine (via an automated analysis tool) which packages have all installable files available on PyPI itself (group A), which have all installable files linked directly from PyPI metadata (group B), and which have installable versions available that are linked only from external homepage/download HTML pages (group C). #. Send mail to maintainers of projects in group A that their project will be automatically configured to ``pypi-explicit`` mode in one month, and similarly to maintainers of projects in group B that their project will be automatically configured to ``pypi-scrape`` mode. Inform them that this change is not expected to affect installability of their project at all, but will result in faster and safer installs for their users. Encourage them to set this mode themselves sooner to benefit their users. #. Send mail to maintainers of packages in group C that their package hosting mode is ``pypi-scrape-crawl``, list the URLs which currently are crawled, and suggest that they either re-host their packages directly on PyPI and switch to ``pypi-explicit``, or at least provide direct links to release files in PyPI metadata and switch to ``pypi-scrape``. Provide instructions and tools to help with these transitions. Second transition phase (installer tools) ----------------------------------------- For the second transition phase, maintainers of installation tools are asked to release two updates. The first update shall provide clear warnings if externally-hosted release files (that is, files whose link is ``rel="external"``) are selected for download, for which projects and URLs exactly this happens, and warn that in future versions externally-hosted downloads will be disabled by default. The second update should change the default mode to allow only installation of ``rel="internal"`` package files, and allow installation of externally-hosted packages only when the user supplies an option (ideally an option specifying exactly which external domains are to be trusted as download sources). When download of an externally-hosted package is disallowed, the user should be notified, with instructions for how to make the install succeed and warnings about the implication (that a file will be downloaded from a site that is not part of the package index). Open Questions / tasks ======================= - Should we introduce some form of PyPI API versioning in this PEP? (it might complicate matters and delay the implementation but is often seen as good practise). - do another round of discussions with installation tool authors and see about incorporating their feedback. There is one known issue in particular from Philip J. Eby who considers a host-based pattern matching algorithm preferable to interpreting "rel" attributes. References ========== .. [1] Donald Stufft, ratio of externally hosted versus pypi-hosted, http://mail.python.org/pipermail/catalog-sig/2013-March/005549.html (XXX need to update this data for all easy_install-supported formats) .. [2] Marc-Andre Lemburg, reasons for external hosting, http://mail.python.org/pipermail/catalog-sig/2013-March/005626.html .. [3] Holger Krekel, Script to remove homepage/download metadata for all releases http://mail.python.org/pipermail/catalog-sig/2013-February/005423.html Acknowledgments ================ Philip Eby for precise information and the basic ideas to implement the transition via server-side changes only. Donald Stufft for pushing away from external hosting and offering to implement both a Pull Request for the necessary PyPI changes and the analysis tool to drive the transition phase 1. Marc-Andre Lemburg, Nick Coghlan and catalog-sig in general for thinking through issues regarding getting rid of "external hosting". Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: [View Less]

1 0

Unify addresses passed to bind() and getaddrinfo()
by Giampaolo Rodolà March 14, 2013

March 14, 2013

>From http://docs.python.org/3/library/socket.html#example ...when using bind(): HOST = "" # Symbolic name meaning all available interfaces ...when using getaddrinfo(): HOST = None # Symbolic name meaning all available interfaces I recently got bitten by this difference in that I passed "" (empy string) to getaddrinfo() and since my DNS server was temporarily down getaddrinfo() hung indefinitely (cset where I fixed this: https://code.google.com/p/pyftpdlib/… [View More]

2 2

Python 2.7 for Windows 9x
by André Gillibert March 13, 2013

March 13, 2013

Hello. Officially Python doesn't support Windows 9x and NT 3.51/4.0 anymore. I won't change this design decision. However, I built python 2.6.8 and 2.7.3 to run on these platforms. This may be useful to people with a very old computer. May you link this project on your python.org Web site? This is available at: <http://gillibert.fr/andre/projects/python/> Thanks. -- Sincerely André Gillibert

2 1

PEP XXX - Competitor with PEP 435: Adding an enum type to the Python standard library
by Ethan Furman March 13, 2013

March 13, 2013

First, I offer my apologies to all who are still battle-weary from the last flurry of enum threads. However, while flufl.enum (PEP 435) is a good package for the use-case it handles, there are plenty of use-cases that it does not handle, or doesn't handle well, that it should not be the implementation in the stdlib. To quote one of my earlier emails: > I'm beginning to see why enums as a class has not yet been added to Python. > We don't want to complicate the language with too many … [View More]

10 29