On Wed, Sep 9, 2015 at 10:48 PM, Andrew Barnert <abarnert(a)yahoo.com> wrote:
> On Sep 9, 2015, at 21:34, Jukka Lehtosalo <jlehtosalo(a)gmail.com> wrote:
>
> I'm not sure if I fully understand what you mean by implicit vs. explicit
> ABCs (and the static/runtime distinction). Could you define these terms and
> maybe give some examples of each?
>
>
> I just gave examples just one paragraph above.
>
> A (runtime) implicit ABC is something that uses a __subclasshook__
> (usually implementing a structural check). So, for instance, any type that
> implements __iter__ is-a Iterable, e.g., according to isinstance or
> issubclass or @singledispatch, because that's what
> Iterable.__subclasshook__ checks for.
>
> A (runtime) explicit ABC is something that isn't implicit, like Sequence:
> no hook, so nothing is-a Sequence unless it either inherits the ABC or
> registers with it.
>
> You're proposing a parallel but separate distinction at static typing
> time. Any ABC that's a Protocol is checked based on a structural check;
> otherwise, it's checked based on inheritance.
>
In my proposal I actually suggest that protocols shouldn't support
isinstance or issubclass (these operations should raise an exception) by
default. A protocol is free to override the default exception-raising
__subclasshook__ to implement a structural check, and a static type checker
would allow isinstance and issubclass for protocols that do this. I'll need
to explain this idea in more detail, as clearly the current explanation is
too easy to misundertand.
Here's a concrete example:
class X(Protocol):
def f(self): ...
class A:
def f(self): print('f')
if isinstance(A(), X): ... # Raise an exception, because no
__subclasshook__ override in X
Previously I toyed with the idea of having a default implementation of
__subclasshook__ that actually does a structural check, but I'm no longer
sure if that would be desirable, as it's difficult to come up with an
implementation that does the right thing in all reasonable cases. For
example, consider a structural type like this that people might want to use
to work around the current limitations of Callable (it doesn't support
keyword arguments, for example):
class MyCallable(Protocol):
def __call__(self, x, y): ...
(This example has some other potential issues that I'm hand-waving away for
now.)
Now how would the default isinstance work? Preferably it should only accept
callables that are compatible with the signature, but doing that check is
pretty difficult for arbitrary functions and should probably be out of
scope for the typing module. Just checking whether __call__ exists would be
too general, as the programmer probably expects that he's able to call the
method with the specific arguments the type suggests. Also, sometimes
checking the argument names would be a good thing to do, but sometimes any
names (as long the the number of arguments is compatible) would be fine.
> This means it's now possible to create supertypes that are implicit at
> runtime but explicit at static typing time (which might occasionally be
> useful), or vice-versa (which I can't imagine why you'd ever want).
>
As I showed above, you wouldn't get the latter unless you really try very
hard (consenting adults and all).
>
> Besides the obvious negatives in having two not-quite-compatible and
> very-different-looking ways of expressing the same concept, this is going
> to lead to people wanting to know why their type checker is complaining
> about perfectly good code ("I tested that constant with isinstance, and it
> really is-a Spammable, and the type checker is inferring its type properly,
> and yet I get an error passing it to a function that wants a Spammable") or
> allowing blatantly invalid code ("I annotated my function to only take
> Spammable arguments, but someone is passing something that calls the
> fallback implementation of my singledispatch function instead of the
> Spammable overload").
>
I agree that having the default nominal/explicit isinstance semantics for a
protocol type would be a very bad idea.
>
> Maybe the solution is to expand your proposal a little: make Protocol
> automatically create a __subclasshook__ (which you listed as an optional
> idea in the proposal), and also change all of the existing stdlib implicit
> ABCs to Protocols and scrap their manual hooks, and also update the
> relevant documentation (e.g., the abc module and the data model section on
> __subclasshook__) to recommend using Protocol instead of implementing a
> manual hook if the only thing you want is structural subtyping. Of course
> the backward compatibility isn't perfect (unless you want to manually munge
> up collections.abc when typing is imported), and people using legacy
> third-party code might need to add stubs (although that seems necessary
> anyway). But for most people, everything should just work as people expect.
> A type is either structurally typed or explicitly (via inheritance or
> registration) types, both at static typing time and a runtime, and that's
> always expressed by the name Protocol. (But for the rare cases when you
> really need a type check that's looser at runtime, you can still write a
> manual hook to handle that.)
>
>
Yeah, this would be nice, but as I argued above, implementing a generic
__subclasshook__ is actually quite tricky.
Jukka
Dear all
I found a BUG in the standard while statement, which appears both in python
2.7 and python 3.4 on my system.
It usually won't appear because I only stumbled upon it after trying to
implement a nice repeat structure. Look:
```
class repeat(object):
def __init__(self, n):
self.n = n
def __bool__(self):
self.n -= 1
return self.n >= 0
__nonzero__=__bool__
a = repeat(2)
```
the meaning of the above is that bool(a) returns True 2-times, and after
that always False.
Now executing
```
while a:
print('foo')
```
will in fact print 'foo' two times. HOWEVER ;-) ....
```
while repeat(2):
print('foo')
```
will go on and go on, printing 'foo' until I kill it.
Please comment, explain or recommend this further if you also think that
both while statements should behave identically.
hoping for responses,
best,
Stephan
Hi!
This is my first time I'm sending an email to the python-ideas mailing
list. I've got an enhancement idea for the built-in print function and I
hope it is as good as I think it is.
Imagine you have a trial.py file like this:
a = 4
b = "Anand"
print("Hello, I am " + b + ". My favorite number is " + str(a) + ".")
OR
print("Hello, I am ", b, ". My favorite number is ", a, ".")
Well I've got an idea for a function named "print_easy" (The only valid
name I could come up with right now).
So print_easy will be a function which will be used like this (instead of
the current print statement in trial.py) :
print_easy("Hello, I am", b, ". My favorite number is", a ".")
Which gives out:
Hello, I am Anand. My favorite number is 4
The work it does is that it casts the variables and it also formats the
sentences it is provided with. It is exclusively for beginners.
I'm 14 and I came up with this idea after seeing my fellow classmates at
school struggling to do something like this with the standard print
statement.
Sure, you can use the format method but won't that be a bit too much for
beginners? (Also, casting is inevitable in every programmer's career)
Please let me know how this sounds. If it gains some traction, I'll work on
it a bit more and clearly list out the features.
Thanks,
Anand.
--
Anand.
Hi folks,
currently, I came across http://pythonwheels.com/ during researching how
to make a proper Python distribution for PyPI. I thought it would be
great idea to tell other maintainers to upload their content as wheels
so I approached a couple of them. Some of them already provided wheels.
Happy being able to have built my own distribution, I discussed the
issue at hand with some people and I would like to share my findings and
propose some ideas:
1) documentation is weirdly split up/distributed and references old material
2) once up and running (setup.cfg, setup.py etc. etc.) it works but
everybody needs to do it on their own
3) more than one way to do (upload, wheel, source/binary etc.) it (sigh)
4) making contact to propose wheels on github or per email is easy
otherwise almost impossible or very tedious
5) reactions went evenly split from "none", "yes", "when ready" to "nope"
None: well, okay
yes: that's good
when ready: well, okay
nope: what a pity for wheels; example:
https://github.com/simplejson/simplejson/issues/122
I personally find the situation not satisfying. Someone proposes the
following solution in form of a question:
Why do developers need to build their distribution themselves?
I had not real answer to him, but pondering a while over it, I found it
really insightful. Viewing this from a different angle, packaging your
own distribution is actually a waste of time. It is a tedious,
error-prone task involving no creativity whatsoever. Developers on the
other hand are actually people with very little time and a lot of
creativity at hand which should spend better. The logical conclusion
would be that PyPI should build wheels for the developers for every
python/platform combination necessary.
With this post, I would like raise awareness of the people in charge of
the Python infrastructure.
Best,
Sven
[Tim, on parallel PRNGs]
>> There are some clean and easy approaches to this based on
>> crypto-inspired schemes, but giving up crypto strength for speed. If
>> you haven't read it, this paper is delightful:
>>
>> http://www.thesalmons.org/john/random123/papers/random123sc11.pdf
[Nathaniel Smith]
> It really is! As AES acceleration instructions become more common
> (they're now standard IIUC on x86, x86-64, and even recent ARM?), even
> just using AES in CTR mode becomes pretty compelling -- it's fast,
> deterministic, provably equidistributed, *and* cryptographically
> secure enough for many purposes.
Excellent - we're going to have a hard time finding something real to
disagree about :-)
> (Compared to a true state-of-the-art CPRNG the naive version fails due
> to lack of incremental mixing, and the use of a reversible transition
> function. But even these are mostly only important to protect against
> attackers who have access to your memory -- which is not trivial as
> heartbleed shows, but still, it's *waaay* ahead of something like MT
> on basically every important axis.)
Except for wide adoption. Most people I bump into never even heard of
this kind of approach. Nobody ever got fired for buying IBM, and
nobody ever got fired for recommending MT - it's darned near a
checklist item when shopping for a language. I may have to sneak the
code in while you distract Guido with a provocative rant about the
inherent perfidy of the Dutch character ;-)
---------- Forwarded message ----------
From: Theo de Raadt
Date: Wed, Sep 9, 2015 at 10:42 AM
Subject: Re: getentropy, getrandom, arc4random()
To: guido(a)python.org
been speaking to a significant go person.
confirmed.
it takes data out of that buffer, and does not zero it behind itself.
obviously for performance reasons.
same type of thing happens with MT-style engines. in practice, they
can be would backwards. a proper stream cipher cannot be turned
backwards.
however, that's just an academic observation. or maybe it indicates
that well-financed groups can get it wrong too.
by the way, chacha arc4random can create random values faster than a
memcpy -- the computation of fresh output is faster than doing
gross-cost of "read" from memory (when cache dirtying is accounted for).
--
--Guido van Rossum (python.org/~guido)
---------- Forwarded message ----------
From: Theo de Raadt
Date: Wed, Sep 9, 2015 at 10:36 AM
Subject: Re: getentropy, getrandom, arc4random()
To: guido(a)python.org
> Yet another thing. Where do you see that Go and Swift have secure random
as
> a keyword? Searching for "golang random" gives the math/rand package as
the
> first hit, which has a note reminding the reader to use crypto/rand for
> security work.
yes, well, look at the other phrase it uses...
that produces a deterministic sequence of values each time a program is
run
it documents itself as being decidely non-random. that documentation
change happened soon after this event:
https://lwn.net/Articles/625506/
these days, the one people are using is found using "go secure random"
https://golang.org/pkg/crypto/rand/
that opens /dev/urandom or uses the getrandom system call depending on
system. it also has support for the windows entropy API. it pulls
data into a large buffer, a cache. then each subsequent call, it
consumes some, until it rus out, and has to do a fresh read. it
appears to not clean the buffer behind itself, probably for
performance reasons, so the memory is left active. (forward secrecy
violated)
i don't think they are doing the best they can... i think they should
get forward secrecy and higher performance by having an in-process
chacha. but you can sense the trend.
here's an example of the fallout..
https://github.com/golang/go/issues/9205
> For Swift it's much the same -- there's an arc4random() in
> the Darwin package but nothing in the core language.
that is what people are led to use.
--
--Guido van Rossum (python.org/~guido)
I'm just going to forward further missives by Theo.
---------- Forwarded message ----------
From: Theo de Raadt
Date: Wed, Sep 9, 2015 at 9:59 AM
Subject: Re: getentropy, getrandom, arc4random()
To: guido(a)python.org
> Thanks. And one last thing: unless Go and Swift, Python has no significant
> corporate resources behind it -- it's mostly volunteers begging their
> employers to let them work on Python for a few hours per week.
i understand because I find myself in the same situation.
however i think you overstate the difficulty involved.
high-availibility random is kind of a new issue.
so final advice from me; feel free to forward as you like.
i think arc4random would be a better API to call on the back side than
getentropy/getrandom.
arc4random can seed initialize with a single getentropy/getrandom call
at startup. that is done automatically. you can then use
arc4random's results to initialize the MT. in a system call trace,
this will show up as one getentropy/getrandom at process startup,
which gets both subsystems going. really cheap.
in the case of longer run times, the userland arc4random PRNG folding
reduces the system calls required. this helps older kernels with
slower entropy creation, taking pressure off their subsystem. driving
everyone towards this one API which is so high performance is the
right goal.
chacha arc4random is really fast.
if you were to create such an API in python, maybe this is how it will
go:
say it becomes arc4random in the back end. i am unsure what advice to
give you regarding a python API name. in swift, they chose to use the
same prefix "arc4random" (id = arc4random(), id = arc4random_uniform(1..n)";
it is a little bit different than the C API. google has tended to choose
other prefixes. we admit the name is a bit strange, but we can't touch
the previous attempts like drand48....
I do suggest you have the _uniform and _buf versions. Maybe apple
chose to stick to arc4random as a name simply because search engines
tend to give above average advice for this search string?
so arc4random is natively available in freebsd, macos, solaris, and
other systems like andriod libc (bionic). some systems lack it:
win32, glibc, hpux, aix, so we wrote replacements for libressl:
https://github.com/libressl-portable/openbsd/tree/master/src/lib/libcrypto/…https://github.com/libressl-portable/portable/tree/master/crypto/compat
the first is the base openbsd tree where we maintain/develop this code
for other systems, the 2nd part is scaffold in libressl that makes this
available to others.
it contains arc4random for those systems, and supplies getentropy()
stubs for challenged systems.
we'll admit we haven't got solutions for every system known to man.
we are trying to handle fork issues, and systems with very bad
entropy feeding.
that's free code. the heavy lifting is done, and we'll keep maintaining
that until the end of days. i hope it helps.
--
--Guido van Rossum (python.org/~guido)
It would be incredibly convenient, especially for users of AppVayor's
continuous integration service, if there were a(n official) repository
for chocolatey containing recent releases of python. The official
Chocolatey gallery contains installers for the latest 2.7 and 3.4 (as of
this post). What I am proposing would contain the most commonly used
pythons in testing (2.6 2.7 3.3 3.4 and future releases).
I am perfectly willing to set up a repo for my own use, but am posting
this to see if there is community support...or psf support... for
setting up an official repo.
In a message of Sun, 06 Sep 2015 15:31:16 -0400, Terry Reedy writes:
>On 9/6/2015 1:33 PM, Sven R. Kunze wrote:
>>
>> With this post, I would like raise awareness of the people in charge of
>> the Python infrastructure.
>
>pypa is in charge of packaging. https://github.com/pypa
>I believe the google groups link is their discussion forum.
They have one -- https://groups.google.com/forum/#!forum/pypa-dev
but you can also get them at the disutils mailing list.
https://mail.python.org/mailman/listinfo/distutils-sig
I think, rather than discussion, it is 'people willing to write code'
that they are short of ...
Laura