From victor.stinner at gmail.com Fri Aug 5 15:24:25 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Fri, 5 Aug 2016 21:24:25 +0200 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: Message-ID: Hi, Would it be possible to take a decision on the PEP 522 and PEP 524? Deadline for new features in Python 3.6 is in one month or something like that, no? My PEP: https://www.python.org/dev/peps/pep-0524/ "PEP 524 -- Make os.urandom() blocking on Linux" Nick's PEP: https://www.python.org/dev/peps/pep-0522/ "PEP 522 -- Allow BlockingIOError in security sensitive APIs" Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Fri Aug 5 15:42:53 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 05 Aug 2016 12:42:53 -0700 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: Message-ID: <57A4EC3D.50005@stoneleaf.us> On 08/05/2016 12:24 PM, Victor Stinner wrote: > Would it be possible to take a decision on the PEP 522 and PEP 524? > Deadline for new features in Python 3.6 is in one month or something > like that, no? > > My PEP: > https://www.python.org/dev/peps/pep-0524/ > "PEP 524 -- Make os.urandom() blocking on Linux" > > Nick's PEP: > https://www.python.org/dev/peps/pep-0522/ > "PEP 522 -- Allow BlockingIOError in security sensitive APIs" Can someone write a brief summary of the differences between the two? -- ~Ethan~ From victor.stinner at gmail.com Fri Aug 5 19:13:41 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Sat, 6 Aug 2016 01:13:41 +0200 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: <57A4EC3D.50005@stoneleaf.us> References: <57A4EC3D.50005@stoneleaf.us> Message-ID: Le 5 ao?t 2016 9:42 PM, "Ethan Furman" a ?crit : > Can someone write a brief summary of the differences between the two? Oh, it's hard to summarize. Let me try. As you may expect, my summary is far from being fair :-D -- The two PEPs propose a very different behaviour when os.urandom() would block: raise an exception (522) or wait (524). The PEP 522 forces developers to explicitly handle a rare case (when urandom blocks). The PEP 524 proposes to be optimistic and hope that if urandom hangs, it doesn't hang too long. The corner case of the corner case is when urandom blocks really too long (longer than 60 seconds, or simply forever). The PEP 524 doesn't handle it (block). The PEP 522 makes the exceptional corner case as important as the common case (urandom just blocks a few seconds, or don't block at all). -- Both PEPs want to make Python more secure: don't fall back on the "weak" /dev/urandom (define weak: XXX) in os.urandom() before system urandom is initialized. Most differences between the two PEPs only impact applications calling os.urandom() very early during system initialization (before system urandom initialization) on a system with very slow entropy source or just no entropy (VM, embeded device, ...). The PEP 522 proposes to raise an exception on such case. It forces developers to modify their code to decide how to handle such corner case: wait a few seconds, switch to a weaker entropy source (and maybe log a waning/error), etc. The PEP 524 (mine) proposes to block. Applications don't need to be modified. The expectation is that the kernel will be able to get enough entropy fast enough. By the way, blocking on system urandom is not something new, SSH has the same behaviour for example (try SSH on such VM with no entropy...). -- The PEP 524 proposes also to add a new function os.getrandom() for people who understand low level stuff and security and want to enhance their application on the low entropy case. It allows to reimplement the PEP 522 on Linux in a few lines of pure Python, so give control when urandom would block (no black magic, just call os.getrandom(os.GRND_NOBLOCK) which raises BlockingIOError). The PEP 522 proposes a new function to wait for system urandom initialization. Something similar to the PEP 524 but it requires to modify all applications to use it (to get PEP 524 behaviour). -- IMO PEP 524 has a lower impact on backward compatibility and is easier to implement. The risk of the PEP 524 is that developers start to expect that os.urandom() will *never* block which simply cannot be implemented on all platforms. Both PEP are specific to Linux (even if Solaris will benefit of the same enhancement), but even just on Linux os.urandom() can still block (don't raise the expected BlockingIOError) on Linux older than 3.17. Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Aug 5 22:39:35 2016 From: guido at python.org (Guido van Rossum) Date: Fri, 5 Aug 2016 19:39:35 -0700 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: Thanks, Victor! I've (mostly) read both PEPs and your summary. It seems there are several risks that need to be weighed. 1. An important secret is generated using insufficient entropy. 2. An app blocks unnecessarily. 3. An app crashes unexpectedly. 4. Bad code gets cargo-culted (e.g. through StackOverflow). Both PEPs prevent bad secrets (1), which the status quo (in theory) could lead to this problem. PEP 524 (make os.urandom() block) can cause blocking (2), but prevents crashes (3). PEP 522 (make it raise) can cause crashes (3) but prevents blocking (2). Re (4): With PEP 524, people worried about blocking may be driven to unnecessarily write more complicated code using os.getrandom(). With PEP 522, people worried about crashes may be driven to unnecessarily call secrets.wait_for_system_rng() or put try/except blocks catching raise BlockingIOError around all their os.urandom()-based calls. There's some doubt that (2) or (3) will actually ever happen, because you'd have to be really early in the startup process. On several platforms it is known to be impossible (e.g. on Mac you run Python before the kernel has enough entropy), on other platforms there is no way to avoid the blocking (e.g. Windows), and the proposals really only differ on Linux (and a few other systems, like Solaris, that have getrandom()). My own biggest worry is about (4), cargo-culting -- undoubtedly people will worry about (2) or (3) and they will pass on "robust" code that is unnecessarily complicated and risks being wrong (since the failure mode is *very* hard to reproduce for testing). The cargo-cult code shown in PEP 524 (https://www.python.org/dev/peps/pep-0524/#best-effort-rng) is much worse than that recommended by PEP 522 (secrets.wait_for_system_rng() -- though I could also imagine people wrapping os.urandom() in try/except). But for me personally, it is much easier to stop worrying about a tiny chance of blocking (2) than it would be to believe that the chance of crashes (3) is truly so small that I don't have to do anything about it. Especially since on non-Linux platforms there is a chance of blocking anyway, with no way to prevent it. It's easy to stop worrying about something you can't control. (I don't really worry about that big asteroid that's going to hit the earth in the next millennium either. :-) So I'm in favor of PEP 524. I think I would be even more in favor of it if it didn't add os.getrandom(), since then the whole possibility of cargo-culting unnecessary countermeasures would be pretty much gone (though for the die-hards there's always ctypes...). But I also agree with the idea of exposing the platform's primitive operations when they exist. --Guido On Fri, Aug 5, 2016 at 4:13 PM, Victor Stinner wrote: > Le 5 ao?t 2016 9:42 PM, "Ethan Furman" a ?crit : >> Can someone write a brief summary of the differences between the two? > > Oh, it's hard to summarize. Let me try. As you may expect, my summary is far > from being fair :-D > > -- > > The two PEPs propose a very different behaviour when os.urandom() would > block: raise an exception (522) or wait (524). > > The PEP 522 forces developers to explicitly handle a rare case (when urandom > blocks). > > The PEP 524 proposes to be optimistic and hope that if urandom hangs, it > doesn't hang too long. > > The corner case of the corner case is when urandom blocks really too long > (longer than 60 seconds, or simply forever). > > The PEP 524 doesn't handle it (block). The PEP 522 makes the exceptional > corner case as important as the common case (urandom just blocks a few > seconds, or don't block at all). > > -- > > Both PEPs want to make Python more secure: don't fall back on the "weak" > /dev/urandom (define weak: XXX) in os.urandom() before system urandom is > initialized. > > Most differences between the two PEPs only impact applications calling > os.urandom() very early during system initialization (before system urandom > initialization) on a system with very slow entropy source or just no entropy > (VM, embeded device, ...). > > The PEP 522 proposes to raise an exception on such case. It forces > developers to modify their code to decide how to handle such corner case: > wait a few seconds, switch to a weaker entropy source (and maybe log a > waning/error), etc. > > The PEP 524 (mine) proposes to block. Applications don't need to be > modified. The expectation is that the kernel will be able to get enough > entropy fast enough. By the way, blocking on system urandom is not something > new, SSH has the same behaviour for example (try SSH on such VM with no > entropy...). > > -- > > The PEP 524 proposes also to add a new function os.getrandom() for people > who understand low level stuff and security and want to enhance their > application on the low entropy case. It allows to reimplement the PEP 522 on > Linux in a few lines of pure Python, so give control when urandom would > block (no black magic, just call os.getrandom(os.GRND_NOBLOCK) which raises > BlockingIOError). > > The PEP 522 proposes a new function to wait for system urandom > initialization. Something similar to the PEP 524 but it requires to modify > all applications to use it (to get PEP 524 behaviour). > > -- > > IMO PEP 524 has a lower impact on backward compatibility and is easier to > implement. > > The risk of the PEP 524 is that developers start to expect that os.urandom() > will *never* block which simply cannot be implemented on all platforms. Both > PEP are specific to Linux (even if Solaris will benefit of the same > enhancement), but even just on Linux os.urandom() can still block (don't > raise the expected BlockingIOError) on Linux older than 3.17. > > Victor > > > _______________________________________________ > Security-SIG mailing list > Security-SIG at python.org > https://mail.python.org/mailman/listinfo/security-sig > -- --Guido van Rossum (python.org/~guido) From victor.stinner at gmail.com Sat Aug 6 04:32:06 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Sat, 6 Aug 2016 10:32:06 +0200 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: Le 6 ao?t 2016 04:39, "Guido van Rossum" a ?crit : > 4. Bad code gets cargo-culted (e.g. through StackOverflow). > Re (4): With PEP 524, people worried about blocking may be driven to > unnecessarily write more complicated code using os.getrandom(). With > PEP 522, people worried about crashes may be driven to unnecessarily > call secrets.wait_for_system_rng() or put try/except blocks catching > raise BlockingIOError around all their os.urandom()-based calls. What can we do to reduce this issue? Promote the best recipes in the documentation of the random and/or secrets module? Add Nick's secrets.wait_for_system_rng()? I have to confess that I don't like my own examples :-) I wrote them to show that you can reimplement the PEP 522 use cases and examples in a few lines. The worst example is "try system urandom, if it would block, use the random module". IMO this use case is artificial. If you need security, the random module must not be used. If you don't need security, why would you take the risk of blocking your application (2) with os.urandom()? Always use the random module no? Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Aug 6 04:46:55 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 6 Aug 2016 18:46:55 +1000 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: On 6 August 2016 at 18:32, Victor Stinner wrote: > Le 6 ao?t 2016 04:39, "Guido van Rossum" a ?crit : >> 4. Bad code gets cargo-culted (e.g. through StackOverflow). > >> Re (4): With PEP 524, people worried about blocking may be driven to >> unnecessarily write more complicated code using os.getrandom(). With >> PEP 522, people worried about crashes may be driven to unnecessarily >> call secrets.wait_for_system_rng() or put try/except blocks catching >> raise BlockingIOError around all their os.urandom()-based calls. > > What can we do to reduce this issue? Promote the best recipes in the > documentation of the random and/or secrets module? Add Nick's > secrets.wait_for_system_rng()? At the moment, PEP 522 doesn't propose making the secrets API block implicitly. I was already starting to have doubts about that, and given Guido's feedback, I think I should change it so that it does. That would give the following overall outcome: - the random APIs will never block (but shouldn't be used for secrets) - the secrets APIs will block if they need to (including secrets.wait_for_system_rng()) - os.urandom() may raise BlockingIOError if you don't wait for the system RNG first - random.SystemRandom() may raise BlockingIOError if you don't wait for the system RNG first And if in the latter two cases someone is directed to the secrets module to wait for the system RNG to be ready (e.g. in the error message we raise), they may find that secrets offers a higher level API for whatever they were trying to do anyway. Meanwhile, folks that want to do something other than block if the system RNG isn't ready (like log potentially relevant details of the system encountering the lack of entropy) can just catch BlockingIOError, rather than needing to use platform specific APIs like os.getrandom(). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Sat Aug 6 06:14:23 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 6 Aug 2016 20:14:23 +1000 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: On 6 August 2016 at 18:46, Nick Coghlan wrote: > On 6 August 2016 at 18:32, Victor Stinner wrote: >> Le 6 ao?t 2016 04:39, "Guido van Rossum" a ?crit : >>> 4. Bad code gets cargo-culted (e.g. through StackOverflow). >> >>> Re (4): With PEP 524, people worried about blocking may be driven to >>> unnecessarily write more complicated code using os.getrandom(). With >>> PEP 522, people worried about crashes may be driven to unnecessarily >>> call secrets.wait_for_system_rng() or put try/except blocks catching >>> raise BlockingIOError around all their os.urandom()-based calls. >> >> What can we do to reduce this issue? Promote the best recipes in the >> documentation of the random and/or secrets module? Add Nick's >> secrets.wait_for_system_rng()? > > At the moment, PEP 522 doesn't propose making the secrets API block > implicitly. I was already starting to have doubts about that, and > given Guido's feedback, I think I should change it so that it does. OK, I've made this change now: https://github.com/python/peps/commit/5392cf9fb86d983b2f06694b742318000ad8bdc2 It turned out to have the nice property of making secrets.token_bytes a blocking drop-in replacement for os.urandom, so I appended a "; see secrets.token_bytes()" to the proposed error message. This should make the "boilerplate" answer either using secrets.token_bytes unconditionally, or else a backwards compatibility dance to use it if available, and fall back to os.urandom otherwise. I also tried to make it more explicit that application frameworks like Django that can make more assumptions about their use cases can easily prevent the BlockingIOError from ever coming up by calling secrets.wait_for_system_rng() when it's available. Most of the other changes were clearing out references to things that have already been handled outside the PEP process (i.e. agreeing that os.getrandom() is useful to expose as a platform feature, agreeing that SipHash initialisation and random module initialisation shouldn't wait for the system RNG) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From victor.stinner at gmail.com Sat Aug 6 06:23:45 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Sat, 6 Aug 2016 12:23:45 +0200 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: An alternative would be to add to my PEP 524 an *optional* random.SystemRandomNonblock which is basically the PEP 522 (raise if it would block). "Optional"... or maybe make it always available but block(!) on some platforms? (Bad idea IMO) I dislike the idea of adding 2 new functions to generate random in the same PEP (getrandom, SystemRandomNonblock), it's already hard enough to pick to right one in Python 3.5... Victor Le 6 ao?t 2016 10:46 AM, "Nick Coghlan" a ?crit : > On 6 August 2016 at 18:32, Victor Stinner > wrote: > > Le 6 ao?t 2016 04:39, "Guido van Rossum" a ?crit : > >> 4. Bad code gets cargo-culted (e.g. through StackOverflow). > > > >> Re (4): With PEP 524, people worried about blocking may be driven to > >> unnecessarily write more complicated code using os.getrandom(). With > >> PEP 522, people worried about crashes may be driven to unnecessarily > >> call secrets.wait_for_system_rng() or put try/except blocks catching > >> raise BlockingIOError around all their os.urandom()-based calls. > > > > What can we do to reduce this issue? Promote the best recipes in the > > documentation of the random and/or secrets module? Add Nick's > > secrets.wait_for_system_rng()? > > At the moment, PEP 522 doesn't propose making the secrets API block > implicitly. I was already starting to have doubts about that, and > given Guido's feedback, I think I should change it so that it does. > > That would give the following overall outcome: > > - the random APIs will never block (but shouldn't be used for secrets) > - the secrets APIs will block if they need to (including > secrets.wait_for_system_rng()) > - os.urandom() may raise BlockingIOError if you don't wait for the > system RNG first > - random.SystemRandom() may raise BlockingIOError if you don't wait > for the system RNG first > > And if in the latter two cases someone is directed to the secrets > module to wait for the system RNG to be ready (e.g. in the error > message we raise), they may find that secrets offers a higher level > API for whatever they were trying to do anyway. > > Meanwhile, folks that want to do something other than block if the > system RNG isn't ready (like log potentially relevant details of the > system encountering the lack of entropy) can just catch > BlockingIOError, rather than needing to use platform specific APIs > like os.getrandom(). > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Aug 6 10:29:15 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 7 Aug 2016 00:29:15 +1000 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: On 6 August 2016 at 20:23, Victor Stinner wrote: > An alternative would be to add to my PEP 524 an *optional* > random.SystemRandomNonblock which is basically the PEP 522 (raise if it > would block). "Optional"... or maybe make it always available but block(!) > on some platforms? (Bad idea IMO) No, we don't want anything new added to the random module for this - outside backwards compatibility considerations, random.SystemRandom should be superseded entirely for security sensitive purposes by the module level APIs in the secrets module. > I dislike the idea of adding 2 new functions to generate random in the same > PEP (getrandom, SystemRandomNonblock), it's already hard enough to pick to > right one in Python 3.5... With the changes to PEP 522, secrets.token_bytes will be a cross-platform blocking API regardless of which underlying implementation model we choose - either inheriting that behaviour from os.urandom() (PEP 524), or adding it when encountering BlockingIOError (PEP 522). That means the essential question becomes: Should os.urandom() just be secrets.token_bytes() without a default number of bytes requested? Or does it make more sense to use it to expose the Linux sys.getrandom() non-blocking behaviour to Python code in a platform independent way? Since we're going to have the two level API anyway (os module vs secrets), and have two different behaviours we'd like to expose (blocking vs non-blocking with notification), the latter design is the one I ended up converging on: high level API with implicit blocking, low level API that never blocks, but may throw an exception. It's not where I expected to end up when I first wrote the PEP, but that's the PEP process for you :) Cheers, Nick. > > Victor > > > Le 6 ao?t 2016 10:46 AM, "Nick Coghlan" a ?crit : >> >> On 6 August 2016 at 18:32, Victor Stinner >> wrote: >> > Le 6 ao?t 2016 04:39, "Guido van Rossum" a ?crit : >> >> 4. Bad code gets cargo-culted (e.g. through StackOverflow). >> > >> >> Re (4): With PEP 524, people worried about blocking may be driven to >> >> unnecessarily write more complicated code using os.getrandom(). With >> >> PEP 522, people worried about crashes may be driven to unnecessarily >> >> call secrets.wait_for_system_rng() or put try/except blocks catching >> >> raise BlockingIOError around all their os.urandom()-based calls. >> > >> > What can we do to reduce this issue? Promote the best recipes in the >> > documentation of the random and/or secrets module? Add Nick's >> > secrets.wait_for_system_rng()? >> >> At the moment, PEP 522 doesn't propose making the secrets API block >> implicitly. I was already starting to have doubts about that, and >> given Guido's feedback, I think I should change it so that it does. >> >> That would give the following overall outcome: >> >> - the random APIs will never block (but shouldn't be used for secrets) >> - the secrets APIs will block if they need to (including >> secrets.wait_for_system_rng()) >> - os.urandom() may raise BlockingIOError if you don't wait for the >> system RNG first >> - random.SystemRandom() may raise BlockingIOError if you don't wait >> for the system RNG first >> >> And if in the latter two cases someone is directed to the secrets >> module to wait for the system RNG to be ready (e.g. in the error >> message we raise), they may find that secrets offers a higher level >> API for whatever they were trying to do anyway. >> >> Meanwhile, folks that want to do something other than block if the >> system RNG isn't ready (like log potentially relevant details of the >> system encountering the lack of entropy) can just catch >> BlockingIOError, rather than needing to use platform specific APIs >> like os.getrandom(). >> >> Cheers, >> Nick. >> >> -- >> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From guido at python.org Sat Aug 6 13:21:09 2016 From: guido at python.org (Guido van Rossum) Date: Sat, 6 Aug 2016 10:21:09 -0700 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: I apologize, Maybe I wasn't clear. I'm voting in favor of Victor's PEP 524, i.e. making os.urandom() always blocking, over introducing an exception so rare that it's impossible to provoke without mocking entirely. We may be trying to steer people away from os.urandom(), but it's a venerable API that has been around and stable since Python 2.4. Telling people "oh, BTW, you must now catch BlockingIOError in your code that calls os.urandom()" is a problem for straddling code, since BlockingIOError doesn't even exist in Python 2 (though its base class, OSError, does). Having to think about the consequences of os.urandom() blocking is necessary regardless of which PEP is implemented, since on some platforms it will block (though rarely). And the correct response is almost always "so let it block in those rare cases, it's just going to be a little hiccup". People working in an asyncio world may want to send secure random calls off to the thread pool using e.g. `loop.run_in_executor(None, os.urandom, 128)` -- but they have to think that way anyways because of other platforms, and they will have to do this for the recommended higher-level secure random APIs too. I really see a much bigger downside to adding the possibility that os.urandom() raises BlockingIOError, compared to accepting the possibility that it may block (which is hardly news). There is one thing that is still really unresolved for me, and that is a good understanding of how likely this feared event, "not having enough entropy" actually is, for environments where Python may actually be used. My main question is, can it occur in situations *other* than during very early startup? What's the answer for various platforms? Once I'm past this boot phase, can I safely assume os.urandom() will never block, or is there still a possibility for a system to run out of entropy later (say, by excessive calls to os.urandom(), possibly in another process)? The text of https://www.python.org/dev/peps/pep-0522/#adding-secrets-wait-for-system-rng suggests that that is *not* a possibility (since it recommends putting that call in __main__). Anyways, if the answer ends up being "yes, some systems may occasionally run out of entropy during normal operation", I would count that as a further point against PEP 522. But, assuming I am asked for a vote, my vote goes to Victor's PEP 524, making os.urandom() occasionally block even on Linux, and adding os.getrandom() on those platforms that have it. --Guido On Sat, Aug 6, 2016 at 7:29 AM, Nick Coghlan wrote: > On 6 August 2016 at 20:23, Victor Stinner wrote: >> An alternative would be to add to my PEP 524 an *optional* >> random.SystemRandomNonblock which is basically the PEP 522 (raise if it >> would block). "Optional"... or maybe make it always available but block(!) >> on some platforms? (Bad idea IMO) > > No, we don't want anything new added to the random module for this - > outside backwards compatibility considerations, random.SystemRandom > should be superseded entirely for security sensitive purposes by the > module level APIs in the secrets module. > >> I dislike the idea of adding 2 new functions to generate random in the same >> PEP (getrandom, SystemRandomNonblock), it's already hard enough to pick to >> right one in Python 3.5... > > With the changes to PEP 522, secrets.token_bytes will be a > cross-platform blocking API regardless of which underlying > implementation model we choose - either inheriting that behaviour from > os.urandom() (PEP 524), or adding it when encountering BlockingIOError > (PEP 522). > > That means the essential question becomes: Should os.urandom() just be > secrets.token_bytes() without a default number of bytes requested? Or > does it make more sense to use it to expose the Linux sys.getrandom() > non-blocking behaviour to Python code in a platform independent way? > > Since we're going to have the two level API anyway (os module vs > secrets), and have two different behaviours we'd like to expose > (blocking vs non-blocking with notification), the latter design is the > one I ended up converging on: high level API with implicit blocking, > low level API that never blocks, but may throw an exception. It's not > where I expected to end up when I first wrote the PEP, but that's the > PEP process for you :) > > Cheers, > Nick. > >> >> Victor >> >> >> Le 6 ao?t 2016 10:46 AM, "Nick Coghlan" a ?crit : >>> >>> On 6 August 2016 at 18:32, Victor Stinner >>> wrote: >>> > Le 6 ao?t 2016 04:39, "Guido van Rossum" a ?crit : >>> >> 4. Bad code gets cargo-culted (e.g. through StackOverflow). >>> > >>> >> Re (4): With PEP 524, people worried about blocking may be driven to >>> >> unnecessarily write more complicated code using os.getrandom(). With >>> >> PEP 522, people worried about crashes may be driven to unnecessarily >>> >> call secrets.wait_for_system_rng() or put try/except blocks catching >>> >> raise BlockingIOError around all their os.urandom()-based calls. >>> > >>> > What can we do to reduce this issue? Promote the best recipes in the >>> > documentation of the random and/or secrets module? Add Nick's >>> > secrets.wait_for_system_rng()? >>> >>> At the moment, PEP 522 doesn't propose making the secrets API block >>> implicitly. I was already starting to have doubts about that, and >>> given Guido's feedback, I think I should change it so that it does. >>> >>> That would give the following overall outcome: >>> >>> - the random APIs will never block (but shouldn't be used for secrets) >>> - the secrets APIs will block if they need to (including >>> secrets.wait_for_system_rng()) >>> - os.urandom() may raise BlockingIOError if you don't wait for the >>> system RNG first >>> - random.SystemRandom() may raise BlockingIOError if you don't wait >>> for the system RNG first >>> >>> And if in the latter two cases someone is directed to the secrets >>> module to wait for the system RNG to be ready (e.g. in the error >>> message we raise), they may find that secrets offers a higher level >>> API for whatever they were trying to do anyway. >>> >>> Meanwhile, folks that want to do something other than block if the >>> system RNG isn't ready (like log potentially relevant details of the >>> system encountering the lack of entropy) can just catch >>> BlockingIOError, rather than needing to use platform specific APIs >>> like os.getrandom(). >>> >>> Cheers, >>> Nick. >>> >>> -- >>> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > > > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -- --Guido van Rossum (python.org/~guido) From donald at stufft.io Sat Aug 6 13:34:36 2016 From: donald at stufft.io (Donald Stufft) Date: Sat, 6 Aug 2016 13:34:36 -0400 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: > On Aug 6, 2016, at 1:21 PM, Guido van Rossum wrote: > > > There is one thing that is still really unresolved for me, and that is > a good understanding of how likely this feared event, "not having > enough entropy" actually is, for environments where Python may > actually be used. My main question is, can it occur in situations > *other* than during very early startup? What's the answer for various > platforms? Once I'm past this boot phase, can I safely assume > os.urandom() will never block, or is there still a possibility for a > system to run out of entropy later (say, by excessive calls to > os.urandom(), possibly in another process)? The text of > https://www.python.org/dev/peps/pep-0522/#adding-secrets-wait-for-system-rng > suggests that that is *not* a possibility (since it recommends putting > that call in __main__). > > Anyways, if the answer ends up being "yes, some systems may > occasionally run out of entropy during normal operation", I would > count that as a further point against PEP 522. For all of the major platforms that I can think of, once os.urandom is out of this early boot phase it will never block (or it will never block ever on that platform). This covers: * Linux * Windows * OS X * FreeBSD * OpenBSD * Solaris (I think) I have no idea what more obscure platforms like AIX do, I suspect they?ll behave like older Linux though, where /dev/urandom will never block and might give bad data. This means that once you?ve gotten any data from a urandom that could possibly block, it will never block. I could be wrong though. Essentially, you?re waiting for the device to be fully initialized, and once it is initialized it is initialized, it will never revert to an ?uninitialized? state. This has a side effect that if someone wanted to say, ensure that os.urandom was non-blocking before binding to a port with an asyncio daemon they could simply call ``os.urandom(1)`` which will either return immediately if the urandom device is already initialized or block until it is initialized. > But, assuming I am asked for a vote, my vote goes to Victor's PEP 524, > making os.urandom() occasionally block even on Linux, and adding > os.getrandom() on those platforms that have it. > I agree, though I?m neutral on os.getrandom. ? Donald Stufft From tim.peters at gmail.com Sat Aug 6 13:58:36 2016 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 6 Aug 2016 12:58:36 -0500 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: [Guido] > ... > There is one thing that is still really unresolved for me, and that is > a good understanding of how likely this feared event, "not having > enough entropy" actually is, for environments where Python may > actually be used. My main question is, can it occur in situations > *other* than during very early startup? What's the answer for various > platforms? Once I'm past this boot phase, can I safely assume > os.urandom() will never block, or is there still a possibility for a > system to run out of entropy later (say, by excessive calls to > os.urandom(), possibly in another process)? No such platforms have been identified in any of these messages, so "no" is the answer - for now ;-) Under the covers, all these things use _some_ crypto-strength but deterministic PRNG. So the only time they _may_ get in real trouble is at startup, waiting to initialize the CSPRNG's state from "enough" random noise (even then, sane environments save a file of gibberish before shutdown to use to seed the CSPRNG "immediately" at the next boot). The best systems periodically mix fresh "entropy" into the CSPRNG's state all along, but don't wait for it after initialization. > ... > But, assuming I am asked for a vote, my vote goes to Victor's PEP 524, > making os.urandom() occasionally block even on Linux, and adding > os.getrandom() on those platforms that have it. +1 here :-) From ncoghlan at gmail.com Sun Aug 7 12:14:30 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 8 Aug 2016 02:14:30 +1000 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: On 7 August 2016 at 03:21, Guido van Rossum wrote: > There is one thing that is still really unresolved for me, and that is > a good understanding of how likely this feared event, "not having > enough entropy" actually is, for environments where Python may > actually be used. My main question is, can it occur in situations > *other* than during very early startup? What's the answer for various > platforms? Once I'm past this boot phase, can I safely assume > os.urandom() will never block, or is there still a possibility for a > system to run out of entropy later (say, by excessive calls to > os.urandom(), possibly in another process)? The text of > https://www.python.org/dev/peps/pep-0522/#adding-secrets- > wait-for-system-rng > suggests that that is *not* a possibility (since it recommends putting > that call in __main__). > > Anyways, if the answer ends up being "yes, some systems may > occasionally run out of entropy during normal operation", I would > count that as a further point against PEP 522. > I see folks encountering the new exception proposed in one of two ways: 1. They're writing Linux system initialisation software, and forgot the system RNG may not be ready yet 2. They're running security sensitive Python software on a misconfigured hosting platform that isn't seeding the entropy pool correctly (either in a VM or on an embedded system) For the first case, I think either approach to blocking (implicit or explicit) is fine. However, the concern I have with PEP 524 is that in the second case, it makes it incredibly hard for an operations team (who probably aren't going to be Python experts, and are frequently going to be running software they didn't write) to debug the problem - rather than a crashed application with a full Python traceback (which they can take back to the dev team or vendor and ask "What does this mean?", or else look up on the internet themselves), all the platform operators will have to go on is "This application hangs at startup". strace should at least be able to tell them that it's hanging in a getrandom() kernel call, but it's still going to take a pretty capable sysadmin to be able to figure out what's going on. In a lot of ways, I see it as being similar to our dependency on the Linux platform locale being set correctly to get boundary processing right: if you get an exception, the problem *isn't* generally with the application, it's with the way Linux has been configured. The same holds here - if you get BlockingIOError from os.urandom under PEP 524, there's nothing wrong with your application, but there *is* something wrong with your environment (since security sensitive Python code should only be run after the system RNG is ready) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sun Aug 7 12:17:27 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 8 Aug 2016 02:17:27 +1000 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: On 8 August 2016 at 02:14, Nick Coghlan wrote: > In a lot of ways, I see it as being similar to our dependency on the Linux > platform locale being set correctly to get boundary processing right: if > you get an exception, the problem *isn't* generally with the application, > it's with the way Linux has been configured. The same holds here - if you > get BlockingIOError from os.urandom under PEP 524, there's nothing wrong > with your application, but there *is* something wrong with your environment > (since security sensitive Python code should only be run after the system > RNG is ready) > Oops, that reference should have been to PEP 522. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Sun Aug 7 12:28:02 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 07 Aug 2016 09:28:02 -0700 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: <57A76192.5070207@stoneleaf.us> On 08/07/2016 09:14 AM, Nick Coghlan wrote: > On 7 August 2016 at 03:21, Guido van Rossum wrote: >> >> There is one thing that is still really unresolved for me, and that is >> a good understanding of how likely this feared event, "not having >> enough entropy" actually is, for environments where Python may >> actually be used. My main question is, can it occur in situations >> *other* than during very early startup? What's the answer for various >> platforms? Once I'm past this boot phase, can I safely assume >> os.urandom() will never block, or is there still a possibility for a >> system to run out of entropy later (say, by excessive calls to >> os.urandom(), possibly in another process)? The text of [PEP 522] >> suggests that that is *not* a possibility (since it recommends putting >> that call in __main__). >> >> Anyways, if the answer ends up being "yes, some systems may >> occasionally run out of entropy during normal operation", I would >> count that as a further point against PEP 522. > > I see folks encountering the new exception proposed in one of two ways: > > 1. They're writing Linux system initialisation software, and forgot the > system RNG may not be ready yet > 2. They're running security sensitive Python software on a misconfigured > hosting platform that isn't seeding the entropy pool correctly (either > in a VM or on an embedded system) > > For the first case, I think either approach to blocking (implicit or explicit) is fine. > > However, the concern I have with PEP 524 is that in the second case, it makes it incredibly hard for an operations team (who probably aren't going to be Python experts, and are frequently going to be running software they didn't write) to debug the problem - rather than a crashed application with a full Python traceback (which they can take back to the dev team or vendor and ask "What does this mean?", or else look up on the internet themselves), all the platform operators will have to go on is "This application hangs at startup". strace should at least be able to tell them that it's hanging in a getrandom() kernel call, but it's still going to take a pretty capable sysadmin to be able to figure out what's going on. > > In a lot of ways, I see it as being similar to our dependency on the Linux platform locale being set correctly to get boundary processing right: if you get an exception, the problem *isn't* generally with the application, it's with the way Linux has been configured. The same holds here - if you get BlockingIOError from os.urandom under PEP [522], there's nothing wrong with your application, but there *is* something wrong with your environment (since security sensitive Python code should only be run after the system RNG is ready) +1 If I had not been involved in these discussions about early linux startup, virtual machines, and os.urandom I would be completely mystified by the error presented when I ran across it (stalled and eventually killed process), with no clue about the nature of the problem. At this point we have concrete examples of the harm caused by blocking on os.urandom -- do we have any actual use-cases where it is hurtful to raise instead? -- ~Ethan~ From donald at stufft.io Sun Aug 7 12:33:23 2016 From: donald at stufft.io (Donald Stufft) Date: Sun, 7 Aug 2016 12:33:23 -0400 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: <57A76192.5070207@stoneleaf.us> References: <57A4EC3D.50005@stoneleaf.us> <57A76192.5070207@stoneleaf.us> Message-ID: <4D5E4C0E-4321-4394-A360-F5346833CCF5@stufft.io> > On Aug 7, 2016, at 12:28 PM, Ethan Furman wrote: > > At this point we have concrete examples of the harm caused by blocking on os.urandom -- do we have any actual use-cases where it is hurtful to raise instead? The use cases there are basically any time it would have only blocked for say, half a second or so. It?s hard to point out a specific use case because we?ve never had an error raising there, we?ve either just silently given them bad data because /dev/urandom wasn?t initialized or we blocked and they didn?t notice because it only blocked for a short time. I suspect that the ?can block for a short time? will be the dominant case, because the system generally gets entropy quite quickly in most scenarios. The only time it can?t really is if Python is the only thing running early enough in the boot process *and* that thing is calling os.urandom. The problem we had that started this thread was SipHash initialization calling a blocking urandom by a script called by systemd prior to the point where systemd would attempt to reseed urandom from previous boots and prior to the point that systemd parallelizes the boot process. Basically any other time the time to block will be relatively short (and in fact, you see daemons like OpenSSH blocking on start up for similar reasons). ? Donald Stufft From guido at python.org Sun Aug 7 13:56:55 2016 From: guido at python.org (Guido van Rossum) Date: Sun, 7 Aug 2016 10:56:55 -0700 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: <4D5E4C0E-4321-4394-A360-F5346833CCF5@stufft.io> References: <57A4EC3D.50005@stoneleaf.us> <57A76192.5070207@stoneleaf.us> <4D5E4C0E-4321-4394-A360-F5346833CCF5@stufft.io> Message-ID: Can we stop the discussion please? I have picked a winner. The loser may not like it, but the discussion is OVER. On Sun, Aug 7, 2016 at 9:33 AM, Donald Stufft wrote: > >> On Aug 7, 2016, at 12:28 PM, Ethan Furman wrote: >> >> At this point we have concrete examples of the harm caused by blocking on os.urandom -- do we have any actual use-cases where it is hurtful to raise instead? > > The use cases there are basically any time it would have only blocked for say, half a second or so. It?s hard to point out a specific use case because we?ve never had an error raising there, we?ve either just silently given them bad data because /dev/urandom wasn?t initialized or we blocked and they didn?t notice because it only blocked for a short time. > > I suspect that the ?can block for a short time? will be the dominant case, because the system generally gets entropy quite quickly in most scenarios. The only time it can?t really is if Python is the only thing running early enough in the boot process *and* that thing is calling os.urandom. The problem we had that started this thread was SipHash initialization calling a blocking urandom by a script called by systemd prior to the point where systemd would attempt to reseed urandom from previous boots and prior to the point that systemd parallelizes the boot process. > > Basically any other time the time to block will be relatively short (and in fact, you see daemons like OpenSSH blocking on start up for similar reasons). > > ? > Donald Stufft > > > -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Sun Aug 7 14:33:58 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 07 Aug 2016 11:33:58 -0700 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: <4D5E4C0E-4321-4394-A360-F5346833CCF5@stufft.io> References: <57A4EC3D.50005@stoneleaf.us> <57A76192.5070207@stoneleaf.us> <4D5E4C0E-4321-4394-A360-F5346833CCF5@stufft.io> Message-ID: <57A77F16.7010600@stoneleaf.us> On 08/07/2016 09:33 AM, Donald Stufft wrote: > On Aug 7, 2016, at 12:28 PM, Ethan Furman wrote: >> Guido, not trying to change your mind, just trying to understand. >> At this point we have concrete examples of the harm caused by blocking on os.urandom -- do we have any actual use-cases where it is hurtful to raise instead? > The problem we had that started this thread was SipHash initialization calling a blocking urandom by a script called by systemd prior to the point where systemd would attempt to reseed urandom from previous boots and prior to the point that systemd parallelizes the boot process. So if we work around the problem in SipHash, the issue goes away? And does that work-around mean SipHash may not be robust for that instance of Python, but any Python process running that early should be short-lived anyway, so any security issues become vanishingly rare? -- ~Ethan~ From donald at stufft.io Sun Aug 7 14:45:45 2016 From: donald at stufft.io (Donald Stufft) Date: Sun, 7 Aug 2016 14:45:45 -0400 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: <57A77F16.7010600@stoneleaf.us> References: <57A4EC3D.50005@stoneleaf.us> <57A76192.5070207@stoneleaf.us> <4D5E4C0E-4321-4394-A360-F5346833CCF5@stufft.io> <57A77F16.7010600@stoneleaf.us> Message-ID: <89D9E1DE-9CFA-4A3A-AC77-9FD13AF2B30D@stufft.io> > On Aug 7, 2016, at 2:33 PM, Ethan Furman wrote: > > So if we work around the problem in SipHash, the issue goes away? The issue goes away in the sense that starting the Python interpreter *at all* no longer relies on urandom being initialized. If someone uses Python early enough and calls os.urandom (directly or indirectly) then the same problem would occur again for that program. Working around the problem in SipHash simply moves the problem from anytime you try to use the Python interpreter early in the boot process, to anytime you ask for secure random from os.urandom early in the boot process. > And does that work-around mean SipHash may not be robust for that instance of Python, but any Python process running that early should be short-lived anyway, so any security issues become vanishingly rare? This is correct. The security properties of SipHash basically only matter for something that accepts a lot of untrusted input *and* lives a long time. This basically ends up only pretty much only applying to some sort of network available daemon (not entirely, but it?s the main case). It?s also true that the quality of random from urandom doesn?t go from something entirely predictable to entirely random at the exact moment the pool is fully initialized. The quality of random numbers get better the closer to pool initialization you are. This isn?t good enough for many use cases, but for SipHash it?s likely going to get reasonably OK random even in these early boot cases for that particular use case. As a hypothetical, if we wanted to push the needle even further we could *not* work around the SipHash problem and push that need to work around it onto folks calling Python that early by setting a static PYTHONHASHSEED, but the cost is not likely worth the reward. ? Donald Stufft From victor.stinner at gmail.com Sun Aug 7 15:49:39 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Sun, 7 Aug 2016 21:49:39 +0200 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: I am sorry but I'm in holiday and I'm unable to understand if your (Guido) email means that the PEP 524 is accepted, or if the PEP still needs to be reworked? Can someone help me? I'm lost. :-( (Why is this specific topic so much annoying? :-)) Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sun Aug 7 19:11:09 2016 From: guido at python.org (Guido van Rossum) Date: Sun, 7 Aug 2016 16:11:09 -0700 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: Sorry, PEP 524 is accepted, and PEP 522 is rejected. Let os.urandom() be blocking, and let os.getrandom() be added. Congrats, Victor! On Sun, Aug 7, 2016 at 12:49 PM, Victor Stinner wrote: > I am sorry but I'm in holiday and I'm unable to understand if your (Guido) > email means that the PEP 524 is accepted, or if the PEP still needs to be > reworked? > > Can someone help me? I'm lost. :-( > > (Why is this specific topic so much annoying? :-)) > > Victor -- --Guido van Rossum (python.org/~guido) From victor.stinner at gmail.com Sun Aug 7 19:41:25 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Mon, 8 Aug 2016 01:41:25 +0200 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: 2016-08-08 1:11 GMT+02:00 Guido van Rossum : > Sorry, PEP 524 is accepted, and PEP 522 is rejected. Let os.urandom() > be blocking, and let os.getrandom() be added. Congrats, Victor! Ok. I changed the status of my PEP 524 from Draft to Accepted. I will now start to work on the implementation. For Nick's PEP 522, I don't know if its status should be updated to Rejected or Superseded (by the PEP 524). I prefer to let Nick changes the status of his PEP ;-) Victor From ncoghlan at gmail.com Sun Aug 7 22:37:57 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 8 Aug 2016 12:37:57 +1000 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: On 8 August 2016 at 09:41, Victor Stinner wrote: > 2016-08-08 1:11 GMT+02:00 Guido van Rossum : > > Sorry, PEP 524 is accepted, and PEP 522 is rejected. Let os.urandom() > > be blocking, and let os.getrandom() be added. Congrats, Victor! > > Ok. I changed the status of my PEP 524 from Draft to Accepted. I will > now start to work on the implementation. > > For Nick's PEP 522, I don't know if its status should be updated to > Rejected or Superseded (by the PEP 524). I prefer to let Nick changes > the status of his PEP ;-) > Rejected, but I'm still quite concerned by the lack of operator input into this discussion, particularly when we're going against what the Linux kernel developers themselves decided to do - Ted T'so flicked the equivalent switch for the Linux kernel (to make /dev/urandom blocking) and doing so caused some of the systems in their CI fleet to fail. >From a pure developer point of view, I completely understand Guido's perspective that blocking feels safer than risking throwing an exception, as well as wanting to be able to call the issue done and not worry about it anymore. However, from an operations perspective, it means the discussion will move downstream to see whether we (Fedora) agree this is the right behaviour for the *system* Python, or whether we should patch that to throw the error instead of implicitly blocking. Such divergence would be unfortunate (if we ultimately decide to go that way), but managing disagreements with upstreams about appropriate default behaviour is one of the reasons distros *have* the ability to carry patches in the first place. At the very least, I'll be proposing we do this while the 3.6 beta releases are in Fedora Rawhide as a way of gathering objective data about the scope of the problem from ABRT (Fedora's automatic bug reporting tool, which can automatically collect and submit Python stack traces, but can't readily detect system hangs). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sun Aug 7 23:32:55 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 8 Aug 2016 13:32:55 +1000 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: On 8 August 2016 at 12:37, Nick Coghlan wrote: > However, from an operations perspective, it means the discussion will move > downstream to see whether we (Fedora) agree this is the right behaviour for > the *system* Python, or whether we should patch that to throw the error > instead of implicitly blocking. Such divergence would be unfortunate (if we > ultimately decide to go that way), but managing disagreements with > upstreams about appropriate default behaviour is one of the reasons distros > *have* the ability to carry patches in the first place. > > At the very least, I'll be proposing we do this while the 3.6 beta > releases are in Fedora Rawhide as a way of gathering objective data about > the scope of the problem from ABRT (Fedora's automatic bug reporting tool, > which can automatically collect and submit Python stack traces, but can't > readily detect system hangs). > For folks curious as to what I mean here, this isn't a declaration that Fedora *is* going to diverge from upstream in terms of the way os.urandom() behaves in Python 3.6+. Rather, it's a statement that I think we need more data directly from Fedora's users before deciding whether or not it makes sense to abide by the cross-platform upstream behaviour, or carry a patch that changes the behaviour specifically for the system Python installation: https://lists.fedorahosted.org/archives/list/python- devel at lists.fedoraproject.org/thread/UAB7JJ5VPW2W2QEERZ4HIQZZB3QMB2H5/ While the interests of Linux distro users and CPython upstream users are generally pretty well aligned, the alignment isn't 100%, and distros carrying patches on behalf of their user base is what makes up the difference. The Fedora Rawhide experiment I'm proposing in that email to the Fedora Python list should give us the data we (Fedora) need to decide whether or not this is one of those cases where it makes sense for us to carry a patch - if we get zero hits from the exception in ABRT, then it means the default blocking behaviour should be relatively safe (since people won't be encountering it), so we can drop the patch before the F26 Beta release, and Guido will have a solid data point backing up his design instincts. If we *do* get hits on the exception, then exactly what we do will depend on the nature of those hits, and in particular whether or not the change is helping folks find misconfigured Fedora environments they hadn't previously noticed, or if they're spurious notifications in situations where just blocking for a few hundred milliseconds would have resolved the problem on its own (as tested by inserting a "python -c 'import os; os.getrandom(1)" before whatever application startup is triggering the new exception). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Mon Aug 8 00:22:47 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 8 Aug 2016 14:22:47 +1000 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: On 8 August 2016 at 13:32, Nick Coghlan wrote: > The Fedora Rawhide experiment I'm proposing in that email to the Fedora > Python list should give us the data we (Fedora) need to decide whether or > not this is one of those cases where it makes sense for us to carry a patch > - if we get zero hits from the exception in ABRT, then it means the default > blocking behaviour should be relatively safe (since people won't be > encountering it), so we can drop the patch before the F26 Beta release, and > Guido will have a solid data point backing up his design instincts. If we > *do* get hits on the exception, then exactly what we do will depend on the > nature of those hits, and in particular whether or not the change is > helping folks find misconfigured Fedora environments they hadn't previously > noticed, or if they're spurious notifications in situations where just > blocking for a few hundred milliseconds would have resolved the problem on > its own (as tested by inserting a "python -c 'import os; os.getrandom(1)" > before whatever application startup is triggering the new exception). > I started thinking a bit more about the outcomes we'd be looking for from such an experiment [1], and that reminded of the fact we could potentially do that with a much lower level of divergence if the upstream implementation issued a runtime warning when it needed to fall back to blocking behaviour. That is, rather that just calling "getrandom(size, 0)" unconditionally, the current logic for trying "getrandom(size, GRND_NONBLOCK)" first could be kept, and only the fallback to reading from "/dev/random/" changed to instead call "getrandom(size, 0)" with a preceding call to PyErr_Warn. That warning on its own would address almost all my lingering concerns with the implicit blocking approach, since Python's normal warning control machinery can be used to turn it into an exception if that's the desired behaviour in a given context, and the only policy decision distros would need to make for their system Python is whether they treat that warning as an error by default or not. Regards, Nick. [1] https://lists.fedorahosted.org/archives/list/python-devel at lists.fedoraproject.org/message/HHTMUX6PKPFASE3CNHRHU2JQCWTUD7ZZ/ -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Mon Aug 8 00:24:32 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 8 Aug 2016 14:24:32 +1000 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: On 8 August 2016 at 14:22, Nick Coghlan wrote: > On 8 August 2016 at 13:32, Nick Coghlan wrote: > >> The Fedora Rawhide experiment I'm proposing in that email to the Fedora >> Python list should give us the data we (Fedora) need to decide whether or >> not this is one of those cases where it makes sense for us to carry a patch >> - if we get zero hits from the exception in ABRT, then it means the default >> blocking behaviour should be relatively safe (since people won't be >> encountering it), so we can drop the patch before the F26 Beta release, and >> Guido will have a solid data point backing up his design instincts. If we >> *do* get hits on the exception, then exactly what we do will depend on the >> nature of those hits, and in particular whether or not the change is >> helping folks find misconfigured Fedora environments they hadn't previously >> noticed, or if they're spurious notifications in situations where just >> blocking for a few hundred milliseconds would have resolved the problem on >> its own (as tested by inserting a "python -c 'import os; os.getrandom(1)" >> before whatever application startup is triggering the new exception). >> > > I started thinking a bit more about the outcomes we'd be looking for from > such an experiment [1], and that reminded of the fact we could potentially > do that with a much lower level of divergence if the upstream > implementation issued a runtime warning when it needed to fall back to > blocking behaviour. That is, rather that just calling "getrandom(size, 0)" > unconditionally, the current logic for trying "getrandom(size, > GRND_NONBLOCK)" first could be kept, and only the fallback to reading from > "/dev/random/" changed to instead call "getrandom(size, 0)" with a > preceding call to PyErr_Warn. > Sorry, unhelpful typo there: the fallback is to "/dev/urandom", I just mistyped it and missed the error on proofreading. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From victor.stinner at gmail.com Mon Aug 8 05:59:53 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Mon, 8 Aug 2016 11:59:53 +0200 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: os.urandom() is already blocking in Python 3.5.0 and 3.5.1 :-) For example on Fedora, no need for rawhide: Fedora 24 provides Python 3.5.1 with a blocking os.urandom() :-) I don't know the exact Python 3.5 version of Ubuntu Xental. Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Mon Aug 8 08:40:04 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 8 Aug 2016 22:40:04 +1000 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: <57A4EC3D.50005@stoneleaf.us> Message-ID: On 8 August 2016 at 19:59, Victor Stinner wrote: > os.urandom() is already blocking in Python 3.5.0 and 3.5.1 :-) > > For example on Fedora, no need for rawhide: Fedora 24 provides Python > 3.5.1 with a blocking os.urandom() :-) > Surprisingly, it doesn't, as due to the way the Fedora buildroots are set up in Koji the "HAVE_GETRANDOM_SYSCALL" configure check ends up returning False when the system Python RPM gets built: https://mail.python.org/pipermail/security-sig/2016-June/000060.html With 3.5.2 reverting to the old behaviour anyway, there's no compelling reason to address that build environment discrepancy for 3.5, but we (Fedora) are going to have to do something about it for Python 3.6 in F26 so that os.getrandom() gets defined properly and os.urandom() can be made blocking (with a warning when it does). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Mon Aug 8 11:57:25 2016 From: barry at python.org (Barry Warsaw) Date: Mon, 8 Aug 2016 11:57:25 -0400 Subject: [Security-sig] Take a decision for os.urandom() in Python 3.6 In-Reply-To: References: Message-ID: <20160808115725.0db60d58@subdivisions.wooz.org> On Aug 08, 2016, at 11:59 AM, Victor Stinner wrote: >I don't know the exact Python 3.5 version of Ubuntu Xental. Ubuntu 16.04 LTS (Xenial Xerus) has 3.5.1-ish in the primary archive pocket, but 3.5.2-ish in xenial-updates, which most people will have enabled. Of course, the devil is in the details, in this case the patches cherry picked and otherwise on top of those base upstream versions. Cheers, -Barry From christian at python.org Mon Aug 15 13:12:45 2016 From: christian at python.org (Christian Heimes) Date: Mon, 15 Aug 2016 19:12:45 +0200 Subject: [Security-sig] Pending security features for 3.6 Message-ID: Hi, (2nd attempt, first mail didn't make it) I have a bunch of tickets with security-related improvements or features for Python 3.6. Most of the tickets come with patches and tests. Some of the patches might be outdated or conflict with tip. I have branches on my private github fork for all patches. Please review the patches and decide which features you like to include in future releases. Make ssl module compatible with OpenSSL 1.1.0 --------------------------------------------- http://bugs.python.org/issue26470 https://github.com/tiran/cpython/commits/feature/openssl110 https://github.com/tiran/cpython/commits/feature/openssl110_27 OpenSSL 1.1.0 changes several APIs, e.g. it makes structs opaque. The ticket has patches for 2.7 and 3.x series. It should be applied to all Python versions that are open for security patches. Add ChaCha20 Poly1305 to SSL ciphers ------------------------------------ http://bugs.python.org/issue27766 https://github.com/tiran/cpython/commits/feature/chacha20 The ticket changes the default cipher list and moves ChaCha20 Poly1305 up front. For now the patch makes only sense with OpenSSL 1.1.0 since 1.0.2 does not include the cipher. I expect to see backports, though. It should be applied to all Python versions, too. ssl: add public API for IA-32 processor capabilities vector ----------------------------------------------------------- http://bugs.python.org/issue27768 This ticket doesn't have a patch yet. I'm going to move code from ticket 27766 to a separate ticket. Alex and Cory have requested to make the API public. Add AF_ALG (Linux Kernel crypto) to socket module ------------------------------------------------- http://bugs.python.org/issue27744 https://github.com/tiran/cpython/commits/feature/af_alg AF_ALG is a Linux-only socket it to interface with Kernel space crypto. It's limited but has a couple of really useful properties, e.g. zero-copy hashing of files with sendfile() or storing key material securely in Kernel memory. Add BLAKE2 to hashlib --------------------- http://bugs.python.org/issue26798 https://github.com/tiran/cpython/commits/feature/blake2 BLAKE2 is a fast and powerful hash algorithm. It's as secure as SHA-2 family, faster than MD5 and has built-in features like MAC support, variable output length, salting and personalization. Donald uses BLAKE2 for PyPI. The patch was refused on python-dev because it introduces too much new code. Add SHA-3 and SHAKE (Keccak) support ------------------------------------ http://bugs.python.org/issue16113 https://github.com/tiran/cpython/commits/feature/sha3 SHA-3 is the successor of SHA-2. Like BLAKE2 the patch was refused on python-dev because it introduces too much new code. Add truncated SHA512/224 and SHA512/256 --------------------------------------- http://bugs.python.org/issue26834 https://github.com/tiran/cpython/commits/feature/sha512truncated Truncated SHA512/224 and SHA512/256 use the SHA512 algorithm instead of SHA256 algorithm. Like SHA384 it's SHA512 with a different init vector and truncated output. Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From donald at stufft.io Mon Aug 15 13:46:22 2016 From: donald at stufft.io (Donald Stufft) Date: Mon, 15 Aug 2016 13:46:22 -0400 Subject: [Security-sig] Pending security features for 3.6 In-Reply-To: References: Message-ID: > On Aug 15, 2016, at 1:12 PM, Christian Heimes wrote: > > Add BLAKE2 to hashlib > --------------------- > http://bugs.python.org/issue26798 > https://github.com/tiran/cpython/commits/feature/blake2 > > BLAKE2 is a fast and powerful hash algorithm. It's as secure as SHA-2 > family, faster than MD5 and has built-in features like MAC support, > variable output length, salting and personalization. Donald uses BLAKE2 > for PyPI. The patch was refused on python-dev because it introduces too > much new code. This in particular is something I?m very hoping will land. I?m hoping to transition PyPI over to primarily using blake2 (though will need others for backwards compatibility) and not having blake2 in the stdlib makes this much less feasible. ? Donald Stufft From cory at lukasa.co.uk Tue Aug 16 04:25:55 2016 From: cory at lukasa.co.uk (Cory Benfield) Date: Tue, 16 Aug 2016 09:25:55 +0100 Subject: [Security-sig] Pending security features for 3.6 In-Reply-To: References: Message-ID: For what it?s worth, I?d like to highlight the things that are extremely important to my area of the world (namely, securing HTTPS connections). > On 15 Aug 2016, at 18:12, Christian Heimes wrote: > > Make ssl module compatible with OpenSSL 1.1.0 > --------------------------------------------- > http://bugs.python.org/issue26470 > https://github.com/tiran/cpython/commits/feature/openssl110 > https://github.com/tiran/cpython/commits/feature/openssl110_27 > > OpenSSL 1.1.0 changes several APIs, e.g. it makes structs opaque. The > ticket has patches for 2.7 and 3.x series. It should be applied to all > Python versions that are open for security patches. This is extremely important. The 1.1 series of OpenSSL releases is going to be the only collection of OpenSSLs that get support for TLS 1.3, which contains several substantial security and resiliency enhancements. The fact that they?re dramatically changing their API, while annoying for backported Python releases, is not a good reason not to backport this. We should backport to 2.7 and the active 3.x releases for sure. > Add ChaCha20 Poly1305 to SSL ciphers > ------------------------------------ > http://bugs.python.org/issue27766 > https://github.com/tiran/cpython/commits/feature/chacha20 > > The ticket changes the default cipher list and moves ChaCha20 Poly1305 > up front. For now the patch makes only sense with OpenSSL 1.1.0 since > 1.0.2 does not include the cipher. I expect to see backports, though. It > should be applied to all Python versions, too. There?s no reason not to backport this too. ChaCha20-Poly1305 is not currently a security enhancement over the state of the art in TLS (AES-GCM), but it has performance advantages on some platforms and, more importantly, provides us another good AEAD to move to if AES-GCM is broken in any form. Backporting this would also be advantageous, though not required for Requests or Twisted, which have already provided their equivalent patches. > ssl: add public API for IA-32 processor capabilities vector > ----------------------------------------------------------- > http://bugs.python.org/issue27768 > > This ticket doesn't have a patch yet. I'm going to move code from ticket > 27766 to a separate ticket. Alex and Cory have requested to make the API > public. I noted above that ChaCha20-Poly1305 performs better on some platforms. Specifically, it performs better on platforms without the AES-NI extended instruction set. Ideally on platforms without those instructions we?d prioritise ChaCha20-Poly1305 over AES-GCM, but right now we cannot ask that question from Python code. This API would allow us to do so. It?s not urgent, and I don?t mind if we don?t backport it, but it?d be extremely useful to have access to the API (and, to be clear, Requests will almost certainly use the API if it?s available from Python code, *even* if it?s private). The rest are all good, but matter far less to the TLS crowd. =) Cory From victor.stinner at gmail.com Tue Aug 16 13:28:47 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Tue, 16 Aug 2016 19:28:47 +0200 Subject: [Security-sig] Implementation of the PEP 524 Message-ID: Hi, I wrote a first implementation of the PEP 524 (make os.urandom blocking): * https://bugs.python.org/issue27776 : "PEP 524: Make os.urandom() blocking on Linux" * https://bugs.python.org/issue27778 : "PEP 524: Add os.getrandom()" My patches are now waiting for your review :-) You should also review changes made to prepare the issue #27776: https://hg.python.org/cpython/rev/980e2c781810 https://hg.python.org/cpython/rev/265644bad99e https://hg.python.org/cpython/rev/86d0d74bc2e1 https://hg.python.org/cpython/rev/ad141164c792 The last change (close fd on error) is not a really bugfix, it's more a new feature. In Python 3.5, it wasn't needed to close the file descriptor since Py_FatalError() was called immediatly. Victor From ncoghlan at gmail.com Wed Aug 17 12:27:32 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 18 Aug 2016 02:27:32 +1000 Subject: [Security-sig] Pending security features for 3.6 In-Reply-To: References: Message-ID: On 16 August 2016 at 03:12, Christian Heimes wrote: > Hi, > > (2nd attempt, first mail didn't make it) > > I have a bunch of tickets with security-related improvements or features > for Python 3.6. Most of the tickets come with patches and tests. Some of > the patches might be outdated or conflict with tip. I have branches on > my private github fork for all patches. > > Please review the patches and decide which features you like to include > in future releases. I think they all make sense for Python 3.6 - while I acknowledge the maintainability concerns raised on python-dev with the expansion of security related features, I also think that's an ongoing sustainability problem we need to tackle by getting commercial redistributors to better earn their support fees, rather than refusing to allow the work to be done in the interim. For Python 3.5 and 2.7, we're probably due for a successor to PEP 466 that syncs the ssl support in those versions with the 3.6 version of the module - that should hopefully be less controversial this time around. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Wed Aug 17 12:39:58 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 18 Aug 2016 02:39:58 +1000 Subject: [Security-sig] Implementation of the PEP 524 In-Reply-To: References: Message-ID: On 17 August 2016 at 03:28, Victor Stinner wrote: > Hi, > > I wrote a first implementation of the PEP 524 (make os.urandom blocking): > > * https://bugs.python.org/issue27776 : "PEP 524: Make os.urandom() > blocking on Linux" > * https://bugs.python.org/issue27778 : "PEP 524: Add os.getrandom()" > > My patches are now waiting for your review :-) Thanks for tackling this, Victor! I'll do a proper review tomorrow (OK, technically, later today), but could we take a slightly different approach to handling the new "blocking" parameter in py_getrandom? Specifically, I'd like to still make the initial call with GRND_NONBLOCK, then have conditional handling of EAGAIN such that: - if blocking is not set, it behaves as it does now - if blocking *is* set, it prints a warning to stderr, before trying again without the GRND_NONBLOCK flag That would address the main problem I was worried about in PEP 522, which is folks potentially being faced with an unexpected application or service hang, and few clues about where to start in debugging it. The only downside I see is needing two syscalls instead of one in the blocking case, but I'd expect the "blocking" part to be the main delay there, rather than the second syscall. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From victor.stinner at gmail.com Wed Aug 17 13:16:00 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Wed, 17 Aug 2016 19:16:00 +0200 Subject: [Security-sig] Implementation of the PEP 524 In-Reply-To: References: Message-ID: 2016-08-17 18:39 GMT+02:00 Nick Coghlan : > - if blocking *is* set, it prints a warning to stderr, before trying > again without the GRND_NONBLOCK flag I implemented the PEP 524. Such warning is not part of the PEP. Sorry, but I'm not interested to reopen the discussion on the PEP since the PEP was accepted... Victor From ncoghlan at gmail.com Thu Aug 18 08:10:08 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 18 Aug 2016 22:10:08 +1000 Subject: [Security-sig] Implementation of the PEP 524 In-Reply-To: References: Message-ID: On 18 August 2016 at 03:16, Victor Stinner wrote: > 2016-08-17 18:39 GMT+02:00 Nick Coghlan : >> - if blocking *is* set, it prints a warning to stderr, before trying >> again without the GRND_NONBLOCK flag > > I implemented the PEP 524. Such warning is not part of the PEP. > > Sorry, but I'm not interested to reopen the discussion on the PEP > since the PEP was accepted... OK, I'll review it on that basis, and then submit a follow-up RFE and patch to add the warning. I'd like to minimise the divergence between upstream and Fedora, and we'll at least have the warning in the downstream version. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Thu Aug 18 09:03:53 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 18 Aug 2016 23:03:53 +1000 Subject: [Security-sig] Implementation of the PEP 524 In-Reply-To: References: Message-ID: On 18 August 2016 at 22:10, Nick Coghlan wrote: > On 18 August 2016 at 03:16, Victor Stinner wrote: >> 2016-08-17 18:39 GMT+02:00 Nick Coghlan : >>> - if blocking *is* set, it prints a warning to stderr, before trying >>> again without the GRND_NONBLOCK flag >> >> I implemented the PEP 524. Such warning is not part of the PEP. >> >> Sorry, but I'm not interested to reopen the discussion on the PEP >> since the PEP was accepted... > > OK, I'll review it on that basis, and then submit a follow-up RFE and > patch to add the warning. Mostly +1 from me on the code and test changes in both patches (I did have a few requests for clarification and confirmation, but nothing major). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Thu Aug 18 21:09:52 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 19 Aug 2016 11:09:52 +1000 Subject: [Security-sig] Implementation of the PEP 524 In-Reply-To: References: Message-ID: On 18 August 2016 at 22:10, Nick Coghlan wrote: > OK, I'll review it on that basis, and then submit a follow-up RFE and > patch to add the warning. > > I'd like to minimise the divergence between upstream and Fedora, and > we'll at least have the warning in the downstream version. It belatedly occurs to me that I should mention a relevant piece of my work history that may help explain why I'm so strongly in favour of providing some kind of visible notification as to why the Python process is blocked waiting for the kernel (and am happy to tackle it myself as a follow-up patch post-PEP implementation): I spent a couple of years as the development lead for Red Hat's main hardware integration testing system, https://beaker-project.org/ While thankfully rare, the single most difficult operating system level regressions for people to debug in that environment are those where: - the system hangs on startup - before the sshd daemon is running - with no relevant messages on the system console log The cases where guest recipes do this aren't *as* bad (since you can still ssh into the VM host and poke around in the guest via the hypervisor tooling), but when it happens on bare metal, you either need to have physical access to the relevant hardware lab yourself, or get help from a colleague who does. For the os.urandom change, the first two points on that list are entirely out of CPython's control (since they depend on what, if anything, is calling os.urandom() early in the Linux boot cycle), but we *can* influence the third one (by emitting a warning on stderr before we block). Adding such a warning via a downstream patch in Fedora would mostly be sufficient to address the risk for the users of Fedora's & Red Hat's Beaker installations (aside from a few situations involving booting other Linux distros as VM guests), since the two cases where I expect we may see problems are those I mentioned in PEP 522: - testing on ARM hardware that doesn't have a proper entropy source set up - regression tests that run VMs without configuring a proper entropy source for the VM and seeing ":: RuntimeWarning: Waiting for system RNG" in the system console log should be enough to put people on the right path to identifying the root cause of the problem. However, other Linux distros tend to have similar regression testing systems (e.g. it was the Debian one that picked up the issue with 3.5.0), so it makes sense to me to offer this as a general courtesy upstream, rather than just handling it for Fedora and derivatives downstream. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From christian at python.org Fri Aug 26 08:28:57 2016 From: christian at python.org (Christian Heimes) Date: Fri, 26 Aug 2016 14:28:57 +0200 Subject: [Security-sig] Pending security features for 3.6 In-Reply-To: References: Message-ID: <1bde00b0-6f45-191c-8010-87d132e1b831@python.org> Hi, thanks for the feedback. Since my last mail a couple of things have happened. Victor has reviewed my AF_ALG patch and I got some feedback on a new variant of setsockopt() on python-dev. The patch is almost ready. I have submitted updated patch for SHA-3 and BLAKE2 support. Both need a final review and ACK. OpenSSL 1.1 has been released and block ciphers with small blocks have been found insecure. This affects 3DES i our default cipher list. OpenSSL 1.1.0 has removed 3DES, which broke one test. I'm going to update my OpenSSL 1.1 patch soonish. I have two more security tickets in the queue. Please give feedback. Remove 3DES from cipher list (sweet32 CVE-2016-2183) ---------------------------------------------------- https://bugs.python.org/issue27850 Fix for https://sweet32.info/ ssl: get list of enabled ciphers -------------------------------- https://github.com/tiran/cpython/tree/feature/openssl_ciphers https://bugs.python.org/issue27866 Counter part of SSLContext.set_ciphers(), SSLContext.get_ciphers() returns list of dicts with enabled ciphers. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From christian at cheimes.de Fri Aug 26 08:23:46 2016 From: christian at cheimes.de (Christian Heimes) Date: Fri, 26 Aug 2016 14:23:46 +0200 Subject: [Security-sig] Pending security features for 3.6 In-Reply-To: References: Message-ID: <1343888d-366c-65a1-f854-08ddf219f00c@cheimes.de> Hi, thanks for the feedback. Since my last mail a couple of things have happened. Victor has reviewed my AF_ALG patch and I got some feedback on a new variant of setsockopt() on python-dev. The patch is almost ready. I have submitted updated patch for SHA-3 and BLAKE2 support. Both need a final review and ACK. OpenSSL 1.1 has been released and block ciphers with small blocks have been found insecure. This affects 3DES i our default cipher list. OpenSSL 1.1.0 has removed 3DES, which broke one test. I'm going to update my OpenSSL 1.1 patch soonish. I have two more security tickets in the queue. Please give feedback. Remove 3DES from cipher list (sweet32 CVE-2016-2183) ---------------------------------------------------- https://bugs.python.org/issue27850 Fix for https://sweet32.info/ ssl: get list of enabled ciphers -------------------------------- https://github.com/tiran/cpython/tree/feature/openssl_ciphers https://bugs.python.org/issue27866 Counter part of SSLContext.set_ciphers(), SSLContext.get_ciphers() returns list of dicts with enabled ciphers. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: