[Cryptography-dev] Create Fernet API allowing streaming encryption and decryption from file-like objects.

[Jean-Paul Calderone]

Michael Iverson <dr.michael.iverson at gmail.com> writes:

> On Mon, Jan 19, 2015 at 10:47 AM, Jean-Paul Calderone <
> jean-paul at clusterhq.com> wrote:
>
>>
>> A file-like interface is not particularly well-suited for asynchronous
>> operation.  Have you looked into whether Tornado and Cyclone would
>> actually be able to use such an interface?
>>
>>
> I don't see an immediate reason why it could not. I could be wrong on this
> point, however.
>
> For example, here is an excerpt from a simple twisted protocol I wrote. The
> protocol spawns an external process, and reads data from stdout.
> The class has a attribute 'request', which is the file handle generating
> the HTTP response.
>
> For each chunk of data it reads, it (asynchronously) calls
>
>     def outReceived(self, data):
>         self.request.write(data)
>         self.request.flush()
>
> Most of the asynchronous behavior involves waiting for IO or network
> responses, and processing chunks (or entire responses) when available. This
> seems to fit this proposal well.
>

Hm.  I'm not sure I quite follow how this is applicable.  It may be that
I've misunderstood the interface you're suggesting.

In this example, the source of input is asynchronous (`outReceived`
events) and the destination for output is non-blocking (`request.write`
and `request.flush`).

On both sides, there are no considerations for flow-control (though I
happen to know that `self` can probably do something like
`self.transport.pauseProducing()` if it wants to slow down the input
source).  Without flow control, you may end up with your entire dataset
buffered in memory anyway.  The traditional way flow control is
implemented with the file interface is by making `read` or `write` into
a blocking operation.

So, I think flow control is one general shortcoming of the file-based
interface.

A related concern is the tendancy to write asynchronous unfriendly code
using file-based interfaces.  If someone writes a handy utility that
wants to operate on an entire file, it's difficult or impossible to use
that utility in an otherwise asynchronous program (since it will take
arbitrarily long to complete and wants to run all at once).  This is
probably not as serious a concern since people who aren't writing
asynchronous code will almost always write things that aren't
asynchronous friendly and there's nothing about this specific proposal
that makes that any worse.

Jean-Paul

>
>> And it would certainly be bad if this were the *only* interface
>> Cryptography provided to certain functionality (which, fortunately,
>> isn't what you're suggesting, I think).
>>
>
> That is definitely not what I'm suggesting. Just an alternate interface for
> a specific option.
>
>
>>
>> Jean-Paul
>> _______________________________________________
>> Cryptography-dev mailing list
>> Cryptography-dev at python.org
>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>
>
>
>
> -- 
> Dr. Michael Iverson
> Director of Information Technology
> Hatteras Printing
> _______________________________________________
> Cryptography-dev mailing list
> Cryptography-dev at python.org
> https://mail.python.org/mailman/listinfo/cryptography-dev