HTTP compression support for http.server

I have reported an issue in the tracker (https://bugs.python.org/issue30576) and proposed a Pull Request on the Github CPython repository ( https://github.com/python/cpython/pull/2078) to make http.server in the standard library support HTTP compression (gzip). I have been suggested to require feedback from core devs : - should HTTP compression be supported ? - if so, should it be supported by default ? It is the case in the PR, where a number of content types, eg text/html, are compressed if the user agent accepts the gzip "encoding" - if not, should the implementation of http.server be adapted so that subclasses could implement it ? For the moment the only way to add it is to modify method send_head() of SimpleHTTPRequestHandler My opinion is that it should be supported : http.server is not meant to be a full-featured HTTP server, but compression it is a basic and normalized feature of HTTP servers, it is supported by most browsers including on smartphones, it reduces network load, and it's very easy to implement (cf. the Pull Request). For the same reason, I recently added browser cache to http.server (PR #298 <https://github.com/python/cpython/pull/298>). I also think that it should be supported by default for the most common content types (text/html, text/css, application/javascript...) ; the implementation is based on a list of types to compress (SimpleHTTPServer.compressed_types) that can be modified at will, eg set to the empty list to disable compression.

On 7/20/2017 3:15 AM, Pierre Quentel wrote:
Full response on the issue. Briefly
I have been suggested to require feedback from core devs : - should HTTP compression be supported ?
Yes.
- if so, should it be supported by default ?
No, contrary to painfully wrought policy.
Add subclass.
-- Terry Jan Reedy

The opinion of some random guy on the list... On Thu, Jul 20, 2017 at 12:15 AM, Pierre Quentel <pierre.quentel@gmail.com> wrote:
I have been suggested to require feedback from core devs : - should HTTP compression be supported ?
Yes. You are quite right, it's pretty standard stuff these days.
I'm pretty wary of compression happening by default -- i.e. someone runs exactly the same code with a newer version of Python, and suddenly some content is getting compressed. - if not, should the implementation of http.server be adapted so that
subclasses could implement it ? For the moment the only way to add it is to modify method send_head() of SimpleHTTPRequestHandler
sure -- though it would be nice for folks to be able to use compression without going through that process. The implementation is based on a list of types to compress
(SimpleHTTPServer.compressed_types) that can be modified at will, eg set to the empty list to disable compression.
How about having it be an empty list by default and have one or more lists of common types be pre-populated and available in the SimpleHTTPServer namespace. that is: SimpleHTTPServer.compressed_types = SimpleHTTPServer.standard_compressed_types Or may be a method to turn on the "standard" set -- though if it really is simply a list, better to expose that so it's obvious that you can create your own list or examine and edit the existing one(s). Thanks for doing this -- nice feature! -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Tue, Jul 25, 2017 at 2:20 AM, Chris Barker <chris.barker@noaa.gov> wrote:
FWIW I'm quite okay with that. HTTP already has a mechanism for negotiating compression (Accept-Encoding), designed to be compatible with servers that don't support it. Any time a server gains support for something that clients already support, it's going to start happening as soon as you upgrade. Obviously this kind of change won't be happening in a bugfix release of Python, so it would be part of the regular checks when you upgrade from 3.6 to 3.7 - it'll be in the NEWS file and so on, so you read up on it before you upgrade. ChrisA

2017-08-05 12:49 GMT+02:00 Barry <barry@barrys-emacs.org>:
In the latest version of the Pull Request, only gzip is supported. But your comment makes me think that the code should probably be more modular so that subclasses of SimpleHTTPRequestHandler could handle other algorithms.

On 7/20/2017 3:15 AM, Pierre Quentel wrote:
Full response on the issue. Briefly
I have been suggested to require feedback from core devs : - should HTTP compression be supported ?
Yes.
- if so, should it be supported by default ?
No, contrary to painfully wrought policy.
Add subclass.
-- Terry Jan Reedy

The opinion of some random guy on the list... On Thu, Jul 20, 2017 at 12:15 AM, Pierre Quentel <pierre.quentel@gmail.com> wrote:
I have been suggested to require feedback from core devs : - should HTTP compression be supported ?
Yes. You are quite right, it's pretty standard stuff these days.
I'm pretty wary of compression happening by default -- i.e. someone runs exactly the same code with a newer version of Python, and suddenly some content is getting compressed. - if not, should the implementation of http.server be adapted so that
subclasses could implement it ? For the moment the only way to add it is to modify method send_head() of SimpleHTTPRequestHandler
sure -- though it would be nice for folks to be able to use compression without going through that process. The implementation is based on a list of types to compress
(SimpleHTTPServer.compressed_types) that can be modified at will, eg set to the empty list to disable compression.
How about having it be an empty list by default and have one or more lists of common types be pre-populated and available in the SimpleHTTPServer namespace. that is: SimpleHTTPServer.compressed_types = SimpleHTTPServer.standard_compressed_types Or may be a method to turn on the "standard" set -- though if it really is simply a list, better to expose that so it's obvious that you can create your own list or examine and edit the existing one(s). Thanks for doing this -- nice feature! -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Tue, Jul 25, 2017 at 2:20 AM, Chris Barker <chris.barker@noaa.gov> wrote:
FWIW I'm quite okay with that. HTTP already has a mechanism for negotiating compression (Accept-Encoding), designed to be compatible with servers that don't support it. Any time a server gains support for something that clients already support, it's going to start happening as soon as you upgrade. Obviously this kind of change won't be happening in a bugfix release of Python, so it would be part of the regular checks when you upgrade from 3.6 to 3.7 - it'll be in the NEWS file and so on, so you read up on it before you upgrade. ChrisA

2017-08-05 12:49 GMT+02:00 Barry <barry@barrys-emacs.org>:
In the latest version of the Pull Request, only gzip is supported. But your comment makes me think that the code should probably be more modular so that subclasses of SimpleHTTPRequestHandler could handle other algorithms.
participants (5)
-
Barry
-
Chris Angelico
-
Chris Barker
-
Pierre Quentel
-
Terry Reedy