[New-bugs-announce] [issue43332] def _tunnel(self): - uses multiple network writes, possibly causing unnecessary implementation complexity on the receiving end
report at bugs.python.org
Fri Feb 26 15:18:00 EST 2021
New submission from Zveinn <sveinn at zkynet.io>:
Hey, some time ago I ran into some code in the cpython code I thought might be possible to improve it a little bit.
This code specifically.
Notice how the self.send() method is used multiple time to construct the CONNECT request. When the network load is high, these different parts actually get split into separate network frames.
This causes additional meta data to be sent, effectively costing more bandwidth and causing the request to be split into multiple network frames.
This has some interesting behavior on the receiving end as well.
If you send everything as a single network frame, then the receiver can read the entire thing in a single read call. If you send multiple frames, the main reader pipe now needs a temporary buffer to encapsulate the multiple calls.
Because of this, sending requests as many network frames actually causes a rise in processing complexity on the receiving end.
Here is a github issue I made about this problem some time ago: https://github.com/psf/requests/issues/5384
In this issue you will find detailed information and screenshots.
My recommendation would be to construct the query as a whole before using a single self.send() to send the whole payload in one network frame. Even if we ignore the added complexity on the receivers end, the gain in network performance is worth it.
components: C API
title: def _tunnel(self): - uses multiple network writes, possibly causing unnecessary implementation complexity on the receiving end
versions: Python 3.10
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce