On 11/27/2015 03:39 PM, Phil Mayers wrote:
On 27/11/15 14:05, Jonathan Ballet wrote:
- how many tries did it require * if there were several tries, the timing of each ones
Typically, application-layer code doesn't retry a DNS lookup; rather the c or other runtime will handle this, for example getaddrinfo() in glibc, according to settings read from /etc/resolv.conf or compiled-in defaults.
So it depends on whether you want to emulate "typical" application code, a specific application stack that may or may not do it's own resolution (e.g. modern browsers) or something else.
That's a fair point, and I would like, as best as it can, to be as close as possible as a "typical" application; the goal really would be to measure the network conditions the applications are facing. Although I understand I won't be able to get the retries number if the underlying code is using getaddrinfo() or something like this, I was thinking that twisted.names was maybe offering a "hand-made" resolver, which was producing the UDP packets itself and offered a way to plug some code there to measure these retries; I haven't checked yet. But in any case, I guess it's going a little bit against my previous point which was to try to measure things "as used by a 'typical' application" (which isn't going to use twisted.names custom resolver.) (Actually, I was surprised to discover that, although it can be configured from the command-line, `dig` doesn't report the number of tries/retries it makes, you can only deduce it by looking at the overall command execution time.)
* how long does it take to get the first bytes of the endpoint - how long does it take to complete the TCP connection handshake - the status of the packets exchanged (how many retries, how many packets lost, etc.)
Some of this is available in platform-specific APIs e.g. SIOCGSTAMP and TCP_INFO socket options available on Linux.
In general, any timings you make based on return of control from kernel will include error relating to system/scheduling issues. If you're concerned about getting raw, on-the-wire timings, this is extremely difficult without being in-kernel, and even then various issues - TCP offload for example - can end up hiding data from you.
I will have a look at these options, as I will be running my tests under Linux anyway. It's probably out of scope of Twisted anyway, but I could also retrieve the packets sent on the wire by listening on the related network interface set in promiscuous mode and correlate packets together. It's ... "slightly" more work though...
How far can I do this kind of things with Twisted? I know I can somewhat easily get the timings of the name resolution, the TCP connection handshake also and the time to first byte(s), but what about the packets? I haven't look at the code of Twisted Names yet, but if it's doing the DNS request by itself, I may be able to plug-in somewhere and have my request counter and the timers associated, but I'm not sure if the underlying details of the TCP protocol are exposed to the upper layer such as Twisted?
Only via platform-specific options.
To do this kind of thing "reliably", you'd need to reimplement TCP in user-space.
But the info above may be a helpful start.
Thanks for your answer Phil, I'll see what I can come up with! Jonathan