[Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)

Cameron Simpson cs at zip.com.au
Wed Apr 4 04:46:56 CEST 2012

On 04Apr2012 01:45, Victor Stinner <victor.stinner at gmail.com> wrote:
| > | get_clock() returns None if no clock has the requested flags, whereas
| > | I expected an exception (LookupError or NotImplementError?).
| >
| > That is deliberate. People can easily write fallback like this:
| >
| >  clock = get_clock(T_MONOTONIC|T_HIRES) or get_clock(T_MONOTONIC)
| Why not passing a a list of set of flags? Example:
| # try to get a monotonic and steady clock,
| # or fallback to a steady clock,
| # or fallback to a monotonic clock,
| # or fallback to the system clock

That's interesting. Ethan Furman suggested multiple arguments to be
combined, whereas yours bundles multiple search criteria in one call.

While it uses a bitmask as mine does, this may get cumbersome if we went
with Nick's "use strings!" suggestion.

| haypo_perf_counter = get_clock(HIGHRES, MONOTONIC|STEADY, STEADY,
| # try to get a high-resolution clock
| # or fallback to a monotonic and steady clock,
| # or fallback to a steady clock,
| # or fallback to a monotonic clock,
| # or fallback to the system clock
| On Windows, haypo_steady should give GetTickCount (MONOTONIC|STEADY)
| and haypo_perf_counter should give QueryPerformanceCounter

Sounds ok to me. I am not familiar with the Windows counters and am
happy to take your word for it.

| Hum, I'm not sure that haypo_highres uses the same clocks than
| time.perf_counter() in the PEP.
| > If one wants an exception it is easy to follow up with:
| >  if not clock:
| >    raise RunTimeError("no suitable clocks on offer on this platform")
| And if don't read the doc carefuly and forget the test, you can a
| "NoneType object is not callable" error.

Excellent! An exception either way! Win win!

| > | get_clock() doesn't remember if a clock works or not (if it raises an
| > | OSError) and does not fallback to the next clock on error. See
| > | "pseudo-codes" in the PEP 418.
| >
| > I presume the available clocks are all deduced from the platform. Your
| > pseudo code checks for OSError at fetch-the-clock time. I expect that
| > to occur once when the module is loaded, purely to populate the table
| > of avaiable platform clocks.
| It's better to avoid unnecessary system calls at startup (when the
| time module is loaded), but you may defer the creation of the clock
| list, or at least of the flags of each clock.

Yes indeed. I think this should be deferred until use.

| > Note that you don't need to provide a clock list at all; get_clock(0
| > will use ALL_CLOCKS by default, and hires() and monotonic() should each
| > have their own default list.
| A list of clocks and a function are maybe redundant. Why not only
| providing a function?

Only because the function currently only returns one clock.
The picky user may want to peruse all the clocks inspecting other
metadata (precision etc) than the coarse flag requirements.

There should be a way to enumerate the available clock implementation;
in my other recent post I suggest either lists (as current), a
get_clocks() function, or a mode parameter to get_clock() such as
_all_clocks, defaulting to False.

| > Regarding the choice itself: as the _caller_ (not the library author),
| > you must decide what you want most. You're already planning offering
| > monotonic() and hires() calls without my proposal!
| My PEP starts with use cases: it proposes one clock per use case.
| There is no "If you need a monotonic, steady and high-resolution clock
| ..." use case.

Yes. but this is my exact objection to the "just provide hires() and
steady() and/or monotonic()" API; the discussion to date is littered
with "I can't imagine wanting to do X" style remarks. We should not be
trying to enumerate the user case space exhaustively. I'm entirely in
favour of your list of use cases and the approach of providing hires() et
al to cover the thought-common use cases. But I feel we really _must_
provide a way for the user with a not-thought-of use case to make an
arbitrary decision.

get_clock() provides a simple cut at the "gimme a suitable clock"
approach, with the lists or other "get me an enumeration of the
available clocks" mechanism for totally ad hoc perusal if the need

This is also my perhaps unstated concern with Guido's "the more I think about
it, the more I believe these functions should have very loose guarantees, and
instead just cater to common use cases -- availability of a timer with
minimal fuss is usually more important than the guarantees"

The easy to use hires() etc must make very loose guarentees or they will
be useless too often. That looseness is fine in some ways - it provides
availability on many platforms (all?) and discourages the user from
hoping for too much and thus writing fragile code. But it also PREVENTS
the user from obtaining a really good clock if it is available (where
"good" means their partiuclar weirdo feature requirements).

So I think there should be both - the easy and simple calls, and a
mecahnism for providing all clocks for the user to chose with arbitrary
criteria and fallback.

| The "highres" name was confusing, I just replaced it with
| time.perf_counter() (thanks Antoine for the name!).
| time.perf_counter() should be used for benchmarking and profiling.

I've been wondering; do we distinguish between clocks and counters. In my
mind a clock or timer has a very linear relationship with "real time",
the wall clock. A counter, by comparison, may measure CPU cycles or
kernel timeslice ticks or python opcode counts or any number of other
time-like resource consumption things.

I've been presuming we're concerned here with "clocks" and not counters.

| > Taking your query "Should
| > I use MONTONIC_CLOCKS or HIRES_CLOCKS when I would like a monotonic and
| > high-resolution clock" is _already_ a problem. Of course you must call
| > monotonic() or hires() first under the current scheme, and must answer this
| > question anyway. Do you prefer hires? Use it first! No preference? Then the
| > question does not matter.
| I mean having to choose the flags *and* the list of clocks is hard. I
| would prefer to only have to choose flags or only the list of clocks.
| The example was maybe not the best one.

Yah; I think I made a followup post where I realised you may have meant

The use of default arguments is meant to make it easy to use flags
and/or lists or even neither (which for get_clock() at least would
always get you a clock because a wrapper for time.time() is always
provided). In my mind, usually just flags of course.

| > | If you have only one list of clocks, how do sort the list to get
| > | QueryPerformanceCounter when the user asks for highres and
| > | GetTickCount when the user asks for monotonic?
| >
| > This is exactly why there are supposed to be different lists.
| > You have just argued against your objection above.
| You can solve this issue with only one list of clocks if you use the
| right set of flags.

No you can't, not in general. If there are multiple clocks honouring
those flags you only ever get the first one in the list. The point of the
MONOTONIC_CLOCKS list etc is that the lists may be differently ordered
to provide quality of clock within that domain. Suppose I ask for
steady_clock(MONOTONIC); I would probably prefer a more-steady clock
over a more-precise/hires clock. And converse desires when asking for
hires_clock(). So different list orders, at least in principle.

If it turns out empirically that this isn't the case then all the names
can in fact refer to the same list. But offering only one list _name_
prevents offering these nuances when other platforms/clocks come in the

| > | So we would have:
| > |
| > | GetTickCount.flags = T_MONOTONIC | T_STEADY | T_HIGHRES
| > |
| > | Even if GetTickCount has only an accuracy of 15 ms :-/
| >
| > T_HIGHRES is a quality call, surely? If 15ms is too sloppy for a "high
| > resolution, the is should _not_ have the T_HIRES flag.
| So what is the minimum resolution and/or accuracy of the HIGHRES flag?

No idea. But you must in principle have one in mind to offer the hires()
call at all in the PEP, or be prepared to merely offer the most hires
clock available regardless. In this latter case you would always mark
that clock as having the HIRES flag and the problem is solved. It would
be good to have metadata to indicate how hires a partiulcar clock is.

| > | Could you please update your code according to my remarks? I will try
| > | to integrate it into the PEP. A PEP should list all alternatives!
| >
| > Surely.
| >
| > The only updates I can see are to provide the flat interface
| > (instead of via clock-object indirection) and the missing hires_clock()
| > and monotonic_clock() convenience methods.
| A full implementation would help to decide which API is the best one.
| "Full" implementation:
|  - define all convinience function
|  - define all list of clocks

Ok. My current code is here, BTW:
(Finally found a revision independent URL on bitbucket.)

|  - define flags of all clocks listed in the PEP 418: clocks used in
| the pseudo-code of time.steady and time.perf_counter, and maybe also
| time.time

I'll make one. It will take a little while. Will post again when ready. At
present the code compiles and runs (albeit with no platform specific
clocks:-) This table may require fictitious code. Should still compile
I guess...

Cameron Simpson <cs at zip.com.au> DoD#743

The right to be heard does not include the right to be taken seriously.
        - Hubert Horatio Humphrey

More information about the Python-Dev mailing list