[Python-Dev] PEP 0424: A method for exposing a length hint
alexandre.zani at gmail.com
Sun Jul 15 17:38:20 CEST 2012
On Sun, Jul 15, 2012 at 8:08 AM, Mark Shannon <mark at hotpy.org> wrote:
> Brett Cannon wrote:
>> On Sun, Jul 15, 2012 at 10:39 AM, Mark Shannon <mark at hotpy.org
>> <mailto:mark at hotpy.org>> wrote:
>> Nick Coghlan wrote:
>> Right, I agree on the value in being able to return something to
>> say "this cannot be converted to a concrete container".
>> I still haven't seen a use case where the appropriate response
>> to "I don't know" differs from the appropriate response to a
>> hint of zero - that is, you don't preallocate, you just start
>> There seem to be 5 possible classes values of __length_hint__ that an
>> iterator object can provide:
>> 1. Don't implement it at all.
>> 2. Implement __length_hint__() but don't want to return any value.
>> Either raise an exception (TypeError) -- As suggested in the PEP.
>> or return NotImplemented -- my preferred option.
>> 3. Return a "don't know" value:
>> Returning 0 would be fine for this, but the VM might want to
>> differently to "don't know" and 0.
>> __length_hint__() == 0 container should be
>> minimum size.
>> __length_hint__() == "unknown" container starts at
>> default size.
>> 4. Infinite iterator:
>> Could return float('inf'), but given this is a "hint" then
>> returning sys.maxsize or sys.maxsize + 1 might be OK.
>> Alternatively raise an OverflowError
>> I am really having a hard time differentiating infinity with "I don't
>> know" since they are both accurate from the point of view of __length_hint__
>> and its typical purpose of allocation. You have no clue how many values will
>> be grabbed from an infinite iterator, so it's the same as just not knowing
>> upfront how long the iterator will be, infinite or not, and thus not worth
>> 5. A meaningful length. No problem :)
>> Also, what are the allowable return types?
>> 1. int only
>> 2. Any number (ie any type with a __int__() method)?
>> 3. Or any integer-like object (ie a type with a __index__() method)?
>> My suggestion:
>> a) Don't want to return any value or "don't know": return
>> b) For infinite iterators: raise an OverflowError
>> c) All other cases: return an int or a type with a __index__() method.
>> I'm fine with (a), drop (b), and for (c) use what we allow for __len__()
>> since, as Nick's operator.length_hint pseudo-code suggests, people will call
>> this as a fallback if __len__ isn't defined.
> So how does an iterator express infinite length?
> What should happen if I am silly enough to do this:
> This will fail; it should fail quickly.
> Python-Dev mailing list
> Python-Dev at python.org
The PEP so far says: "It may raise a ``TypeError`` if a specific
instance cannot have
its length estimated." In many ways, "I don't know" is the same as
this "specific instance cannot have its length estimated". Why not
just raise a TypeError?
Also, regarding the code Nick posted above, I'm a little concerned
about calling len as the first thing to try. That means that if I
implement both __len__ and __len_hint__ (perhaps because __len__ is
very expensive) __len_hint__ will never be used. It's relatively easy
hint = len_hint(l)
hint = len(l)
More information about the Python-Dev