[New-bugs-announce] [issue31661] Issues with request rate in robotparser
report at bugs.python.org
Mon Oct 2 02:14:07 EDT 2017
New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:
There are issues in implementing support of request rate in robotparser.
req_rate = collections.namedtuple('req_rate',
entry.req_rate = req_rate
entry.req_rate.requests = int(numbers)
entry.req_rate.seconds = int(numbers)
First, a new namedtuple type is created for every entry. This is slow even with recent namedtuple optimizations, and is much slower in 3.6. This wastes a memory, since new type is created for every entry. This is definitely wrong, since req_rate is set to a namedtuple type instead of namedtuple instance. And there is a question why a namedtuple is used here at all. Other classes in this module are not namedtuples.
components: Library (Lib)
nosy: XapaJIaMnu, berker.peksag, serhiy.storchaka
title: Issues with request rate in robotparser
versions: Python 3.6, Python 3.7
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce