[Python-Dev] Mailbox module - timings and functionality changes

[Guido van Rossum]

On Tue, Jun 29, 2010 at 7:49 AM, Steve Holden <steve at holdenweb.com> wrote:
> Steve Holden wrote:
>> Nick Coghlan wrote:
>>> Command line: ./python -m test.regrtest -v test_mailbox
>>>
>>> trunk: Ran 274 tests in 25.239s
>>> py3k: Ran 268 tests in 26.263s
>>>
>>> So I don't see any substantial difference on a Kubuntu 10.04 box (both
>>> builds are recent'ish, but not completely up to date).
>>>
>>> However, the underlying IO access is significantly different between
>>> POSIX and Windows, so there could still be something pathological
>>> happening at the filesystem manipulation layer. My comparisons are
>>> also 2.7 vs 3.2 rather than 2.6 vs 3.1.
>>>
>>> Cheers,
>>> Nick.
>>>
>> Thanks for all the timings! If a Windows user could do the same thing
>> that would help ...
>>
> And there is *definitely a performance issue. I created a Thunderbird
> folder of 26 Google alerts and just parsed then all after reading them
> in from the mailbox.
>
> 2.5 (!):  0.78 sec
> 3.1    : 42.80 sec
>
> Rather than debate the code here perhaps I should just open an issue for
> this? I can then provide both a program and some data, which can be
> added to the tests if appropriate. The issue can clearly stand some
> investigation.

Since you have such a great reproducible test case, could you point
the profiler at it? (Perhaps on a reduced dataset... The profiler
multiples your run time by some number between 2 and 10 IIRC.)

-- 
--Guido van Rossum (python.org/~guido)