[New-bugs-announce] [issue5401] mimetypes.MAGIC_FUNCTION implementation clusterfuck

[Armin Ronacher]

New submission from Armin Ronacher <armin.ronacher at active-4.com>:

Sorry for the harsh words, but when I found that code I nearly freaked
out.  For all those years I was using "from mimetypes import guess_type"
until today I found out that this has horrendous performance problems
due to the fact that the mimetype database is re-parsed on each call.

The reason for this is that mimetypes.guess_type is implemented like this:

def guess_type(...):
    global guess_type
    init()
    guess_type = new_guess_type
    return guess_type(...)

Obviously if the function was imported from the module and not looked up
via standard attribute lookup before each call (by calling it like
mimetypes.guess_type(...)) init() would be called over and over again.

What's the performance impact?  In a small WSGI middleware that serves
static files the *total* performance impact (including HTTP header
parsing, file serving etc.) was 1000%.  Just for guess_type() versus
mimetypes.guess_type() which was called just once per request.

I attached a workaround for that problem that tries to avoid init()
calls after the thing was initialized.

If this is intended behaviour it should be documented but I doubt that
this is a good idea as people don't read documentation it stuff seems to
work.

And google tells me I'm not the first one who invoked guess_type that
way: http://google.com/codesearch?q="from+mimetypes+import+guess_type"

----------
components: Library (Lib)
files: mimetypes-speedup.diff
keywords: easy, needs review, patch, patch
messages: 82992
nosy: aronacher
priority: critical
severity: normal
stage: patch review
status: open
title: mimetypes.MAGIC_FUNCTION implementation clusterfuck
versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1
Added file: http://bugs.python.org/file13225/mimetypes-speedup.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue5401>
_______________________________________