[Patches] [ python-Patches-643711 ] refactoring and documenting ModuleFinder
noreply@sourceforge.net
noreply@sourceforge.net
Thu, 28 Nov 2002 12:28:16 -0800
Patches item #643711, was opened at 2002-11-25 19:52
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=643711&group_id=5470
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Thomas Heller (theller)
Assigned to: Nobody/Anonymous (nobody)
Summary: refactoring and documenting ModuleFinder
Initial Comment:
This is not (yet) a complete patch, it is thought as a
workspace for improvements to
tools/freeze/moduulefinder, so that it can eventually go
into the standard library.
----------------------------------------------------------------------
>Comment By: Thomas Heller (theller)
Date: 2002-11-28 21:28
Message:
Logged In: YES
user_id=11105
Just, I trust your analysis while I don't understand it ;-).
Thinking more about my "ignore" concept shows that it is
unneeded. Warnings can be better removed after
modulefinder has been run by examining the data it collects.
To answer your question: "excludes" does not mean 'may be
missed', it means 'must be excluded' IMO, and no, this was
not the purpose of my patch.
----------------------------------------------------------------------
Comment By: Just van Rossum (jvr)
Date: 2002-11-28 21:17
Message:
Logged In: YES
user_id=92689
Thomas, your _add_badmodule patch actually broke my code...
It took me a while to figure out why, and while at it, I
discovered a blatant stupidity in my own patch: somehow I
completely overlooked the fact that mf.badmodules already
references the modules that did the failed import, so my
module.badimports thing can go. It does the exact same
thing. Almost... and your patch made that clear: the lines
that you patched actually make the badmodules reference lie
(and with your patch it also makes mod.badimports lie ;-):
if module X imports Y.Z, and Z isn't found, it lists Y as an
importer of Y.Z, which isn't true. My any_maybe_missing()
code needs to know whether the __init__.py tried to failed
import _itself_, because if it did (and failed) it's certain
that the submodule is missing. But if it _didn't_, the
"missing submodule" should be listed as "maybe missing", as
it's most likely just a global in the __init__.py. I'll try
to revise my patch tomorrow, it should become simpler.
In the meantime I think it might indeed be useful to
distinguish between "may be missed" and "must be excluded".
Is that what your "ignore" patch tried to do?
----------------------------------------------------------------------
Comment By: Just van Rossum (jvr)
Date: 2002-11-28 19:22
Message:
Logged In: YES
user_id=92689
What's the difference between this "ignore" variable and the
"exclude" list? And why do you need to ignore modules based
on importing module? In my OSX appbuilder stuff I simply
have a list of modules that will be removed from the
'missing' list when reporting:
MAYMISS_MODULES = ['mac', 'os2', 'nt', 'ntpath', 'dos',
'dospath',
'win32api', 'ce', '_winreg', 'nturl2path', 'sitecustomize',
'org.python.core', 'riscos', 'riscosenviron', 'riscospath'
]
(I think I could just add this to the exclude list and get
the same effect.)
Let's please keep it simple!
----------------------------------------------------------------------
Comment By: Thomas Heller (theller)
Date: 2002-11-28 13:04
Message:
Logged In: YES
user_id=11105
Now that it works really fine, I would like to make progress.
This new patch 'thomas1.patch' is against the patched
modulefinder, it is easier to read this way.
First it passes all changes to self.badmodules through the
_add_badmodule() method.
Then, and this part is not really polished, but you may get the
idea, it introduces a new ignored instance variable which is
thought to suppress most of the remaining missing modules
warnings.
----------------------------------------------------------------------
Comment By: Just van Rossum (jvr)
Date: 2002-11-27 12:02
Message:
Logged In: YES
user_id=92689
Whoops, that was a useless patch :-( I managed to forgot to
copy the last three lines of the original scan_code() which
takes care of delayed imports... Fixed in the new patch.
I now also patched Modulefinder.report() to actually use the
new any_missing_maybe() feature.
(Note for the lib doco to be written: we should document
that people should use any_missing[_maybe]() instead of
looking directly at mf.badmodules. mf.badmodules is the raw
list, any_missing_maybe() filters out the false positives.)
I've ran before/after tests on all of Lib/*.py and
Lib/test_*.py and the only differences I found are those
that this patch address, ie. less false positives when
reporting missing modules.
----------------------------------------------------------------------
Comment By: Thomas Heller (theller)
Date: 2002-11-27 09:27
Message:
Logged In: YES
user_id=11105
Running the current CVS ModuleFinder on a simple script
which contains 'import os' with Python 2.2 on Win2k finds 27
modules, while with this patch it only finds 13.
(and unassign Mark)
----------------------------------------------------------------------
Comment By: Just van Rossum (jvr)
Date: 2002-11-27 02:30
Message:
Logged In: YES
user_id=92689
Oh, I just realized ModuleFinder.report() should still be
patched to use any_missing(), or even better,
any_missing_maybe(). Will do if this gets through.
----------------------------------------------------------------------
Comment By: Just van Rossum (jvr)
Date: 2002-11-27 02:26
Message:
Logged In: YES
user_id=92689
Here's a patch. Summary: any_missing() returns less bogus
modules.
- I've rewritten scan_code() more or less from scratch,
factored bits and pieces out for readability.
- keep track of global assignments and failed imports per
module; use this to determine whether the Y in "from X
import Y" is a submodule or just a global name. This is not
100% doable: you can't tell which symbols are imported when
doing a star import of a non-Python module short of actually
importing it.
- added a new method to ModuleFinder: any_missing_maybe(),
which returns *two* lists, one with certain misses, one with
possible misses. The possible misses are *very* often false
alarms, so it's useful to keep this list separate.
any_misses() now simply returns the union of
any_missing_maybe().
I pretty sure this is all 100% backward compatible, so after
some more testing it could be checked in.
----------------------------------------------------------------------
Comment By: Thomas Heller (theller)
Date: 2002-11-26 09:11
Message:
Logged In: YES
user_id=11105
I've removed in current CVS the support for modules listed in
the registry.
----------------------------------------------------------------------
Comment By: Thomas Heller (theller)
Date: 2002-11-26 08:57
Message:
Logged In: YES
user_id=11105
Mark writes in private email: "Modules listed in the registry
was a dumb idea. This whole scheme can die.
AFAIK, no one in the world uses it (including win32all since
the last build)."
----------------------------------------------------------------------
Comment By: Thomas Heller (theller)
Date: 2002-11-25 20:59
Message:
Logged In: YES
user_id=11105
Assign to Mark, hoping for an answer.
----------------------------------------------------------------------
Comment By: Thomas Heller (theller)
Date: 2002-11-25 19:58
Message:
Logged In: YES
user_id=11105
First question for anyone listening: isn't the windows
_try_registry stuff unneeded, because imp.find_module
handles this itself? Even in python15, imp.find_module
("pythoncom") returns 'c:\winnt\system32\pythoncom15.dll'
on my system.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=643711&group_id=5470