Classes that claim to be defined in __builtin__ but aren't
There's a fair number of classes that claim they are defined in __builtin__, but do not actually appear there. For example:
def qual(clazz): ... return clazz.__module__ + '.' + clazz.__name__ ... qual(types.GeneratorType) '__builtin__.generator' qual(types.FunctionType) '__builtin__.function' qual(types.MethodType) '__builtin__.instancemethod' qual(types.NoneType) '__builtin__.NoneType' qual(types.GeneratorType) '__builtin__.generator' __builtin__.generator AttributeError: 'module' object has no attribute 'generator' [[[etc.]]]
IMO classes ought to actually appear in __builtin__ if they claim they are defined there. Doing otherwise breaks reflection, as you have to add a special case for these class names to use the appropriate object from the types module instead. Thoughts? If it isn't desirable to have these names appear in __builtin__, perhaps their '__module__' should be changed to another module where they are defined? James
James Y Knight wrote:
There's a fair number of classes that claim they are defined in __builtin__, but do not actually appear there. For example:
def qual(clazz): ... return clazz.__module__ + '.' + clazz.__name__ ... qual(types.GeneratorType) '__builtin__.generator' qual(types.FunctionType) '__builtin__.function' qual(types.MethodType) '__builtin__.instancemethod' qual(types.NoneType) '__builtin__.NoneType' qual(types.GeneratorType) '__builtin__.generator' __builtin__.generator AttributeError: 'module' object has no attribute 'generator' [[[etc.]]]
IMO classes ought to actually appear in __builtin__ if they claim they are defined there. Doing otherwise breaks reflection, as you have to add a special case for these class names to use the appropriate object from the types module instead. Thoughts?
I agree.
If it isn't desirable to have these names appear in __builtin__, perhaps their '__module__' should be changed to another module where they are defined?
+1 Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
On Aug 9, 2004, at 6:16 PM, Jim Fulton wrote:
James Y Knight wrote:
IMO classes ought to actually appear in __builtin__ if they claim they are defined there. Doing otherwise breaks reflection, as you have to add a special case for these class names to use the appropriate object from the types module instead. Thoughts?
I agree.
I think this should stand for functions as well. help() is a good example. It's a bit confusing for newbies that help.__module__ is 'site', but 'site' isn't in their local namespace. -- Nick
Nick Bastin wrote:
On Aug 9, 2004, at 6:16 PM, Jim Fulton wrote:
James Y Knight wrote:
IMO classes ought to actually appear in __builtin__ if they claim they are defined there. Doing otherwise breaks reflection, as you have to add a special case for these class names to use the appropriate object from the types module instead. Thoughts?
I agree.
I think this should stand for functions as well. help() is a good example. It's a bit confusing for newbies that help.__module__ is 'site', but 'site' isn't in their local namespace.
Yup. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
James Y Knight <foom@fuhm.net> writes:
There's a fair number of classes that claim they are defined in __builtin__, but do not actually appear there. For example:
"__builtin__" is the "I don't know" answer in type_module for non-HEAPTYPEs. I'm certainly not sure that's totally wise...
IMO classes ought to actually appear in __builtin__ if they claim they are defined there. Doing otherwise breaks reflection, as you have to add a special case for these class names to use the appropriate object from the types module instead. Thoughts? If it isn't desirable to have these names appear in __builtin__, perhaps their '__module__' should be changed to another module where they are defined?
Such as? There really isn't a module where e.g. GeneratorType is defined. Cheers, mwh -- 3. Syntactic sugar causes cancer of the semicolon. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html
Michael Hudson wrote:
James Y Knight <foom@fuhm.net> writes:
IMO classes ought to actually appear in __builtin__ if they claim they are defined there. Doing otherwise breaks reflection, as you have to add a special case for these class names to use the appropriate object from the types module instead. Thoughts? If it isn't desirable to have these names appear in __builtin__, perhaps their '__module__' should be changed to another module where they are defined?
Such as? There really isn't a module where e.g. GeneratorType is defined.
Seems perfectly reasonable and useful to add GeneratorType and others to the types module. I have code, for example, like this, in a couple places: def _getACell(o): x = o def y(): y = x return [o for o in gc.get_referrers(x) if type(o).__name__ == 'cell'][0] def _f(): types.CellType = type(_getACell(object())) _f() Jp
Jp Calderone <exarkun@divmod.com> writes:
Michael Hudson wrote:
James Y Knight <foom@fuhm.net> writes:
IMO classes ought to actually appear in __builtin__ if they claim they are defined there. Doing otherwise breaks reflection, as you have to add a special case for these class names to use the appropriate object from the types module instead. Thoughts? If it isn't desirable to have these names appear in __builtin__, perhaps their '__module__' should be changed to another module where they are defined? Such as? There really isn't a module where e.g. GeneratorType is defined.
Seems perfectly reasonable and useful to add GeneratorType and others to the types module. I have code, for example, like this, in a couple places:
Well, it's already there, but types.GeneratorType.__name__ is 'generator'... it could be changed to 'GeneratorType', I guess. Cheers, mwh -- . <- the point your article -> . |------------------------- a long way ------------------------| -- Cristophe Rhodes, ucam.chat
Well, it's already there, but types.GeneratorType.__name__ is 'generator'... it could be changed to 'GeneratorType', I guess.
No, no, no! The "new" convention for built-in type names is all lowercase. CamelCaseType is old and only used in the context of types.py. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Aug 10, 2004, at 11:01 AM, Guido van Rossum wrote:
No, no, no! The "new" convention for built-in type names is all lowercase. CamelCaseType is old and only used in the context of types.py.
Sooo should (for 'generator' in objects that claim to be in __builtins__ but aren't), 1) 'generator' be added to __builtins__ 2) 'generator' be added to types.py and its __module__ be set to 'types' 3) 'generator' be added to <newmodule>.py and its __module__ be set to '<newmodule>' (and a name for the module chosen) James
Sooo should (for 'generator' in objects that claim to be in __builtins__ but aren't), 1) 'generator' be added to __builtins__ 2) 'generator' be added to types.py and its __module__ be set to 'types' 3) 'generator' be added to <newmodule>.py and its __module__ be set to '<newmodule>' (and a name for the module chosen)
I guess (1). Please submit a patch to SF... --Guido van Rossum (home page: http://www.python.org/~guido/)
On Aug 10, 2004, at 8:02 PM, Guido van Rossum wrote:
Sooo should (for 'generator' in objects that claim to be in __builtins__ but aren't), 1) 'generator' be added to __builtins__ 2) 'generator' be added to types.py and its __module__ be set to 'types' 3) 'generator' be added to <newmodule>.py and its __module__ be set to '<newmodule>' (and a name for the module chosen)
I guess (1).
Please submit a patch to SF...
Okay, so, I don't have a patch, but I have made a script to examine the situation. It seems the problem is a little more widespread than I had realized. I grepped through the python C source to find all the type objects, and then verified their existence. Just looking at Objects/*.c, there are 31 classes without appropriate bindings. Additionally, some aren't even valid identifier names. It seems like there's 4 categories here: iterators, descrobject stuffs, list comparison wrappers, and other. I'm not sure it's a good idea to add all these names to the builtins -- perhaps only the ones in the 'other' group? For {Modules,Mac,PC,RISCOS,Python}/*.c, most seem like they could relatively straightforwardly be added to their module, but I haven't examined them too closely. Some likely ought to have constructors added (e.g. the dl.dl('file') could be equivalent to dl.open('file')). __builtin__.dictionary-keyiterator not found (./Objects/dictobject.c) __builtin__.dictionary-valueiterator not found (./Objects/dictobject.c) __builtin__.dictionary-itemiterator not found (./Objects/dictobject.c) __builtin__.tupleiterator not found (./Objects/tupleobject.c) __builtin__.rangeiterator not found (./Objects/rangeobject.c) __builtin__.iterator not found (./Objects/iterobject.c) __builtin__.callable-iterator not found (./Objects/iterobject.c) __builtin__.listiterator not found (./Objects/listobject.c) __builtin__.listreverseiterator not found (./Objects/listobject.c) __builtin__.method_descriptor not found (./Objects/descrobject.c) __builtin__.classmethod_descriptor not found (./Objects/descrobject.c) __builtin__.member_descriptor not found (./Objects/descrobject.c) __builtin__.getset_descriptor not found (./Objects/descrobject.c) __builtin__.wrapper_descriptor not found (./Objects/descrobject.c) __builtin__.method-wrapper not found (./Objects/descrobject.c) __builtin__.sortwrapper not found (./Objects/listobject.c) __builtin__.cmpwrapper not found (./Objects/listobject.c) __builtin__.ellipsis not found (./Objects/sliceobject.c) types.EllipsisType __builtin__.builtin_function_or_method not found (./Objects/methodobject.c) types.BuiltinFunctionType types.BuiltinMethodType __builtin__.dictproxy not found (./Objects/descrobject.c) types.DictProxyType __builtin__.generator not found (./Objects/genobject.c) types.GeneratorType __builtin__.PyCObject not found (./Objects/cobject.c) __builtin__.classobj not found (./Objects/classobject.c) types.ClassType __builtin__.instance not found (./Objects/classobject.c) types.InstanceType __builtin__.instancemethod not found (./Objects/classobject.c) types.MethodType types.UnboundMethodType __builtin__.cell not found (./Objects/cellobject.c) __builtin__.NoneType not found (./Objects/object.c) types.NoneType __builtin__.NotImplementedType not found (./Objects/object.c) types.NotImplementedType __builtin__.frame not found (./Objects/frameobject.c) types.FrameType __builtin__.function not found (./Objects/funcobject.c) types.FunctionType types.LambdaType __builtin__.module not found (./Objects/moduleobject.c) types.ModuleType (the Mac ones I checked on my mac w/ python 2.3.0) _Qt.IdleManager not found (./Mac/Modules/qt/_Qtmodule.c) _Qt.SGOutput not found (./Mac/Modules/qt/_Qtmodule.c) module _OSA not installed (./Mac/Modules/osa/_OSAmodule.c) Nav.NavReplyRecord not found (./Mac/Modules/Nav.c) _Scrap.Scrap not found (./Mac/Modules/scrap/_Scrapmodule.c) module waste not installed (./Mac/Modules/waste/wastemodule.c) MacOS.ResourceFork not found (./Mac/Modules/macosmodule.c) icglue.ic_instance not found (./Mac/Modules/icgluemodule.c) __builtin__.PyHKEY not found (./PC/_winreg.c) __builtin__.drawf not found (./RISCOS/Modules/drawfmodule.c) __builtin__.block not found (./RISCOS/Modules/swimodule.c) __builtin__.traceback not found (./Python/traceback.c) types.TracebackType __builtin__.code not found (./Python/compile.c) types.CodeType __builtin__.symtable entry not found (./Python/symtable.c) __builtin__.tktimertoken not found (./Modules/_tkinter.c) __builtin__.tkapp not found (./Modules/_tkinter.c) __builtin__.arrayiterator not found (./Modules/arraymodule.c) _curses_panel.curses panel not found (./Modules/_curses_panel.c) linuxaudiodev.linux_audio_device not found (./Modules/linuxaudiodev.c) module fl not installed (./Modules/flmodule.c) __builtin__.DB not found (./Modules/_bsddb.c) __builtin__.DBCursor not found (./Modules/_bsddb.c) __builtin__.DBEnv not found (./Modules/_bsddb.c) __builtin__.DBTxn not found (./Modules/_bsddb.c) __builtin__.DBLock not found (./Modules/_bsddb.c) sha.SHA not found (./Modules/shamodule.c) module sv not installed (./Modules/svmodule.c) itertools._grouper not found (./Modules/itertoolsmodule.c) itertools.tee_dataobject not found (./Modules/itertoolsmodule.c) rotor.rotor not found (./Modules/rotormodule.c) module cl not installed (./Modules/clmodule.c) _sre.SRE_Pattern not found (./Modules/_sre.c) _sre.SRE_Match not found (./Modules/_sre.c) _sre.SRE_Scanner not found (./Modules/_sre.c) socket.SSL not found (./Modules/_ssl.c) _curses.curses window not found (./Modules/_cursesmodule.c) parser.st not found (./Modules/parsermodule.c) cStringIO.StringO not found (./Modules/cStringIO.c) cStringIO.StringI not found (./Modules/cStringIO.c) module sunaudiodev not installed (./Modules/sunaudiodev.c) module dbm not installed (./Modules/dbmmodule.c) dl.dl not found (./Modules/dlmodule.c) module fm not installed (./Modules/fmmodule.c) regex.regex not found (./Modules/regexmodule.c) pyexpat.xmlparser not found (./Modules/pyexpat.c) __builtin__.MultibyteCodec not found (./Modules/cjkcodecs/multibytecodec.c) __builtin__.MultibyteStreamReader not found (./Modules/cjkcodecs/multibytecodec.c) __builtin__.MultibyteStreamWriter not found (./Modules/cjkcodecs/multibytecodec.c) bsddb.bsddb not found (./Modules/bsddbmodule.c) module cd not installed (./Modules/cdmodule.c) cPickle.Pdata not found (./Modules/cPickle.c) module al not installed (./Modules/almodule.c) __builtin__.deque_iterator not found (./Modules/collectionsmodule.c) __builtin__.deque_reverse_iterator not found (./Modules/collectionsmodule.c) thread.lock not found (./Modules/threadmodule.c) zlib.Compress not found (./Modules/zlibmodule.c) zlib.Decompress not found (./Modules/zlibmodule.c) gdbm.gdbm not found (./Modules/gdbmmodule.c) ossaudiodev.oss_audio_device not found (./Modules/ossaudiodev.c) ossaudiodev.oss_mixer_device not found (./Modules/ossaudiodev.c)
On Aug 12, 2004, at 5:28 PM, James Y Knight wrote:
Okay, so, I don't have a patch
Submitted, http://www.python.org/sf/1009811 I'll note here, too, that I didn't add all the missing core types, only the ones that weren't iterators, descriptor internal stuff, or list comparison wrappers. For any of those that are at all possible to access from python code, I think they ought to show up somewhere, but I'm not convinced __builtin__ is the right place. Also the ones with "-" in their name need to be renamed.
I have made a script to examine the situation.
Improved it slightly, and found another set of problems. Some modules have a function bound in place of the type: E.g.
select.poll() <select.poll object at 0x401cdea8> isinstance(select.poll(), select.poll) TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types select.poll <built-in function poll> "D'oh!"
Here's the list of these: select.poll not a type. mpz.mpz not a type. itertools.tee not a type. _csv.reader not a type. _csv.writer not a type. md5.md5 not a type. cPickle.Pickler not a type. cPickle.Unpickler not a type. mmap.mmap not a type. xreadlines.xreadlines not a type. James
Guido van Rossum wrote:
Sooo should (for 'generator' in objects that claim to be in __builtins__ but aren't), 1) 'generator' be added to __builtins__ 2) 'generator' be added to types.py and its __module__ be set to 'types' 3) 'generator' be added to <newmodule>.py and its __module__ be set to '<newmodule>' (and a name for the module chosen)
I guess (1).
The problem I see with explicitly adding this stuff to __builtins__ is that tab-completion in the interpreter is suddenly going to have all of this extra stuff that people are not going want to really see that often. The other point I would like to make is that almost everything in __builtins__ is either an exception, a singleton (thinking of True and False), or a function (even thinking of list and dict since you can use their factory methods). The only exceptions I can think of are __name__ and that is just part of the design. Throwing in generator and any of the other types that are defined by the built-in types will go against all of this unless we create factory methods for them which is not really desired since they are returned only in certain situations. I personally prefer option 2. -Brett
Guido van Rossum wrote:
Sooo should (for 'generator' in objects that claim to be in __builtins__ but aren't), 1) 'generator' be added to __builtins__ 2) 'generator' be added to types.py and its __module__ be set to 'types' 3) 'generator' be added to <newmodule>.py and its __module__ be set to '<newmodule>' (and a name for the module chosen)
I guess (1).
[Note: it should be added to __builtin__, not __builtins__ -- the two are related but __builtin__ is the module and __builtins__ is a secret attribute that references the module or its __dict__ for purposes of sandboxing. [Brett C]
The problem I see with explicitly adding this stuff to __builtins__ is that tab-completion in the interpreter is suddenly going to have all of this extra stuff that people are not going want to really see that often.
There's already lots of stuff there that's rarely used.
The other point I would like to make is that almost everything in __builtins__ is either an exception, a singleton (thinking of True and False), or a function (even thinking of list and dict since you can use their factory methods).
But list and dict are *not* functions, they are types, and they can be used for subclassing and type checking too. These are the precedent for placing generator etc. in __builtin__.
The only exceptions I can think of are __name__ and that is just part of the design. Throwing in generator and any of the other types that are defined by the built-in types will go against all of this unless we create factory methods for them which is not really desired since they are returned only in certain situations.
They should not be made factory functions if they aren't already.
I personally prefer option 2.
Bah. The types module shouldn't be the "home" for any types -- it's just a collection of convenient aliases, and mostly they have the wrong names. It should be deprecated. Also note that at least 6 of these "anonymous" built-in types currently have another alias in the 'new' module, which means that they *are* factories already. It would be nice if that module could finally be deprecated. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote: [SNIP]
Bah. The types module shouldn't be the "home" for any types -- it's just a collection of convenient aliases, and mostly they have the wrong names. It should be deprecated.
Well that settles that one. Didn't know you wanted to deprecate 'types'. Then __builtin__ does seem to be the only logical place. -Brett
Guido:
But list and dict are *not* functions, they are types, and they can be used for subclassing and type checking too. These are the precedent for placing generator etc. in __builtin__.
The real question seems to be whether __builtin__ should contain *all* built-in types, including internal ones, or only those intended for public use. Do you have an opinion about that, Guido? I suppose it's reasonable to put them all in __builtin__, since as you say, they can be useful for type checking even if you can't or rarely want to instantiate them yourself. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
The real question seems to be whether __builtin__ should contain *all* built-in types, including internal ones, or only those intended for public use. Do you have an opinion about that, Guido?
I suppose it's reasonable to put them all in __builtin__, since as you say, they can be useful for type checking even if you can't or rarely want to instantiate them yourself.
That's where I stand, at about +0.5. I don't think it's super important that all builtins be particularly well-known or useful -- it's easy enough to have a section in the documentation for "lesser-known builtins". --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (8)
-
Brett C. -
Greg Ewing -
Guido van Rossum -
James Y Knight -
Jim Fulton -
Jp Calderone -
Michael Hudson -
Nick Bastin