cool. first time i build the entire interpreter, 'twas fun :) currently i "retained" support for __members__ and __methods__, so it doesn't break anything and is compatible with 2.6. i really hope it will be included in 2.6 as today i'm using ugly hacks in RPyC to make remote objects appear like local ones. having __dir__ solves all of my problems. besides, it makes a lot of sense of define __dir__ for classes that define __getattr__. i don't think it should be pushed back to py3k. here's the patch: http://sourceforge.net/tracker/index.php?func=detail&aid=1591665&group_id=5470&atid=305470 here's a demo:

...

...

class foo(object): ... def __dir__(self): ... return ["kan", "ga", "roo"] ... f = foo() f <__main__.foo object at 0x00A90C78> dir() ['__builtins__', '__doc__', '__name__', 'f', 'foo'] dir(f) ['ga', 'kan', 'roo'] dir(foo) ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__getattribute__ ', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__ ', '__repr__', '__setattr__', '__str__', '__weakref__'] class bar(object): ... __members__ = ["bow", "wow"] ... b=bar() dir(b) ['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash_ _', '__init__', '__members__', '__module__', '__new__', '__reduce__', '__reduce_ ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 'bow', 'wow']

-tomer On 11/6/06, Guido van Rossum wrote:

Sounds like a good plan, though I'm not sure if it's worth doing in 2.6 -- I'd be happy with doing this just in 3k.

I'm not sure what you mean by "adding a method slot" -- certainly it's possible to define a method __foo__ and call it directly without having a special tp_foo in the type object, and I recommend doing it that way since the tp_foo slots are just there to make things fast; in this case I don't see a need for dir() to be fast.

--Guido

On 11/6/06, tomer filiba wrote:

...

so, if you remember, i suggested adding __dir__ to objects, so as to make dir() customizable, remove the deprecated __methods__ and __members__, and make it symmetrical to other built-in functions.

you can see the original post here: http://mail.python.org/pipermail/python-dev/2006-July/067095.html which was generally accepted by the forum: http://mail.python.org/pipermail/python-dev/2006-July/067139.html

so i went on, now that i have some spare time, to research the issue. the current dir() works as follows: (*) builtin_dir calls PyObject_Dir to do the trick (*) if the object is NULL (dir with no argument), return the frame's locals (*) if the object is a *module*, we're just using it's __dict__ (*) if the object is a *type*, we're using it's __dict__ and __bases__, but not __class__ (so as not to show the metaclass) (*) otherwise, it's a "normal object", so we take it's __dict__, along with __methods__, __members__, and dir(__class__) (*) create a list of keys from the dict, sort, return

we'll have to change that if we were to introduce __dir__. my design is: (*) builtin_dir, if called without an argument, returns the frame's locals (*) otherwise, it calls PyObject_Dir(self), which would dispatch self.__dir__() (*) if `self` doesn't have __dir__, default to object.__dir__(self) (*) the default object.__dir__ implementation would do the same as today: collect __dict__, __members__, __methods__, and dir(__class__). by py3k, we'll remove looking into __methods__ and __members__. (*) type objects and module objects would implement __dir__ to their liking (as PyObject_Dir does today) (*) builtin_dir would take care of sorting the list returned by PyObject_Dir

so first i'd want you people to react on my design, maybe you'd find flaws whatever. also, should this become a PEP?

and last, how do i add a new method slot? does it mean i need to change all type-object definitions throughout the codebase? do i add it to some protocol? or directly to the "object protocol"?

-tomer _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

No objection on targetting 2.6 if other developers agree. Seems this is well under way. good work! On 11/6/06, Nick Coghlan wrote:

tomer filiba wrote:

...

cool. first time i build the entire interpreter, 'twas fun :) currently i "retained" support for __members__ and __methods__, so it doesn't break anything and is compatible with 2.6.

i really hope it will be included in 2.6 as today i'm using ugly hacks in RPyC to make remote objects appear like local ones. having __dir__ solves all of my problems.

besides, it makes a lot of sense of define __dir__ for classes that define __getattr__. i don't think it should be pushed back to py3k.

here's the patch: http://sourceforge.net/tracker/index.php?func=detail&aid=1591665&group_id=5470&atid=305470

As I noted on the tracker, PyObject_Dir is a public C API function, so it's behaviour needs to be preserved as well as the behaviour of calling dir() from Python code.

So the final form of the patch will likely need to include stronger tests for that section of the API, as well as updating the documentation in various places (the dir and PyObject_Dir documentation, obviously, but also the list of magic methods in the language reference).

+1 on targeting 2.6, too.

Cheers, Nick.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:

No objection on targetting 2.6 if other developers agree. Seems this is well under way. good work!

given that dir() is used extensively by introspection tools, I'm not sure I'm positive to a __dir__ that *overrides* the standard dir() behaviour. *adding* to the default dir() list is okay, re- placing it is a lot more questionable. (what about vars(), btw?) </F>

Guido van Rossum wrote:

I think that ought to go into the guidlines for what's an acceptable __dir__ implementation. We don't try to stop people from overriding __add__ as subtraction either.

to me, overriding dir() is a lot more like overriding id() than over- riding "+". I don't think an object should be allowed to lie to the introspection mechanisms. </F>

Guido van Rossum wrote:

...

(what about vars(), btw?)

Interesting question! Right now vars() and dir() don't seem to use the same set of keys; e.g.:

...

...

...

class C: pass ... c = C() c.foo = 42 vars(c) {'foo': 42} dir(c) ['__doc__', '__module__', 'foo']

It makes some sense for vars(x) to return something like

dict((name, getattr(x, name)) for name in dir(x) if hasattr(x, name))

and for the following equivalence to hold between vars() and dir() without args:

dir() == sorted(vars().keys())

+1. This is easy and straightforward to explain, better than "With a module, class or class instance object as argument (or anything else that has a __dict__ attribute), returns a dictionary corresponding to the object's symbol table." Georg

Thomas Heller wrote:

...

...

No objection on targetting 2.6 if other developers agree. Seems this is well under way. good work!

given that dir() is used extensively by introspection tools, I'm not sure I'm positive to a __dir__ that *overrides* the standard dir() behaviour. *adding* to the default dir() list is okay, re- placing it is a lot more questionable.

One part that *I* would like about a complete overridable __dir__ implementation is that it would be nice to customize what help(something) prints.

I don't think you should confuse reliable introspection with the help system, though. introspection is used for a lot more than implementing help(). </F>

Fredrik Lundh wrote:

Guido van Rossum wrote:

...

No objection on targetting 2.6 if other developers agree. Seems this is well under way. good work!

given that dir() is used extensively by introspection tools, I'm not sure I'm positive to a __dir__ that *overrides* the standard dir() behaviour. *adding* to the default dir() list is okay, re- placing it is a lot more questionable.

If a class only overrides __getattr__, then I agree it should only add to __dir__ (most likely by using a super call as Georg suggests). If it overrides __getattribute__, however, then it can actually deliberately block access to attributes that would otherwise be accessible, so it may make sense for it to alter the basic result of dir() instead of just adding more attributes to the end. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

as well as updating the documentation in various places (the dir and PyObject_Dir documentation, obviously, but also the list of magic methods in the language reference).

oops, i meant everything except that -tomer On 11/7/06, tomer filiba wrote:

okay, everything's fixed. i updated the patch and added a small test to: Lib/test/test_builtins.py::test_dir

-tomer

On 11/7/06, Nick Coghlan wrote:

...

tomer filiba wrote:

...

cool. first time i build the entire interpreter, 'twas fun :) currently i "retained" support for __members__ and __methods__, so it doesn't break anything and is compatible with 2.6.

i really hope it will be included in 2.6 as today i'm using ugly hacks in RPyC to make remote objects appear like local ones. having __dir__ solves all of my problems.

besides, it makes a lot of sense of define __dir__ for classes that define __getattr__. i don't think it should be pushed back to py3k.

here's the patch: http://sourceforge.net/tracker/index.php?func=detail&aid=1591665&group_id=5470&atid=305470

As I noted on the tracker, PyObject_Dir is a public C API function, so it's behaviour needs to be preserved as well as the behaviour of calling dir() from Python code.

So the final form of the patch will likely need to include stronger tests for that section of the API, as well as updating the documentation in various places (the dir and PyObject_Dir documentation, obviously, but also the list of magic methods in the language reference).

+1 on targeting 2.6, too.

Cheers, Nick.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org