assignment to __class__ (was Re: Copy constructors)

[Alex Martelli]

"Guido van Rossum" <guido at python.org> wrote in message
news:cpk807e2pg.fsf at cj20424-a.reston1.va.home.com...
    ...
> "Alex Martelli" <aleaxit at yahoo.com> writes:
>
> > "Guido van Rossum" <guido at python.org> wrote:
> > >     http://www.python.org/2.2/descrintro.html
> >
> > I thought I had understood it, but still don't see where it mentions
> > "x.__class__ = whatever" now being forbidden or restricted?
>
> That one's easy. :-)  Assignment to __class__ is not documented for
> classic classes either, so I didn't think it was necessary to mention
> it.

True.  Indeed, __class__ is specifically (and falsely) mentioned
as being read-only (ever since 1.5).  Wish I had noticed that much
earlier and submitted a doc bug about it -- oh well!-)


> > > It would be relatively easy to allow __class__ assignment only if (a)
> > > the new class is a subclass of the old class, and (b) the size of the
> > > new instance is the same as the old instance.  Would this be
sufficient?
> >
> > Does 'size' in this context mean 'number of slots'?
>
> Close enough.  In combination with the subclass requirement, the
> "equal size" requirement means that the subclass doesn't add any
> slots, and that guarantees the safety requirement.

Which is what I assumed in the rest of my post.

> But without the subclass requirement, in order for the __class__
> assignment to be safe, things are more difficult.  E.g. a list
> instance has two "slots" beyond the basic object header: a size and a
> pointer.  Now this class:
>
>   class C(object): __slots__ = ['a', 'b']
>
> also has two slots following the basic object header.  But switching a
> C instance to a list or back would be a disaster, because the
> interpretation of the slots is different.

Yes, that's quite clear.


> > In this case, albeit
> > with somewhat peculiar contortions (requiring new.classobj or the
> > equivalent), I think it would be sufficient for all cases that come to
> > my mind -- I'd just have to put any extra attributes in the __dict__
> > (which I do today for *every* attribute anyway:-).
>
> Sorry, I think you're off on the wrong foot there.

Care to explain why?  You don't, in the following.


> > The "generate empty object through __class__ assignment" trick
> > would also become sort-of-possible again (although of no practical
                                                       ***************
> > interest whatsoever:-) -- to wit:
    *******************
> >
> > def make_empty_copy(any_object):
> >     klass = any_object.__class__
> >     class Empty(klass):
> >         def __init__(self, *args, **kwds): pass
> >     newcopy = Empty()
> >     class Full(Empty):
> >         __init__ = klass.__init__
> >         __name__ = klass.__name__
> >     newcopy.__class__ = Full
> >     return newcopy
> >
> > the Full class is not really the same as any_object.__class__, but
> > nobody's gonna find out (presumably) since it's undistinguishable
> > under normal use of isinstance or any behavior-test whatsoever
> > (or have I forgotten to copy some needed attribute for that?).
>
> I don't think this is a useful kludge.

Do you READ the posts you're answering?  I *did* say and you quoted
(and I've underlined it with asterisks above) that this specific
trick is "of no practical interest whatsoever" -- so why are you
paraphrasing what I just wrote as if you were saying something
different?

> And how is newcopy going to
> acquire all the othe rattributes of any_object.__class__?

That's up to the caller, just like it's up to the caller of
    any_object.__class__.__new__()        # in the new Python
or
    new.instance(any_object.__class__)    # in good old Python
which this useless trick sort-of-mimics.  Isn't this obvious?


> > These classes Empty and Full are examples of what I mean above
> > as roughly 'equivalent' to new.classobj calls:-).  I think I could
> > handle 'real' cases of class-change (worst case) through similar
> > means, i.e. generating on the fly a class that's basically what I
> > need but formally inherits from the object's original class so it
> > can be assigned.  I realize this will not support _deleting_ any
> > method wrt the original class, but that's not a need in any case
> > that easily comes to mind.
>
> It would be easier to bite the bullet and write the metaclass (in C)
> that does the proper safety check.

Except that this requires ahead-of-need design of infrastructure,
or deep refactoring even just to try something out.  Having a
working kludge to try things out (refactoring later if need be)
is much better for incremental, just-in-time design.


> > Yes, but, when servers DO have to stay up, fixing them on the fly,
> > albeit indeed fragile, is a specified constraint.  I guess in some cases
> > one could devise alternate strategies: putting up a new fixed server
> > process on a different machine, port-redirecting all new requests to
> > the new machine, and finally pulling down the old buggy server when
> > the conversations for the requests it was serving at fix-time are gone.
>
> I'm assuming you have to plan for this anyway, since you'll need a way
> to doctor the server in the first place.  So why not make your
> planning easy by having __dynamic__ = 1 somewhere?

No, I don't have to plan *in advance* for this functionality, *in
Python as it stands today*.  This is thanks to Python's rich and
powerful dynamic features, of course.

Consider.  I release PlikServerFramework (PSW) 1.0 -- programmers
who program on top of PSW can write their own plug-in classes and
register with a factory-registrar.  I do not constrain plug-in
classes so they have to inherit from PSWobject -- I just specify
they have to provide a certain signature (certain methods with
certain signatures), since signature-based polymorphism is much
more general and handier.

Then later on I release PSW 1.5, which adds features (rarely do
new releases REMOVE features, though Python's an exception:-0).
[Note: this IS a quip, but actually I consider Python's removal
of 'wrong' old features a peculiar strength -- it's just when
the features slated for removal are 'good' ones that I worry:-)].

One of the new features is, a running server can be upgraded.
PSW-using programmers don't have to change any of their existing
PSW plug-ins -- they can just specify a few new flags, if they
wish, when they submit a plug-in to the factory-registrar, to
specify whether the new plug-in must override an old one and
to which extent (only for new connections, or even for ones
that already exist).  Beautiful, isn't it?  A programmer X may
be using some PSW plug-ins that were separately released by
programmer Y (and *of course* X doesn't want to fork Y's code,
that's a seriously dangerous anti-pattern) and get the new
PSW 1.5 benefits without worries.  Cool!

Only, I can't do it any more in Python 3.0 (or whenever it is
that __dynamic__=0 becomes the default) -- I have to have the
foresight to specify infrastructure, that may come in useful
in future framework releases, right from the very first 1.0
release -- or else I can't offer these new functionalities
without requiring PSW users to modify existing plug-ins (which
might be a serious problem for plug-ins written by others).

This lowers Python's utility for such tasks to closer to that
of (e.g.) Java, and makes other languages (competing for the
same niche as Python, but clever enough to retain a richer
dynamic behavior, such as Ruby) more relatively attractive
compared to Python.


> > *blink* I had never thought Python's philosophy was protecting
> > "typical users" from themselves -- I thought that was the idea
> > of Pascal, Modula-2, &c, to Eiffel, the languages that know what's
> > good for you better than you know yourself, so they'll force you
> > to program the way Wirth (or Meyer) KNOWS is the one right
> > way to program.  As I previously read your recent posts, I thought
> > that the __dynamic__ thing was about performance instead...?
>
> Here we go again.  I mention one reason and it is assumed that this is
> the only reason.  As I've said before, my mind works a lot faster than
> my fingers can type (it seems it's the opposite for you :-), and
> sometimes the rationalization for an idea comes only gradually.

My fingers are pretty fast indeed, but I seriously doubt if you're
qualified to judge on the working speed of my mind; although you're
of course free to disagree, it appears that many readers enjoy my
posts, which suggests a reasonable amount of working-mind in them.

Your (relative) inability to rationalize your (often excellent)
ideas is indeed well-known.  But does that mean we should never
respond to what you say, because what you say is known to be a
probably-imperfect verbalization of reality?


> If you think about it, Python does a lot to protect typical users from
> themselves (why otherwise do you think it's gaining success as an
> educational language)?  For example, an input line of a million bytes
> won't cause a buffer overflow.

Not in Java, either.

> For example, arithmetic overflow is
> not silently truncated.

Not in Java, either.

> For example, almost anything that could cause
> a core dump is caught before it does (and we fix the remaining core
> dumps in real time :-).

Ditto for Java.

> For example, mixing incompatible types in
> expressions causes a TypeError rather than having a random undefined
> side effect.  For example, using an undefined variable name raises a
> NameError rather than silently being equal to zero.  For example, the
> whole division thing.  And so on.

"Better" still, in Java, mixing incompatible types is often
caught at compile-time, and using undefined identifiers is
_always_ caught at compile-time.  Since typical users do not
test their programs anywhere like nearly enough, it's crucial
to provide compile-time catches for their typical mistakes,
particularly typos, isn't it?

So, if protecting "typical users" against their own mistakes
is a priority, it seems that Java fills that niche 'better'
than Python does.

So where's Python's strength?  In *NOT* going as far as Java
in this 'protection' racket (notoriously a lucrative one, but
shady:-) -- in *selectively* protecting against stuff that's
worth protecting against, but no more than that -- and thus
not *getting in the way* the way Java does.

Say I'm using class X from some library.  The library author
didn't think of providing instances of X with a 'fleep'
attribute, yet I need that attribute in X instances for
my use.  In Java, I need much work -- I must subclass X
with my own Y, and arrange for Y instances to be generated
instead of X instances (assuming that's possible -- if
the instances are generated from within some framework,
I'm SOL if the framework author didn't make the factories
user-overridable).  In Python, it's a breeze -- if nothing
else, I can add a fleep to X instances on the fly; but,
better still, I can also (in Python as it stands today)
provide a *DEFAULT* value of fleep for X instances for
which it's not been explicitly set otherwise, just by
doing X.fleep=23.

Ooops -- now, in the new Python, I'll be SOL again (to
some extent) if the framework author lacked foresight:
I'll still be able (presumably, and hopefully) to add
an instance-attribute fleep to specific instances, but
rather than the handy default X.fleep I'll have to
surround every use of x.fleep for some instance x of
X with hasattr tests or try/except statements, so as
to be able to set x.fleep to the default 23 if fleep
was never set for this specific x as yet.  A pity.

In some cases I'll be lucky -- if X instances are
usable as directory keys, for example.  Or maybe I
can concoct a grandiose scheme for "non-invasive
fleep adjunction" based on weak references and id().
Again, these kludges are roughly equivalent to what
Java provides -- if X is hashable I can use that in
Java too, and, in Java too, I can (and have to)
concoct grandiose weak-references-based schemes.

I'd rather not have to, for such an unfortunately
frequent task as "customize to my needs a library/
framework whose author didn't have the foresight
to provide the customizability features I need".

I wish you worked for a few months with, say, the
Microsoft MFC library/framework: I think you'd
come out of the experience with a new, powerful
respect for the way Python (accidentally, it seems)
lets one naturally and effortlessly avoid such
huge wastes of effort.


> > Can we expect variable declarations a few minor releases from
> > now, then?  That would presumably be consistent with the new
> > focus on protecting typical users from themselves.
>
> I'm presuming you're being sarcastic.  We may indeed see *optional*
> variable declarations -- but not required ones.  But realistically,
> even the optional declarations seem far away -- the types-sig is only
> active for about 6 weeks per year, and most of that time is used
> rediscovering where we were a year ago... :-(.

Definitely not a recipe for fast progress.


> > "Of course" as long as the extra devices don't significantly
> > interfere with the tools' previous strengths in terms of cost and
> > power.  When the significant interference is there, there is no "of
> > course" about it -- it becomes a highly problematical trade-off.
>
> Exactly.  What's going on here is that I'm trying to tease out how far
> I should go with the safety device without making the tool unusable.

*unusable* is a *VERY* strong word.  Java is not "unusable".
Neither is C++, or, for that matter, C, or Fortran IV.  In
each case, this is proven by a lot of important and very
successful software systems written and deployed in the various
languages.  So, a language completely bereft of dynamic
feature, or very poor in them, can still to some extent
be 'usable' -- just not as useful as it would be with
simpler and richer dynamic features, such as Python's today.

I don't think your test should be, whether removing features/
"adding safety devices" makes Python "unusable".  Making it
"less useful than before" should hopefully be enough to block
the feature-removal/safety-addition.

> The new classes in Python 2.2 (which are entirely optional -- by
> default you get the same classic classes as in 2.1 and before) are one
> step of a new design.  We'll see how it needs to be tweaked.

The problem from my POV is therefore not with Python 2.2 -- if
the default still allows the dynamic aspects of today's Python,
providing a way for such aspects to be explicitly turned off
is not a disaster from my point of view.  Just like Java's "final"
keyword to turn off class extensibility or method overridability,
it can be abused, but it takes positive malice (what's the
opposite of 'forethought'?-) to abuse it, as it's NOT the
default.  I doubt you remember it, but a potential feature
I've often chatted about in this group is the ability to
explicitly 'lock' a dictionary in various ways (and I'm
tickled pink that I'll now be able to experiment with it
by subclassing built-in dictionaries!-) -- that's optional
explicit stopping of dynamic behavior, too, and without
even the positive performance side-effects that your new
ideas have.  The keywords from my POV in the sentence
"optional explicit stopping of dynamic behavior" are:
    1. most important: OPTIONAL -- there must still be
       a way to get the good old dynamic behavior
    2. very important: EXPLICIT -- ideally, dynamic
       behavior should be the default, with a way to
       EXPLICITLY turn it off, rather than vice-versa

"Optional, explicit stopping" would mean a smooth progress
towards a Greater And Better Python.  Changing defaults,
or, worse, removing some of today's dynamic aspects, even
though they had "just slipped in by mistake", has darker
characteristics... that 'mistake' was serendipity, the
Goddess Fortune smiling upon you.  Don't turn your back
to the Goddess when She smiles -- She's pretty fickle!-)


Alex