Class redefinition and class-changed.

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

I have several comments to offer on this conversation.  I haven't managed
to organize the separate comments into as coherent a whole as I would like.

I believe Patrick's argument that it is necessary to define the
CLASS-CHANGED method -before- redefining the class.  Symbolics has a lot of
experience with doing it the other way and it's unpleasant.  I hope the
reasoning for this has been established now and doesn't need additional
argument (Danny, please speak up if you don't believe it yet).

I don't like Patrick's particular proposal, though.
GET-OBSOLETE-CLASSES-FOR-REDEFINITION is too complicated, because it
returns a whole alist rather than being just the primitive.  In addition, I
don't think the set of "classes implicated in the class-redefinition of
CLASS" is computable until the details of the redefinition are known.
Suppose
  (defclass one () (a b))
  (defclass two (one) (c))
  (defclass three (one) (d))
Now suppose we redefine one as follows:
  (defclass one () (a b d))
Referring to the third paragraph on page 1-11 of 87-002, one and two are
"implicated", but three is not, or at least, whether three is implicated is
implementation-dependent.  But if the new slot had been named e, both two
and three would have been implicated.  I don't think the information in
Patrick's alist is needed anyway; when defining a method, one only defines
it for the obsolete class corresponding to one class.  If a different
method needs to be defined for an obsolete subclass, the function can be
called again.

Thus I would replace Patrick's proposal with GET-OBSOLETE-CLASS-FOR-REDEFINITION
which takes one argument, a class, and returns one value, the obsolete class
that is a copy of the current definition of the class and will be used the next 
time the class is redefined; this is created the first time you ask for it.
Perhaps GET-OBSOLETE-CLASS-FOR-NEXT-REDEFINITION would be a better name.

As Danny proposed, a primitive to get the obsolete classes corresponding to
redefinitions that have already taken place is useful, for instance in case
you didn't realize ahead of time that you would need a CLASS-CHANGED method
and now want to patch up the program.  I would suggest a function
GET-OBSOLETE-CLASSES-FOR-PAST-REDEFINITIONS which takes one argument, a
class, and returns one value, a list of obsolete classes, newest first.

There seems to be some confusion about how class redefinition works in CLOS
and what an "obsolete class" is.  Let me restate what the 87-002 document
says in simpler language:  Redefining a class never creates a new class
object to replace the existing class object; it always modifies the
existing class object.  Thus there are no objects that exactly represent
"versions" of a class.  There is also the concept of "obsolete instances",
which have dynamic extent and are created solely to pass as the first
argument to CLASS-CHANGED.  When CLASS-CHANGED is called as a result of the
user calling CHANGE-CLASS, the obsolete instance is simply an instance of
the old class.  When CLASS-CHANGED is called as a result of updating an
instance to reflect the latest definition of its class, the obsolete
instance contains the slot values of the un-updated instance and therefore
needs to have a class that has those slots: this is an "obsolete class".
An obsolete class is a copy of a class remembering the slots, methods, and
superclasses it had before it was redefined.  Redefining a class only
creates an obsolete class when certain kinds of changes are made to the
class (see the third paragraph on page 1-11 of 87-002).  For each past
version of a class that is sufficiently different from the current version,
there is an obsolete class object.  Note that these are all obsolete versions
of the original class, they are not obsolete versions of each other.

Redefining a class also redefines subclasses of the class, so several
obsolete classes could be created.  87-002 doesn't say so, but I believe
the obsolete subclasses should be subclasses of the obsolete class and not
of the original class.

The second paragraph on page 1-12 of 87-002 could be interpreted as saying
that the only methods that are applicable to an obsolete instance I-sub-O
are the methods M-sub-C that were deleted from the class when it was
redefined.  I do not believe this is what was intended--I believe we
intended to say that all methods that were applicable to instances of the
class before it was redefined are applicable to I-sub-O (unless additional
redefinitions have been done, for instance deleting those methods
entirely).  If we really intended only deleted methods to be applicable, it
would be remarkably useless: for instance, deleted slots could be accessed
by normal means, but non-deleted slots could not be accessed except by
SLOT-VALUE, except in the case where the :ACCESSOR slot-option had been
removed.  Again, in the x-y-rho-theta example, if methods for accessing
values that are conceptually slots but are not physically represented as
slots were not applicable, it would be necessary to violate modularity and
duplicate the functionality of those methods within the CLASS-CHANGED
method.

We have our choice of three ways to define what happens when a class is
redefined:
(1) A new class object is created, the name-to-class-object mapping is
changed, defclass-defined methods implied by the new definition are put on
the new class object, all methods other than defclass-defined methods that
apply to the old class are made to apply also to the new class, subclasses
of the old class are made to be subclasses of the new class instead, and
superclasses of the old class are made to be also superclasses of the new
class.  If we had done this, there would have been an explicit
representation of versions of a class, and there would not have been any
concept of "obsolete classes."
(2) The old class object is modified, defclass-created methods that aren't
created by the new defclass are removed, new defclass-created methods are
added, an anonymous obsolete class object is created, the modification is
propagated to subclasses which may create obsolete class objects for them
that are subclasses of the first obsolete class created, superclasses of
the original class are to be also superclasses of the obsolete class, and
all methods that applied to the original class are made to apply also to
the new class.  (2) is what we did in 87-002.  (2) is not simpler than (1),
but it has the advantage that classes maintain their identity as objects.
(3) There is a third way, which I will describe later in this message.

Referring again to the third paragraph on page 1-11 of 87-002, there is
another use for a primitive to get the obsolete class before redefining the
class.  Not all redefinitions of a class call CHANGE-CLASS and
CLASS-CHANGED.  Suppose someone changes a class in such a way that they
need to call a function to update each instance, but they have not actually
add, removed, or renamed slots.  87-002 does not provide any mechanism for
this.  The programmer would probably resort to adding a dummy slot to force
instance updating.  I think a better approach would be to specify that
calling GET-OBSOLETE-CLASS-FOR-NEXT-REDEFINITION forces the next
redefinition of this class (directly or by redefining a superclass) to
update instances, even if it would not otherwise need to.

There are basically three approaches to the problem of writing a
CLASS-CHANGED method that updates an instance to reflect one particular
edit of the class's definition:
(1) Use the classes of the two arguments to CLASS-CHANGED to encode this
information.  This is what the 87-002 document says, however it doesn't work.
(2) Use a third argument to CLASS-CHANGED to encode the information, such as
a sequence number saying how many times the class had been redefined.  This
seems kludgey and was rejected in the past.
(3) Don't allow such precise control over instance updating in CLOS, instead
only allow CLASS-CHANGED methods that work for updates from -any- version of
the class to the current version.  This is copping out, however it's practical,
since it's what Flavors does.

There are two reasons (1) doesn't work.  First, 87-002 doesn't provide a way
to get the obsolete class for a particular edit, which you need in order to
define a method that applies specifically to that edit.  This is what Patrick's
proposal addresses.  Second, there is a problem when the class is redefined
multiple times.  Suppose we do
  (defclass foo () (a))
  (defclass foo () (a b))
  (defclass foo () (a b c))
  (defclass foo () (a b c d))
and in connection with the third definition we need a CLASS-CHANGED method
to initialize the slot C.  We don't want this method to be called for
instances that already have a slot C and are getting a slot D added, so we
want the method to specialize its first parameter as well as its second.
The problem is that CLOS has no way to define one method that applies to the
first and second obsolete classes, but not to the third obsolete class.  In
order to define this method, we have to know the exact editing history of
the class FOO, which seems unreasonable.  We can't fix this by making the
second obsolete class a subclass of the first obsolete class, because if
the user removes a slot we would have a class that has fewer slots than its
superclass, which CLOS does not allow.  We could call
GET-OBSOLETE-CLASSES-FOR-PAST-REDEFINITIONS to get the entire list of
obsolete classes for the class FOO, and then we could iterate over that
list seeing which obsolete classes had a slot C and which didn't, and
define CLASS-CHANGED methods accordingly, but this starts to look like an
elaborate rigamarole for something that should have been simple.

The above assumes that we believe what page 1-12 of 87-002 implies, namely
that instances are updated directly to the latest version of the class
("the second argument is an instance of the class C").  Alternatively, we
could specify that "ontogeny recapitulates phylogeny" and the instance is
repeatedly updated one step at a time, until it reaches the current
definition of its class.  In the FOO example above, we would define the
CLASS-CHANGED method that initializes C to specialize its first parameter
to the second obsolete class, with confidence that instances of the
original FOO would first get a slot B, then would get a slot C, then would
get a slot D, and in the second of the three steps, our method would be
called.

This means that -both- arguments to CLASS-CHANGED could be instances of
obsolete classes.  It also means that our CLASS-CHANGED method can't
specialize its second parameter, because at first that will be the class
FOO, but after FOO is redefined again, it will be an obsolete class.  If we
don't specialize the second parameter, it seems we could be confused by
CLASS-CHANGED being called when CHANGE-CLASS was used to change an instance
of FOO into an instance of BAR.  This is not a problem if CHANGE-CLASS,
like all other generic functions, first updates its argument to the latest
definition of its class, before changing it to the new class; then a method
specialized to an obsolete class in its first parameter can never be called
with the second parameter an instance of anything other than the next newer
version of the same class.  This all seems slightly kludgey, but does seem
like it should work.

Is there any chance of garbage collecting all these obsolete classes when
there are no longer any instances that haven't been updated yet?  We'd also
like to garbage collect the data structures required to make methods
applicable to the original class also applicable to the obsolete class.
I doubt it's possible to GC this stuff, the whole data structure seems too
interconnected.

The third way to define what happens when a class is redefined, mentioned
earlier, is as follows:  Don't try to use CLASS-CHANGED for both changing
the class of an instance and updating an instance to a new version of its
class.  Invent a new generic function CLASS-REDEFINED, which is called when
an instance is updated to a new version of its class.  This avoids the
complicated concept of "obsolete classes", at the cost of conveying the
former state of the instance in a less object-oriented fashion.  Perhaps
CLASS-REDEFINED would receive three arguments: an instance of the class,
already updated to the latest definition of the class, a property list
associating slot names to slot values, with one entry for each slot that is
no longer present, and a list of slot names of slots that were newly added
to the instance.  Thus when a class is redefined:
(3) The old class object is modified, defclass-created methods that aren't
created by the new defclass are removed, new defclass-created methods are
added, and the modification is propagated to subclasses.  This is simpler
than (1) or (2).

The FOO example given above is implemented by defining a CLASS-REDEFINED
method that checks whether slot C is a new slot before initializing it.
The problem is that there can only be one CLASS-REDEFINED method, since
there is only one class object, so if the class is redefined multiple times
in ways that need CLASS-REDEFINED methods, this one method must contain
unmodular knowledge of all the redefinitions.  In addition, in the
x-y-rho-theta example, if there are methods for accessing values that are
conceptually slots but are not physically represented as slots, the
CLASS-REDEFINED method has to know about the underlying slots used by
these methods, in case those slots have been deleted.

This "third way" seems simpler than the other two, so I hope that someone
can debug the problems mentioned in the previous paragraph.  If not, we
need to make several changes mentioned here and there in this message in
order to make obsolete classes work.