LONG MESSAGE -- Draft of meta-object protocol

[Bobrow.pa@Xerox.COM]

Meta-object Protocol  (DRAFT 6 Feb 87)

The meta-object protocol is based upon a set of objects that are defined
in the system.  The following is a public view of these classes.
Implementations are free to have addtional slots, or simulate slots
using the accessors that are defined.  For all slots specified, there is
a reader with the prefix specified.  For some slots setf methods are
also defined as described below. The notation (setf
<generic-function-name>) is used to refer to such methods.

There are four pairs of classes described: class, standard-class,
slot-description, standard-slot-description, method, standard-method,
generic-function and standard-generic-function.  The first of each pair
represents what is thought of as the minimal or most abstract
functionality to be defined in CLOS.  The second, with prefix standard-,
are the ones described in chapters 1 and 2.

All these classes are required to be instances of standard-class.  All
instances of standard class (these are all classes)inherit from
standard-object which defines a small set of default behaviors.  This
class is effectively defined by:

(defclass standard-object (t)())

Behaviors for standard-object include methods for describe, print-object
and initialize as described in the concepts chapter.
  
There are two major sections, one dealing with the classes and
slot-descriptions, the second with methods and generic-functions.  Both
start with descriptions of the classes and then describe other methods
specialized to the classes, organized by their use in the metaobject
protocol.  Methods that are described in the function chapter are
mentioned but not described.
    

1. Classes and Slot Descriptions
  
Classes are used for two distinct purposes: to describe the structure of
a set of objects, and as a parameter specializer for the definition of
methods.  The description of the structure uses a set of objects that
are instances of slot-descriptions. Classes specify some direct
slot-descriptions, and inherit others from their super-classes.   We
first show the structure of class, and standard class, and then the
classes for slot-descriptions.    


1.1 Class and standard-class

(defclass class (standard-object)
    (direct-supers
     direct-subclasses
     class-precedence-list)
  (:reader-prefix class-))

(defclass standard-class (class)
    (direct-slots
     all-slots
     generic-functions
     prototype)
  (:reader-prefix class-))   


[[Initial values for these slots are all NIL]]

direct-supers is a list of class objects.  If it is changed, then the
updating protocol described below is followed.

There is a setf generic function for class-direct-supers.

(setf class-direct-supers) on class causes the direct-supers slot to get
updated, and calls supers-changed.  See supers-changed.

direct-subclasses is a list of classes.  It is the inverse of the links
from direct-supers.  If x is in the direct-supers of y, then y is in the
direct-subclasses of x.  See add-direct-subclass,
remove-direct-subclass.

There is no setf generic function for class-direct-subclasses. 

class-precedence-list is a list of classes.  It is computed from
direct-supers. It is computed (just) before the first instance of this
class is created, or when any method is defined on this class or any
subclass. It is maintained whenever supers of this class or any super
class change. See compute-class-precedence-list, compatible-super-p,
supers-changed.

There is no setf generic function for class-class-precedence-list.  

direct-slots is a list of slot-description objects.  If it is changed,
then the updating protocol described in section 1.4. 

There is a setf generic function for class-direct-slots.

(setf class-direct-slots) on class signals an error.
(setf class-direct-slots) on standard-class causes the class to get
updated. See slots-changed and section 1.4.
  

all-slots is a list of slot-description objects.  It is computed from
direct-slots of all the classes on class-precedence-list.  It is
computed (just) before the first instance of this class is created, or
with-slots is expanded for any method on this class or subclass.  It is
maintained if the supers or slots of the class or any of its supers are
changed. See collect-slot-descriptions,
compute-effective-slot-description, supers-changed, slots-changed.

There is no setf generic function for class-all-slots.
  
generic-functions is a list of generic-function objects that have a
method defined using this class as a parameter specializer.  A generic
function appears on this list only once no matter how many relevant
methods it contains.  It is kept on a class as a cache to allow access
to methods that depend on the class definition.  This allows updating of
methods when classes change.  It also provides a hook for browsers in an
environment.  It is maintined by add-method.

There is no setf generic function for class-generic-functions.


prototype contains an instance of this class.  It is created the first
time the accessor class-prototype is used.  The  prototype is not
guaranteed to have any slots filled in.  It is to be used as an argument
to allow access to methods specialized to the class that don't use the
contents of an instance of the class.  This is used extensively in the
paarsing of the defclass form, for example.  In general, it allows a
specialized method to be invoked without having to create a new instance
of the class each time.

There is no setf generic function for class-prototype.
 

1.2 Slot-description and standard-slot-description

(defclass slot-description (standard-object)
    (name)
   (:reader-prefix slotd-))

(defclass standard-slot-description (slot-description)
    (initform
     allocation
     type)
   (:reader-prefix slotd-))


name is a symbol that CLtL allows to be used as a variable.  An instance
of any class that contains a slot-description with this name will have
access to a slot of this name.  See concepts document.

There is no setf generic function for slotd-name.

initform is a lisp form.

There is a setf generic function for slotd-initform.

(setf slotd-initform) sets the value of initform and calls
slots-changed.    

allocation is one of :instance, :class, :dynamic.

There is no setf generic function for  slotd-allocation

See the concepts chapter to see how these values are used for
standard-class.

type is a Common Lisp type descriptor

There is no setf generic function for slotd-type 

class-value is the storage for a value in a class.

There is a setf function for slotd-class-value

(setf slotd-class-value) sets the value.


1.3  Creation of classes

Classes are instances of metaclasses.  Hence they can be created by the
ordinary make-instance protocol. The initalization of a class allows
allows two slot-names to be given:

direct-slots with value a list of slot-description objects

direct-supers with value a list of classes (or names of classes) These
names are changed to the classes they refer to (using
class-named-for-metaclass) as the first step in
compute-class-precedence-list.  An error is signalled if a class is not
defined with that name at that time.  

In addition, at the start of the computation of class-precedence-list, a
compatibility check is made to determine whether the class (of this
metaclass) can inherit from the specifed direct-super-classes using:.

compatible-super-p class possible-direct-super

class is the one to get the new possible-direct-super class.  If this
method returns NIL an error is signalled.  The default method for
compatible-super-p allows use of possible-direct-super only if the
metaclasses of class and possible-direct-super are the same.

compute-class-precedence-list class
     &optional (direct-supers (class-direct-supers class))

This computes a total order extracted from the direct-supers of the
class.  The optional argument allows the environment to compute what the
class-precedence-list would be if the direct-supers were those provided.
See concepts chapter for the behavior of compute-class-precedence-list
for class. 

Once the class-precedence list is computed, the set of all
slot-descriptions for the class can be computed.


collect-slot-descriptions class
         &key (direct-supers (class-direct-supers class))
              (direct-slotds (class-direct-slots class))

This generic-function computes a list of slot-descriptions interpreted
as a set.  The importance of the word set is that the order of the
slot-descriptions is not guaranteed to be used by the implementation to
build instance structure. 

No two slot-descriptions in the set can have the same name.  The set of
slot-descriptions is computed using the method
compute-effective-slot-description, called once for each name in the
union of all the slot-descriptions of all the classes in the
class-precedence list.  An implementation is free to add additional
slot-descriptions.

The optional arguments possible-direct-supers and possible-direct-slots
are available so that the environment can ask what the set of slots
would be if those were used instead of the values in the current slots.

compute-effective-slot-description class same-named-slotds 

The argument same-named-slotds is a list of slot-descriptions in
most-specific-first order specified by the class-precedence-list.  The
behavior of this method for the standard-slot-description is found in
the concepts chapter.


1.4 Updating classes

A standard-class once defined can be updated to have different
direct-slots and direct-supers

update-class class &key :direct-supers :direct-slots

where direct-supers is list of classes and direct-slots is a list of
slot-descriptions. It allows only a single updating pass when both
direct-supers and direct-slots are changing.   

To preserve the backlinks from supers to subclasses, the following
generic functions are provided.  supers-changed calls these.

add-direct-subclass class subclass

adds subclass to direct-subclasses if it is not present.

remove-direct-subclass class subclass

removes subclass from direct-subclasses of class if it is present.  It
signals an error if subclass was not in the list.


If any changes are made to supers, the class-precedence-list of this
class and its subclasses may have changed.  The set of slot-descriptions
of this class and subclasses might have changed.  To allow updating of
these slots, and any other cached information in a particular
implementation, the protocol calls the recursive generic function:

supers-changed class old-supers old-slots 
               &optional top-flag &rest rest-args

When this method is invoked, it is assumed that class-direct-supers and
class-direct-slots will return their new values.  The old values are
provided to facilitate making incremental changes. The contract of the
generic function is to update any cached information about supers and
slots in class, and any of its subclasses.  supers-changed is called
recursively on all the subclasses of the class that is changed.  The
variable top-flag, which is T only on the initial call from updating
methods, allows specialized methods to take specific actions for the
class that has changed directly.  The rest-args are provided so that
extra information can be passed down by specializations, if desired.

The supers-changed method for standard-class updates the
class-precedence-list, and the list of all-slots using
compute-class-precedence-list, and collect-slot-descriptions. It also
makes obsolete any class that has a different shape, that is has a
different number or organization for slots.  See below for a further
description of obsolete classes.  supers-changed is called by 
(setf class-direct-supers) and by update-class.

A similar protocol support updating if only the set of slot-descriptions
provided have changed using the generic function slots-changed

slots-changed class old-slots 
               &optional top-flag &rest rest-args

It is assumed any updating to direct-slots has already been done.  The
old-slots are provided to facilitate updating. The contract of the
generic function is to update any cached information about slots in
class, and any of its subclasses.  The variable top-flag, which is T
only on the initial call, allows specialized methods to take specific
actions for the class which directly changed.  The rest-args are
provided so that extra information can be passed down by
specializations, if desired.

The method for standard-class updates the list of all-slots using
collect-slot-descriptions. It also makes obsolete any class that has a
different shape, that is has a different number or organization for
slots.  See below for a further description of obsolete classes.

In order to support caching in generic functions, when a class is
changed, the following protocol is supported

update-method-inheritance class old-supers

In standard-class this causes recomputation of effective methods for all
the generic functions on this class and any subclass.

1.5 Obsolete Classes

If the effect of updating the class is to change the shape of instances
that would be produced by this class, then instances produced according
to the former description are now "obsolete"; that is they no longer
conform to the current description in the class by this name.  When a
class must be made obsolete, a protocol informs the class:  

make-class-obsolete class 

To determine if a class needs to be made obsolete, it is a useful
optimization to know if any instances of this class have ever been made.

has-instances-p class

is a predicate that returns NIL if it knows no instances have ever been
made.  
 
For standard-class, make-class-obsolete ensures that old instances are
now instances of an obsolete-class that knows about the old structure,
and about the new class description.  The next time any instance of this
obsolete class is used either as an argument to slot-value, or as a
specialized argument for a method, 
   (change-class instance updated-class)
is  called to change it to conform to the new class definition.  See
concepts chapter for further discussion   


1.6 Expanding defclass forms

This protocol is defined to allow users to extend the use of defclass in
some limited ways.  For this purpose, we define the abstract syntax of
defclass as follows:

(defclass name (super-name*)
  ((slot-name {slot-prop slot-prop-value}*)*)
  {(class-option-keyword . class-option-args)}*)

That is, slots can have an arbitrary property list following the name,
and all class-options are lists with a keyword.  The only way to specify
the metaclass used by defclass is by using the :metaclass class-option.
The metaclass option is used to get a prtotype of the metaclass for use
by the generic functions specified below

The types of extensions we visualize are ones that would allow
additional slot-properties and class options interpreted by the
metaclass.


expand-defclass metaclass-prototype defclass-form

uses the :metaclass option to obtain a prototype of the metaclass, and
expands the defclass form.

check-defclass-form metaclass-prototype slot-list
                 class-options
signals an error if there are any slot-properties or class-options that
are not recognized.

add-named-class metaclass-prototype
       name super-list slot-list class-options

creates the appropriate class, doing the conversion from a list of names
to a list of super-classes, converting the slot-list to a list of
slot-descriptions of the appropriate kind. The defclass expansion
contains an (add-named-class ...) form.  The other forms in the
expansion of a defclass form are expressions to creating the accessor,
reader and constructor methods specified in the defclass form.

add-named-class checks if these is currently a class defined with the
same name.  If so, it checks to see if it is acceptable to update that
class using the generic function:

class-for-redefinition metaclass-prototype old-class-with-name
                       &rest other-information

class-for-redefinition signals an error if the class cannot be
redefined, else, or returns a class that is to be updated (this might be
a new class or the old one).  On class, class-for-redefinition always
signals an error. standard-class, it updates the class and supports
obsolete instances.

make-slot-description metaclass-prototype 
                      slot-description-list 
creates the appropriate kind of slot-description object for class,
filling it in from slot-description-list 
This is how extensions to the slot-properties can be added without
breaking defclass.
  

compute-super-from-name metaclass-prototype name

is used to obtain a class from a name.  It can use class-named or in a
compile-time environment may want to be shadowed to find a special class
object or definition.



1.7 Class names

The association of a name with a class is supported by the metaclass.
Not all classes need have names.  The basic lookup function is: 

class-named-from-metaclass metaclass-prototype name
      &key :no-error-flag :compile-time-hash-table  

This returns the class with that name, or signals an error if none.  If
no-error-flag=T then returns NIL if none.  If compile-time-hash-table is
given then it is used for looking up names, and for setting them.  This
allows a sompiler to interface with this metaclass specific name lookup.

There is a setf generic function defined for class-named-from-metaclass.
(setf class-named-from-metaclass)
signals an error if name is already in use for another class.   It
no-error-flag is T, then this is a noop. 


Additional support for instances of class is provided by

class-named name &optional no-error-flag

class-named is defined in terms of class-named-from-metaclass.

  
If a class has a name, then it can be found using:

class-name class &optional unnamed-value

Returns the name of a class, if it has one, or returns unnamed-value.  

There is a setf generic function defined for class-name.

(setf class-name) name

We do not need a class-name-using-metaclass since class-name is already
specialized to the class. 

If new-name is NIL, then class will have no name afterwards.  If
new-name is the name of another class, then an error is signalled.  If
class has another name, then class will be first made unnamed, then then
given the new name.  It is noop if class already has that name. 


1.7 Instance Slot Access

The function slot-value is defined to use:

slot-value-using-class class instance-of-that-class slot-name
                       &optional value-if-slot-missing

This method on the metaclass that knows the representation of the
instance, and returns the value.  It calls slot-missing if a slot by
this name is not accessible from this instance.  If
value-if-slot-missing is provided, this value returned instead of
calling slot-missing.

slot-missing instance slot-name

is called if no slot with slot-name is accessible in instance.  The
method for slot-missing on standard-class just signals an error

There is a setf generic function for slot-value-using-class.

(setf slot-value-using-class) sets the value of the specified slot.  It
calls slot-missing-for-setf if a slot by this name is not accessible
from this instance. If value-if-slot-missing is provided in the
slot-value form, this value is returned instead of calling
slot-missing-for-setf

slot-missing-for-setf instance slot-name new-value

The method on class for slot-missing-for-setf signals an error.


slot-exists-p instance slot-name &optional allocation

is a predicate that returns true if there is a slot with name slot-name
accessible from instance.  If allocation is given, then returns true if
it has the given allocation. Returns NIL otherwise

all-slot-names instance &optional allocation
returns the list of all slot-names accessible within this instance,
limiting it to slots with a particular allocation if that argument is
given.


add-dynamic-slot instance slot-name new-value

adds to instance a dynamic slot with value new-value.  Signals an error
if slot-name is already a slot accessible from instance, and not a
dynamic slot.  If it is a dynamic slot then just changes the value.

remove-dynamic-slot instance slot-name

removes the dynamic slot from the instance.  Signals an error if such a
slot does not exist on the instance.

dynamic-slot-value instance slot-name &optional default-value

gets the value from the dynamic slot.  Signals an error if slot is not a
dynamic slot, or returns default-value it is provided.

(setf dynamic-slot-value) new-value

sets the value of the dynamic slot.  Signals an error if slot is not a
dynamic slot, or returns default-value it is provided. 


1.8 Optimizing Slot Access at Compilation

To provide users a hook to optimize access to slots in compiled code,
the protocol provides 

optimize-slot-value class-for-var form

This returns a specialized form for compilation, or original form if it
can do nothing with the form.  It can check in the method about the
parameter-specializer for any arguments in the form.  form look like:
  (slot-value <var-name> '<slot-name>)  

optimize-setf-slot-value class-for-var form

returns a specialized form for compilation, or original form.  form must
be of the form (setf (slot-value <var-name> '<slot-name>...)...)



1.9 Relationships among classes and instances

The class system extends the type system.  There are methods that
correspond to typep and subtypep

classp instance class

Returns T if (class-of instance) is a subclass of class, that is, if
(subclassp (class-of instance) class).

subclassp class1 class2
returns T if class2 is an element of the class precedence list of
class1, that is, if class2 is class1 or any of its superclasses.

[[?? As an alternative, we could extend typep and subtypep directly to
allow class arguments where they now have type-specifier arguments]]




2. Methods and Generic Functions

2.1 Method Structure

(defclass method (object)
    (generic-function
     parameter-specializers
     function)
  (:reader-prefix method-))

(defclass standard-method (method)
    (qualifiers)
  (:reader-prefix method-))

generic-function is NIL or the generic-function of which this method is
a part.

There is no setf generic function for method-generic-function.

parameter-specializers is a list of elements interpeted by
compute-discriminator-code to determine if this method is applicable for
a particular set of arguments.  For standard-method, this list consists
of class objects or lists of the form (QUOTE individual).  See concepts
chapter.

There is no setf generic function for method-parameter-specializers.


function is a function that generic-function will call if the
discriminator-code selects this method as the appropriate effective
method.  This method function can be applied directly to the same
argument list to the generic-function, provided the arguments match
their parameter-specializers.

There is a setf generic function for method-function.  
(setf method-function) fn
signals an error if the fn can not be determined to have  a lambda list
congruent with that of the generic function.  (setf method-function)
calls generic-function-changed.  


qualifiers is a list of non-nil atoms.  

There is no setf generic function for method-qualifiers.


2.2 Generic Function Structure

(defclass generic-function (object)
    (methods
     discriminator-code)
  (:reader-prefix gf-))

(defclass standard-generic-function (generic-function)
    (argument-list
     argument-precedence-order
     method-class
     method-combination
     declarations)
  (:reader-prefix gf-))

methods is a list of method objects.

There is no setf generic function for gf-methods

discriminator-code is a function that actually does the discrimination
when the a generic-function is called.  Implementations are free to make
special classes of generic-function objects that are recognized by the
interpreter, and for which the discrimination code is done in the
interpreter.      

There is no setf generic function for gf-discriminator-code.

For the following, see also the discussion in the function and concepts
chapters.

argument-list is an ordinary CL lambda-list with no aux variables.

There is a setf generic function for gf-argument-list.

If there are any methods on the generic function, it signals an error if
the argument list is not congruent to those of the methods.

argument-precedence-order specifies the order that is to be used in
computing precedence of methods.  It is a permutation of the required
arguments of argument-list.  It is used by compute-discriminator-code to
determine the order of applicability of methods.

There is a setf method for gf-argument-precedence-order. 

If it is used, it checks that the new value is a permutation of the
required arguments of argument-list, or signals an error.  If
argument-precedence-order is different than it was before,
compute-discriminator-code is called to update the gf-function.

method-combination is a list whose first element (car) is the
combination type and the rest (cdr) is a list of parameters for
method-combination.

There is a setf generic-function for gf-method-combination with a method
defined for standard-generic-function.  If it is used, then
compute-discriminator-code is called to update the gf-function.

method-class is a class.  It will be used as the class for methods
defined with defmethod on this generic-function. 

gf-method-class has a setf-method. 

declarations are those provided in def-generic-options.  It is used by
compute-discriminator-code.  

gf-declarations has a setf method.


2.3 Creating Methods and Generic Functions

Instances of methods and generic functions are made using make-instance.
The initialize method for method supports slot-names of
parameter-specializers, argument-list and function.  The initialize
method for generic-function supports properties for all of its slots
except discriminator-code. 

ensure-generic-function symbol &key class make-fn-be-default-p

is documented in the concepts and function chapter.



2.4 Manipulating generic-functions and methods


add-method generic-function method

remove-method generic-function method

get-method generic-function parameter-specializers

generic-function-changed gf &optional hints
  this is called by add-method remove-method and (setf method-function)

get-setf-generic-function symbol

   These are documented in the concepts and function chapter.


2.5 Building Code for Generic Functions

The code used by a generic function is computed by the following method.
All callers of this method know how to install that code. 
 
compute-discriminator-code generic-function

This method returns a function, or some object recognizable to the
implementation dependent code that installs this function in the
generic-function object.

The usual way for people to effect what code is generated is throguh
method combination, and its define-method-combination interface.  This
interface defines a method on the generic-function
compute-effective-method.  

As part of the code, if there is no applicable method for a
generic-function, the generic function should call the method

no-matching-method generic-function arglist
The default method for this generic function signals an error.  

compute-effective-method
     generic-function
     combination-type
     applicable-methods
   &optional parameters

This should return a form that is used for the body of an effective
method composed from the set of applicable methods.  It can use
make-method-call.  See concepts chapter.  The contract between
compute-discriminator-code and compute-effective-method says that it
must be called for all significantly different sets of applicable
methods that can arise from a set of arguments.  

If the user wants to build code using compute-discriminator-code, the
following methods may prove useful.

find-applicable-methods generic-function
            argument-list &optional sorted-flag
returns a list of methods on generic-function that are applicable to the
arguments given.  If sorted-flag the list is sorted most-specific-first.

method-equal method1 method2
tests whether two methods of the same generic-function have the same
specializers and qualifiers.

method-more-specific generic-function
   method1 method2 argument-list

Returns T if method1 is more specific than method2 with respect to the
argument list.


3.0 Defining New Storage Metaclasses

In order to define new mechanisms for storing instances, there must be
some system dependent code.  The interface to that code is:

define-metaclass name size class-of-code

This function creates associated with name

1) a mechanism for allocating structures of this kind
   The structure should be of the size specified.  If size is NIL, then
the allocation routine will expect a parameter indicating the desired
size.

2) a mechanism for recognizing when a structure of this type is asked
for its class.  For such structures, the function (or macro) specified
by class-of-code will return the code.

A new class must also be defined using the same name.  In its
make-instance method it can use:

meta-allocate-instance meta-name &optional size

meta-slot-value meta-name thing position

meta-set-slot-value meta-name thing position new-value



As a mechanism for storing slots, the user should have accessible a
function (macro)

make-memory-block size

and accessor

memory-block-ref block size

In PCL these are just calls to make-vector but something better is
probably needed.