Object creation discussion (at last!)

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

I have returned from consulting my Muse in Montreal and I now have some
thoughts to offer on object creation.  I have some other things too, but I
think it would be better to settle object creation before embarking on
anything else.  I will present this in four parts: facts, philosophy,
proposal, and remarks.  The facts are just a framework for discussion.
Philosophy is in the form of some questions and my proposed answers; to
proceed we need to agree on some consistent set of answers to these questions.


>>> FACTS

Creating an object involves four steps:

  (1) Client calls an object-creation function, passing it some arguments.
      The function is either make-instance or a function created by the
      :constructor option to defclass.

  (2) Allocate storage for the object.

  (3) Fill in the slots.

  (4) Call the initialization methods.

Terminology:

  "initargs" (initialization arguments) are a set of named arguments that
  control object creation and initialization.  Each initarg has a name, which
  is a symbol (not necessarily a keyword), and a value.  The arguments to
  make-instance, excepting the first, taken in pairs, are initargs.


>>> PHILOSOPHY

>> Which of the four steps above should be customizable, and how are they
to be customized?

1. Client can call any function he likes, no other customization necessary.

2. Customizable by client in an implementation-dependent way; for example, in
   systems with areas, client can specify in which area to allocate storage.
   Also customizable by the metaclass, because the metaclass controls the
   stored representation of instances.

3. Customizable by slot-description, which specifies a default value for the
   slot and whether the slot can be filled with a value specified by the
   client; if so, the slot-description specifies the name of the initarg
   whose value is stored into the slot.
   Also customizable by the arglist of the :constructor option to defclass,
   which specifies names of slots into which arguments are stored.

4. Customizable by defining methods.

Only step 2 needs to be customizable by the metaclass.

>> Is make-instance an intra-module or inter-module interface?

Both.

>> Do the arguments to make-instance correspond directly to actual stored
slots, or are they a more abstract concept, whose implementation in terms
of slots or in terms of something else is hidden from the caller?

More abstract, because make-instance is often used as an inter-module
interface.

>> What is the lambda-list of a constructor created by a :constructor
option with no lambda-list specified?

It accepts the same arguments as make-instance, excepting the first.

>> Do constructors call initialization methods?

Yes.

>> Who is responsible for error checking of make-instance arguments?

Part of step 1 is the normal argument checking that happens on any function
call.  For constructors, this is completely normal.  For make-instance, the
behavior is the same as if there was always a constructor created by a
:constructor option with no lambda-list specified and make-instance simply
called it.

This implies that if the client calls make-instance with a misspelled
initarg name, "it is an error" (CLtL pp.62-3) unless :allow-other-keys t
was specified.

>> Can step 2 return an existing object instead of allocating storage,
aborting the remaining steps?

No, to do "interning" you must build a higher-level interface around
make-instance.  Make-instance always makes.

>> Why do we have a :constructor option to defclass?

For speed; make-instance is interpretive, while constructors are compiled,
since they know the exact class that they are constructing, and since they can
be automatically recompiled if the class or any of its superclasses changes.

>> Is speed of object creation important?

Yes, that's why we have :constructor.  Otherwise the user could just
write a defun whose body is a call to make-instance.

>> What assistance does the object system give to the class definer?

Error-checking in step 1; default storage allocation method in step 2;
convenient defclass slot-option syntax to control step 3; convenient
defmethod syntax to control step 4.

>> How should classes' initializations interact when they are mixed?

The set of valid initargs for a class is the union of the initargs defined by
the class and its superclasses.  Storage allocation is controlled by the
metaclass.  A slot is filled by a given initarg if the class or any of its
superclasses so specifies, thus more than one initarg can fill the same slot.
If multiple initargs that fill the same slot are supplied, which value ends up
in the slot is indeterminate.  Default values for slots inherit as discussed
in 87-002.

>> How should initialization methods combine?

Initialization methods combine in a way that is convenient for the common case
where the methods are independent, and for the less common but still typical
case where some initializations need to be performed after the object has been
otherwise fully initialized.  There is no need for method shadowing since each
method initializes things relevant to its own class.

A specialized type of method-combination is used in which unqualified methods
behave like the :before methods of standard method-combination.  This means
the common case is handled without resorting to either qualified methods or
call-next-method.  :after methods, as in standard method-combination, are
allowed, to handle that less common case.  :around methods could be allowed,
although examination of the corresponding thing in Flavors shows that it
doesn't seem to be used for anything reasonable.

>> How many generic functions are involved in object creation, and what
are their names?

MAKE-INSTANCE is not generic; customizing it would do no good, since a client
can call a constructor instead of calling make-instance, and constructors are
not generic.

ALLOCATE-INSTANCE class initargs... is the generic function for which the
user can define methods to customize step 2.  Standard method combination
is used.

INITIALIZE-INSTANCE instance initargs... is the generic function for which the
user can define methods to customize step 4.  The specialized type of
method combination described above is used.

Both of these generic functions should not be called by the user.  They are
called by make-instance and by constructors.  After calling allocate-instance
and before calling initialize-instance, make-instance or the constructor fills
in the slots.


>>> PROPOSAL

(Information that would duplicate the answers to the philosophy questions
omitted for the sake of brevity.)

The :initarg slot-option specifies that this slot can be filled in, and
specifies the initarg name.  This slot-option can be given more than once.

There is no defclass option that specifies initargs for all the slots,
because that would endorse a particular convention for naming initargs.

The :default-initargs defclass option is followed by alternating initarg
names and forms.  If an initarg is not specified by the client nor by
a :default-initargs option in a more specific class, the form is evaluated
in the lexical environment of the defclass and the resulting value is used
for the initarg.

The class-initargs function takes a class and returns a list of initarg-name
symbols.  These are all of the initargs accepted by make-instance of that
class.

The class-slot-initargs function takes a class and returns an alist of
(initarg-name slot-name) pairs.  These are all the initargs that fill slots.

The arguments to make-instance are first a class or a class-name, followed by
alternating initarg names and values.  Make-instance and constructors return
one value, the newly created object.

Each initarg can be associated with one or more of the four steps.  Note that
these are not mutually exclusive.  Note that in general the client does not
know or care which steps an initarg is associated with.

  1. :allow-other-keys serves its usual function, as in all &key argument
     lists.  It isn't possible to define additional initargs for step 1.
  
  2. Initargs that control storage allocation are defined by defining an
     allocate-instance method that accepts the initarg as an &key argument.
     For example, in systems with areas :area is an initarg for step 2.
     The standard does not require any initargs to exist for step 2.
  
  3. Initargs that fill slots are defined by the :initarg slot-option.
  
  4. Initargs for initialization methods are defined by defining an
     initialize-instance method that accepts the initarg as an &key argument.

(defmethod initialize-instance ((object class1) &key foo)
   (f object foo))
is an example of an initialization method.  Evaluating this defmethod is
sufficient to make :foo an acceptable initarg for class1 and its subclasses.

Each initialize-instance method is called with the initargs that it accepts.
In practice this is probably implemented by calling all of the
initialize-instance methods with all of the initargs and automatically adding
&allow-other-keys to the lambda-list in defmethod, however the standard should
not guarantee this, i.e. it should not guarantee what an initialize-instance
method with an &rest parameter will see.  A constructor should be allowed to
call the initialize-instance methods directly, without going through a
generic-function dispatch.

The parameters in the lambda-list of a constructor can specify either
slot names or initargs.  The rules are as follows:

  If a parameter name is eq to a slot name, the argument fills the slot
  regardless of whether a :initarg slot-option was specified.

  Otherwise, if a parameter name is eq to an initarg name, the argument
  specifies the value of the initarg.

  If neither of the first two rules hold, and the parameter's keyword
  (if the parameter is not an &key parameter, the corresponding keyword
  is computed by the same rule used for &key parameters) is eq to an
  initarg name, the argument specifies the value of the initarg.

If none of these three rules holds, the :constructor defclass option
is erroneous.  (Because of the forward-reference rules for defclass,
this error cannot be signalled at the time the defclass is evaluated.
We can say that it is signalled at some later time when the constructor
is actually made.)


>>> REMARKS

I don't like the pseudo-words "initarg" and "initform", but in the absence of
a better name for "initform", I'm suggesting "initarg" in order to be
consistent.

To make constructor functions work in the face of forward references to
superclasses in defclass, we may need to introduce compile-flavor-methods. 
Another reason for having compile-flavor-methods was discussed at the MIT
meeting last summer.  What's a good name for it?

I think we need a way in the meta-object protocol for the code that makes
constructor functions to determine that the standard allocate-instance method
is being used, so it can be inlined.  The way Symbolics implements
constructors is such that it is much more efficient if allocate-instance is
not actually called and there is no possibility of changing the storage
representation other than by redefining the defclass.

Should there be a way to create constructor functions for anonymous classes?

If standard meta-objects are created by make-instance, rather than by
constructors, we need to decide what initarg naming convention they are
going to use.  Currently 87-002 says they are made by constructors, but
I think in a later meeting we decided to use make-instance instead.

This is a first draft and I might suggest changes myself, as well as inviting
suggestions for changes from the rest of the group.