Another try on object creation

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

Draft of a new object creation proposal based on CLOS subcommittee
discussions July 2, 1987.

-foo- means the word foo in italics.  FOO means the word foo in boldface.


ONE NEW IDEA YOU HAVEN'T SEEN BEFORE

There is a new lambda-list-keyword, &METHOD-KEY, which is only valid in
DEFGENERIC-OPTIONS and DEFGENERIC-OPTIONS-SETF.  The syntax of the lambda-list
in these macros becomes

  ({-var-}+
   [&optional {-var-}*]
   [&rest -var-]
   { [&key {-var- | ((-keyword- -var-))}* [&allow-other-keys]]
     | [&method-key [&allow-other-keys]] })

The meaning of &METHOD-KEY is that the specific set of named arguments
accepted by the generic function varies depending on the positional arguments.
The named arguments accepted by the generic function for a particular call are
the union of the named arguments accepted by the applicable methods.  There is
no attempt to exclude methods that are applicable but are not actually called.
Note that in standard method combination, all applicable methods are
potentially callable, if CALL-NEXT-METHOD is used.  A method that has &REST,
but not &KEY, does not affect the set of acceptable named arguments.  If the
lambda-list of any applicable method or of the DEFGENERIC-OPTIONS has
&ALLOW-OTHER-KEYS, all named arguments are accepted by the generic function.

The implementation of &METHOD-KEY is in two parts:  The macro expansion of
DEFMETHOD, when the generic function uses &METHOD-KEY, is altered to save a
list of the acceptable named-argument names in a slot of the method object and
to put &ALLOW-OTHER-KEYS into the lambda-list of the function.  If the
lambda-list already contains &ALLOW-OTHER-KEYS, then that symbol is stored in
place of the list of acceptable named-argument names (which would be
infinite).  The function METHOD-NAMED-ARGUMENTS retrieves this list or symbol.
Secondly, the generic-function-to-method dispatching mechanism must check the
validity of the argument list when the generic function uses &METHOD-KEY and
does not use &ALLOW-OTHER-KEYS.  This is accomplished by collecting the
acceptable named-argument names from the applicable methods and checking the
arguments against the union of those lists.  If for any applicable method
METHOD-NAMED-ARGUMENTS returns &ALLOW-OTHER-KEYS, the whole check is skipped.


CONCEPTS TO BE ADDED TO 87-002

-named argument-.  We use the term "named argument" instead of "keyword
argument" (as in CLtL) for &key arguments, because CL-Cleanup issue
KEYWORD-ARGUMENT-NAME-PACKAGE has stated that the names of &key arguments
do not have to be keyword symbols.

-named argument name-.  The symbol that identifies a named argument in
an argument list.  This is typically a keyword, but is not required to be.
The named-argument name should not be confused with the variable name
of the parameter variable.  These two symbols typically have the same
name and are typically in different packages, but that is not required.

-initarg-.  An initarg (initialization argument) is a named argument that
can be used to control object creation and initialization.  The &key
arguments to MAKE-INSTANCE are initargs.  Each initarg has a name, which is
a symbol, and may have a value, which is any Lisp object.  It is often
convenient to use keyword symbols to name initargs, but the name of an
initarg can be any symbol, including NIL.

-initarg list-.  An initarg list (initialization argument list) is a list
of alternating initarg names and values.  Its structure is identical to a
property list and also identical to an &key argument list.  As in those
lists, if an initarg name appears more than once in an initarg list, the
leftmost occurrence supplies the value and the remaining occurrences are
ignored.  The arguments to MAKE-INSTANCE, after the first, are an initarg
list.  As in an &key argument list, :ALLOW-OTHER-KEYS can appear in an
initarg list, and if its value is non-NIL, error-checking of initarg names
is disabled.

-slot-filling initarg-.  An initarg associated with a slot.  If the initarg
has a value, the value is stored into the slot of the newly-created object,
overriding any initform associated with the slot.  -(What about shared
slots?)-  A single initarg can fill more than one slot.

-method-implemented initarg-.  An initarg associated with a method.  When
an object is created, the method is called with the initarg's value as an
argument and the method uses the value in any way it likes.  If the initarg
has no value, the method's lambda-list supplies a default value.  A single
initarg can be implemented by more than one method.  An initarg can be both
slot-filling and method-implemented.


CHANGES TO 87-002 FEATURES

DEFCLASS gets a new :INITARG slot option, which is followed by a symbol.
The symbol becomes the name of a slot-filling initarg for this class.

DEFCLASS gets a new :DEFAULT-INITARGS option, which is followed by an initarg
list.  Each value in this list is a form that is evaluated by MAKE-INSTANCE if
the initarg does not already have a value.  The forms are evaluated in the
lexical environment in which the DEFCLASS form was evaluated.

Method-implemented initargs are defined simply by defining a method for
INITIALIZE-INSTANCE or ALLOCATE-INSTANCE; each named-argument name in the
method's lambda-list becomes a method-implemented initarg for all classes for
which this method is applicable.

Initarg inheritance: The effective set of slot-filling initargs for a class C
is the union of the slot-filling initargs defined by C and its superclasses.
The effective set of method-implemented initargs for a class C is determined
by method inheritance.

Default-initargs inheritance: [same as for the :INITFORM slot option]

Changes to Lambda-list Congruence Rules (p.1-20): Rules 1, 2, and 6 remain
the same, except for wording problems, while rules 3-5 need to be replaced
to implement &METHOD-KEY and to fix the interaction among &KEY, &REST, and
&ALLOW-OTHER-KEYS.  The new rules for congruence are the following:

  These rules define the congruence of a set of lambda-lists, including the
  lambda-list of each method for a given generic function and the lambda-list
  specified with DEFGENERIC-OPTIONS, if present.  For -SETF methods, these
  rules apply to the effective lambda-list produced by combining the two
  specified lambda-lists according to the rules on page nnn.

  1. Each lambda-list must have the same number of required parameters.

  2. Each lambda-list must have the same number of optional parameters.
  Each method can supply a different default for an optional parameter.

  3. If any lambda-list uses &REST, &KEY, or &METHOD-KEY, each lambda-list
  must use one or more of these.  Note that &METHOD-KEY is only valid in
  DEFGENERIC-OPTIONS.

  4. If the DEFGENERIC-OPTIONS does not use &METHOD-KEY, or there is no
  DEFGENERIC-OPTIONS, each method that uses &KEY and does not use
  &ALLOW-OTHER-KEYS must specify the same named-argument names.

  5. The use of &ALLOW-OTHER-KEYS need not be consistent across lambda-lists.

  6. The use of &AUX need not be consistent across methods.

Rules when initargs are duplicated:

  The :INITARG slot-option may be specified more than once for a given slot.

  A single initarg can initialize more than one slot if the same initarg name
  appears in more than one :INITARG slot-option.

  If two initargs that initialize the same slot, with the same or different
  names, are given in the arguments to MAKE-INSTANCE, the leftmost of these
  initargs in the initarg list prevails.

  If two different initargs that initialize the same slot have default values,
  the initarg that appears in a :INITARG slot-option in the most specific
  class prevails, or if they appeared in the same class, the one leftmost in
  the slot-options list prevails.

  It is valid for a given initarg name to be defined more than once as a
  slot-filling initarg, as a method-implemented initarg, or both.


NEW FUNCTIONS TO BE ADDED

In this section, I have only sketched each function, for the sake of brevity.
Full writeups can be constructed once the overall framework has been agreed
upon.  Functions are in alphabetical order.  By coincidence, all functions
listed are generic and expected to specialize on their first argument.

(ALLOCATE-INSTANCE class &method-key &allow-other-keys) => instance

Metausers can replace the system-supplied, implementation-dependent
method for this.

(CHECK-INITARGS class initarg-list)

Metausers could replace the system-supplied method that implements the
normal rules for initarg validity.

(CLASS-ALL-INITARGS class) => list of initarg names (includes inherited)

(CLASS-DIRECT-INITARGS class) => list of initarg names

(CLASS-ALL-INITARG-DEFAULTS class)
   => ((initarg-name default-value-function)...)

(CLASS-DIRECT-INITARG-DEFAULTS class)
   => ((initarg-name default-value-function)...)

(CLASS-ALL-SLOT-INITARGS class) => ((initarg-name slot-name...)...)

(CLASS-DIRECT-SLOT-INITARGS class) => ((initarg-name slot-name...)...)

(COMPUTE-APPLICABLE-METHODS generic argument-list) => list of methods

(DEFAULT-INITARGS class initarg-list) => initarg-list

The system-supplied method implements the :DEFAULT-INITARGS class option.
[This could specialize on instance instead of class, but I don't
see any point to that.]

(ENCACHE-INITARG-INHERITANCE class)

This is called by the system at least once before a class is instantiated,
and is called again whenever anything relevant changes.  System-supplied
methods for this conspire with methods for CHECK-INITARGS, etc., to make
MAKE-INSTANCE faster.  Users with their own caching needs can add methods
for this generic function.

(ENCACHE-METHOD-INHERITANCE class)

(ENCACHE-SLOT-INHERITANCE class)

(INITIALIZE-INSTANCE instance &method-key &allow-other-keys)

Users define :AFTER methods for this to create method-implemented initargs.
The primary method for this is system-supplied and takes care of the
slot-filling initargs.

(MAKE-INSTANCE class &key -initargs-...) => instance

(METHOD-NAMED-ARGUMENTS method) => list of symbols or &ALLOW-OTHER-KEYS

(SLOT-BOUNDP instance slot-name) => boolean

Allows writing INITIALIZE-INSTANCE methods that only initialize slots if they
haven't been initialized already.

(SLOT-MAKUNBOUND instance slot-name) => NIL


PROCEDURAL DEFINITION OF MAKE-INSTANCE

MAKE-INSTANCE behaves as if it was defined as follows, except that certain
optimizations are permitted, as detailed below.

(defmethod make-instance ((class standard-class) &rest initargs)
  (setq initargs (default-initargs class initargs))
  (check-initargs class initargs)  
  (let ((instance (apply #'allocate-instance class initargs)))
    (apply #'initialize-instance instance initargs)
    instance))

(defmethod make-instance ((class-name symbol) &rest initargs)
  (apply #'make-instance (symbol-class class-name) initargs))

Optimization is possible, including inlining and constant-folding of method
lookup and method bodies, provided that the programming environment either
prohibits redefining these methods or updates everything when they are
redefined.  A possible example implementation would be that MAKE-INSTANCE has
a separate method for every class, which is automatically written and compiled
by the system.

This optimization relies on the ENCACHE-INITARG-INHERITANCE generic function
and some unpublished slots of STANDARD-CLASS.

Because of optimization, methods for the generic functions listed may not
actually be called on every call to MAKE-INSTANCE, or may not receive
exactly the arguments that would be expected.  For example, CHECK-INITARGS
may actually be called before DEFAULT-INITARGS rather than after, if it has
already been determined that the default initargs will pass CHECK-INITARGS.

Additional explicit details of permissible optimization will need to be set
forth.


MEETING OF STATED DESIGN GOALS

Lexical proximity of concepts--the declaration of an initarg as valid,
the specification of what it does, and the default if it is not supplied
are all together, in a slot specifier or in a method lambda-list.

Simple ways to do simple things--slot-filling initargs don't require the
user to write any code.  Method-implemented initargs work just like
ordinary function arguments as far as the user is concerned.

Minimal number of new languages--the only addition to Common Lisp is
&METHOD-KEY.

Ability to do everything at some level--the underlying procedural level
is available.  Functions to access all the direct and inherited
information are documented.

Underlying mechanism is exposed so user can use it for other things,
rather than abusing instance creation as the only way to access the
mechanism--the combination of &method-key, method-named-arguments,
compute-applicable-methods, and encache-initarg-inheritance provides
everything the user needs.