Method Combination Proposal

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

[Please ignore the earlier version of this message.  I forgot, when writing
the examples, that we had agreed back in October that it was an error to
have only auxiliary methods, a difference from Flavors.  This version has
the examples corrected.]

Danny, your letter arrived at a good time since I've been thinking about
method combination again.  I hope we can finish up the method combination
issue this week.  I think your letter can be divided into two parts:
first: the explanation of the overall framework and the design rationale;
second: the particular syntax of the define-method-combination macro.  I
agree with pretty much everything in the first part, and I think it's
substantially improved over my version.

I don't think the second part is as good.  Looking back over the old
discussion, the motivation for the find-methods scheme was stated to be
simplicity, but in each version it gets more and more complicated.  I
don't think it really achieves its goal.  Allow me to offer a few other
criticisms, briefly:  The :qualifier-set option duplicates the
information in the arguments to find-methods, with a different syntax,
and with a different expressive power (unless one resorts to the escape
to a predicate function); this duplicate information will be highly
error-prone.  The :qualifier-set syntax doesn't seem to allow a way to
say anything about unqualified methods and doesn't seem to have the
flexibility to deal with methods with more than one qualifier.  The
examples are hard to understand since the :qualifier-set syntax used in
the :and example is not the same as the syntax described earlier, and
because the backquotes and commas are messed up.

The thing about my old proposed syntax for define-method-combination that I
never explained very well is the reason for dividing it into two parts.
The first part defines the meaning of the method qualifiers, while the
second part defines how the methods are glued together.  It's important
that the definition of the method qualifiers is all in one place, and I
think it's important not to mix it up with other stuff.  I realized last
week that most of the support functions are superfluous; this allows a
significant simplification of the design.

A survey of Symbolics object-oriented code shows that the form of standard
method combination with the primary methods in most-specific-last order is
not needed, so the standard method combination does not need to take any
arguments (another simplification relative to my earlier proposals).  The
same survey showed that most-specific-last order is quite useful with some
kinds of method combination that invoke all the methods (rather than
shadowing), specifically progn, list, append, and, and or (this list is in
decreasing order of popularity).  The survey also uncovered a user-defined
method combination type that takes &key arguments, hence my proposal still
includes arguments in the :METHOD-COMBINATION option to DEFGENERIC-OPTIONS,
even though the standard kind of method combination does not use them.

Here is my revised proposal for define-method-combination syntax.  This
goes along with the first part of Danny's message, so I won't repeat
that.  Lowercase Lisp words should be understood to be in italics.

(DEFINE-METHOD-COMBINATION name lambda-list                     [macro]
      ({(variable {qualifier-pattern}+ {keyword argument}*)}*)
  {declaration | doc-string}*
  {form}*)

name is a symbol, usable as a name for this in the :METHOD-COMBINATION
option to DEFGENERIC-OPTIONS.  By convention, non-keyword, non-NIL symbols
are usually used.

lambda-list is an ordinary lambda-list, which receives any arguments given
in the :METHOD-COMBINATION option to DEFGENERIC-OPTIONS.

A list of method-group specifiers follows.  These define all method
qualifiers that can be used with this type of method combination.  Each
specifier contains a variable, one or more qualifier-patterns, and zero or
more keyword options.  The variable is bound to a list of methods that
match at least one of the qualifier-patterns, in most-specific-first order.
Keyword options are described in the advanced facilities section below.  A
method matches a qualifier-pattern if the method's list of qualifiers is
EQUAL to the qualifier-pattern, except that the symbol * in a
qualifier-pattern matches anything.  Thus a qualifier-pattern can be the
empty list () (which matches unqualified methods), the symbol * (which
matches all methods), an undotted list, or a list that ends in a dotted
pair whose cdr is * (which matches any number of additional qualifiers).

Qualifier-patterns are tested in the order written, so if a method could
be a member of more than one method-group, it joins only the first such
group.  If an applicable method does not match any qualifier-pattern, the
system reports an error, or a warning, that the method is not appropriate
for the kind of method-combination being used.

Declarations at the head of the body are positioned directly inside of
bindings created by the lambda-list, and outside of the bindings of the
method-group variables.  If a doc-string is present, it documents the
method-combination type.

The value of the last form in the body is the Lisp form that specifies how
the methods are combined.  Erroneous conditions detected by the body should be
reported with WARN; implementations will add any necessary contextual
information to the warning.  It is permissible for an implementation to
define the functions WARN and MAKE-METHOD-CALL by enclosing the body forms
in a FLET.


MAKE-METHOD-CALL method-list &key :operator :around             [function]

Returns a form whose car is operator (default is PROGN) and whose cdr is
a list of forms that call the methods.  MAKE-METHOD-CALL can only be called
from the body of a DEFINE-METHOD-COMBINATION.

If :around is true (default is false) and the length of method-list is
greater than 1, the result is a form that calls the first method and
arranges for CALL-NEXT-METHOD to reach the rest of the methods, in the
order they appear in method-list.  In this case :operator must not be
specified.

Each element of method-list can be either a method-object or a form.  When
a form is given, it is converted into a method (conceptually by enclosing
it in a LAMBDA-expression) when necessary.

If method-list is NIL, the result is NIL.  If method-list contains one
element and PROCLAIM-IDENTITY-WITH-ONE-ARGUMENT of the operator has been
done, the result is just a form that calls the one method, with no
invocation of the operator.

As a convenience, if method-list is a method-object, it is automatically
converted to a one-element list of that method.

There are additional keywords mentioned under "Advanced Facilities".


METHOD-QUALIFIERS method                                        [function]

Returns a list of the method's qualifiers.  This is particularly useful
when the body forms perform additional filtering or processing of the
method-group lists.  For example,

     (setq methods (remove-duplicates methods
                                      :from-end t
                                      :key #'method-qualifiers
                                      :test #'equal))


PROCLAIM-IDENTITY-WITH-ONE-ARGUMENT symbol                      [function]

Tells the object system that a form (symbol form1) can be optimized to just
form1.  Note that if symbol is a function (rather than a special operator)
the optimization is only strictly valid if form1 returns exactly one value.
This is not checked.  PROCLAIM-IDENTITY-WITH-ONE-ARGUMENT is intended to be
called at top level and is not intended to affect the compiler.  Its main
purpose is to enable the optimization of using an existing method as the
effective method.  The object system's optimizer knows about PROGN,
MULTIPLE-VALUE-PROG1, and MULTIPLE-VALUE-PROG2, as well as any operators
proclaimed with PROCLAIM-IDENTITY-WITH-ONE-ARGUMENT.


MULTIPLE-VALUE-PROG2 form result-form {form}*                   [macro]

The obvious extension of MULTIPLE-VALUE-PROG1.


>>> EXAMPLES:

;The default method-combination technique
(define-method-combination standard ()
        ((around (:around))
         (before (:before))
         (primary ())
         (after (:after)))
  ;; Must have a primary method
  (unless primary
    (warn "A primary method is required."))
  (make-method-call `(,@around
                      (multiple-value-prog2
                        ,(make-method-call before)
                        ,(make-method-call primary :around t)
                        ,(make-method-call (reverse after))))
                    :around t))

(define-method-combination and ()
        ((methods () (:and)))
  (make-method-call methods :operator 'and))

;A more complete version of the preceding
(define-method-combination and (&optional (order ':most-specific-first))
        ((around (:around))
         (primary () (:and)))
  ;; Process the order argument
  (case order
    (:most-specific-first)
    (:most-specific-last (setq primary (reverse primary)))
    (otherwise
      (warn "~S is an invalid order.~@
    :most-specific-first and :most-specific-last are the possible values."
            order)))
  ;; Must have a primary method
  (unless primary
    (warn "A primary method is required."))
  (make-method-call `(,@around
                      ,(make-method-call primary :operator 'and))
                    :around t))


>>> SHORT FORM:

(DEFINE-METHOD-COMBINATION name operator [doc-string])          [macro]

Defines name as a type of method combination that produces a Lisp form
(operator method-call method-call...).  Operator can be the name of a
function, the name of a macro, or the name of a special form.  This
alternate syntax of DEFINE-METHOD-COMBINATION is provided for convenience
and is recognized when the second subform is a non-null symbol.  By
convention, name and operator are often the same symbol, but this is not
required.

This method-combination type recognizes unqualified methods and methods
with one qualifier, the keyword symbol with the same name as name, as
primary methods.  It also recognizes methods with one qualifier, :around,
as auxiliary methods that behave the same as :around methods in standard
method combination.

A method-combination type defined this way always takes one optional
argument, order, defaulting to :most-specific-first.  A value of
:most-specific-last reverses the order of the primary methods, without
affecting the order of the auxiliary methods.

The last example above is, in fact, an example of the macro expansion of
the short form expression (define-method-combination and and).


>>> ADVANCED FACILITIES:

These facilities could be included if we want a comprehensive standard, or
could be omitted for the time being if we prefer a stripped-down standard.
I've only included facilities whose usefulness has been demonstrated in
Flavors.

Keyword options for method-group specifiers in define-method-combination:

:ORDER -- the argument is a form evaluating to :MOST-SPECIFIC-FIRST or
:MOST-SPECIFIC-LAST.  If it evaluates to any other value, the system issues
a warning.  This is a convenience, to avoid having to write the CASE expression
seen in the last example above, and does not add any expressive power.

:DESCRIPTION -- the argument is a format-string used in (APPLY #'FORMAT
stream description (METHOD-QUALIFIERS method)).  In most cases the
description will not contain any format directives.  The description should
be concise (one or two words).  If no description is specified, a default
description is chosen based on the variable name, the qualifier patterns,
and whether this method-group includes the unqualified methods.  This
description mechanism tells programming environment tools the meaning of
method qualifiers.

Additional keyword arguments for MAKE-METHOD-CALL:

:ARGLIST list-of-forms
Uses the specified forms as the arguments to each method instead of
using the arguments to the generic function.

:APPLY list-of-forms
Same as :arglist except that the last form in the list evaluates to
a list of arguments and the call is made with apply rather than funcall.
You must not specify both :arglist and :apply.

This is a bit more complicated than it seems at first, because there are
three potentially different argument lists involved.  (1) The arguments
supplied to the generic function.  (2) The arguments passed from the
generic function to the methods.  (3) The arguments actually received by a
method.  (2) differs from (1) in certain proposals for discrimination on
optional arguments, such as the &generic-optional one and the :interface
one.  (3) differs from (2) in implememtations that have a special calling
sequence for methods.  The :arglist/:apply keywords for make-method-call
allow you to change the (2) arguments; any implementation-dependent
transformation from (2) to (3) still applies.

It is not a good idea to change arguments that are used for discrimination,
since this could cause a method to be called with an argument that does
not match its parameter specializer.

WITH-METHOD-ARGUMENT-ACCESSORS lambda-list &body body           [macro]

This macro is convenient when changing the arguments passed to the methods.
Each parameter in lambda-list is bound to a form that accesses the
corresponding method argument, and then body is executed.  These accessor
forms come from the hidden information mentioned above.  Lambda-list must
match the (2) arguments mentioned above.