Some open issues

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

    Date: 26 Oct 87  1130 PST
    From: Dick Gabriel <RPG@SAIL.Stanford.EDU>

Note: I haven't looked at today's draft yet.  But here are some comments
on open issues, some of them based on talking with Danny Bobrow and some
of them based on questions from Dick Gabriel.

    1. What is the story on the idea of built-in? 

The key point here is that CLOS does not specify exactly which types are
built-in.  We split "standard type class" into two things: the types
that are defined in Steele's book, which are candidates for being
built-in, and the built-in-class metaclass, which defines the ones that
actually are built-in in a given implementation.  So we're allowing any
Common Lisp data type to be either built-in or a standard-class in a
particular implementation, but you can check the metaclass to see if it
is built-in.  Also we're allowing implementations to make new
implementation-dependent built-in types, so the only thing that's
guaranteed -not- to be built-in are the standard classes you define
yourself.  Also the structure classes I guess.

Danny points out that in the future we might want to make an incompatible
change to Common Lisp to require some particular type that is already in
Steele's book to be a standard-class rather than a built-in-class, whereas
in the initial version of CLOS this will be at the implementation's option.
For instance, one might want to do this to streams.  Moon speaking again:
I don't see any CLtL types that are indubitably candidates for this, so
I think we can put off that issue.

It's implied by a paragraph on p.1-17, but it could be made clearer
in the commentary on Figure 1-1 that there can be additional,
implementation-dependent elements in the class precedence lists
of these types.  CLOS only requires that the elements listed by
present and be in the order specified.

I'll try to send out in a separate message a cleanup of the built-in-class
message that Danny and Gregor sent, to be more appropriate to go into the
document.  It's late, I guess that message won't go out until Tuesday.

    2. UPDATE-INSTANCE-STRUCTURE and CLASS-CHANGED needed to be parallel
    in how and who does initialization. Danny proposes that default primary methods
    handle it in both cases. My view is that unless there is some meta-object
    way to eliminate the initialization step, the only way is by putting 
    them into a primary method that can be shadowed or eliminated.

    In talking to Danny last night, I came to believe that he thinks there
    would be no meta-object way to get at the initialization in this step.

Actually, all of UPDATE-INSTANCE-STRUCTURE, CLASS-CHANGED, and
INITIALIZE-INSTANCE can usefully be consistent.  This means that
for all three of these the default primary method is in charge of
initializing newly added slots that are not already initialized,
by evaluating the :initform and storing the result into the slot.
For UPDATE-INSTANCE-STRUCTURE and CLASS-CHANGED, values of slots
that already existed are transported into the updated instance
by something "at the bit level", before the generic function is
called.  All three of these default primary methods are permitted
to conspire with the bit level to optimize out initforms that
neither produce nor depend on side-effects, by storing them into
the slots before the generic function is called rather than in the
default primary method.  This optimization is still valid, but
not useful, when the default primary method has been shadowed.
Also implementations are permitted to open-code the body of the
default primary method in a way that makes it customized to a
specific class, rather than being forced to obtain information
from the class object and interpret it.  However, this does not
change the user's ability to shadow the default primary method,
thus this optimization has to be done by open-coding rather than
by automatically generating a customized method on a more specific
class than STANDARD-OBJECT (I think this is grotesque, but it's the
only way to preserve the property that apparently people find highly
desirable of being able to turn off the evaluate-the-initforms
behavior without using a :AROUND method to turn everything off).

Changes from what we have now:
  INITIALIZE-INSTANCE is not changed
  UPDATE-INSTANCE-STRUCTURE -- the first bullet in the Remarks goes
   away.  Preserving the values of old slots is done at the bit level.
  CLASS-CHANGED -- responsibility for evaluating the :initforms moves
   from CHANGE-CLASS to CLASS-CHANGED (in the latest version of the
   document I have, chapters 1 and 2 are inconsistent about this).
   Responsibility for the rest of the structural transformation
   remains with CHANGE-CLASS.

I think this shows that the use of standard method combination is wrong
for all three of these, because we have to keep telling people "don't
use :before methods, you'll be surprised, and don't use primary methods
unless you really mean to shadow the primary method."  However, I know
I'm not going to change any minds on that point.  If I was designing
this in a vacuum, I think I would make all three of these use PROGN
:MOST-SPECIFIC-LAST method combination, and then the user would not be
able to tell, except by using :AROUND methods, whether the initforms
are done by the default primary method or by the caller, and there
would be no way to write a method that does something unexpected,
so we wouldn't have to warn people about pitfalls.

    3. What's the resolution on the CPL for NULL, SYMBOL, LIST et al?

The only one that's broken is the CPL for NULL.  Either (null symbol list
sequence t) or (null list sequence symbol t) would be acceptable to me
and to Danny; the real problem is the (null list symbol sequence t) that
we have now.  But we could say that was just a typo.  Danny said he wanted
to check with Gregor in case G. had any strong opinion.  I notice the
latest draft has (null symbol list sequence t) and I'm willing to live
with that.

    4. What terminology for ``default primary method'' should we use?

We should define a class named STANDARD-OBJECT (this was the best name
Danny and I came up with), which is the least specific class, except for
T, in the CPL of any instance whose meta-class is STANDARD-CLASS.  The
reason for existence of STANDARD-OBJECT is to allow there to be default
methods for instances of standard classes, that don't apply to instances
of other classes (built-in, structure, what have you).  I hope with some
tweaking of the English the above paragraph can go into the document.
Most of the CPL examples will need to have STANDARD-OBJECT added to them.

There are several places in the spec which refer to the "default primary
method" when in fact they should be referring to the "primary method on
standard-object".  The examples of this I can find right now are:

  update-instance-structure (pg 1-13)
  class-changed (pg 1-15)
  initialize-instance (pg 1-34)

(For class-changed, we have to say which parameter is specialized.
I think the answer is, "both").

The point in these examples is that these "system supplied" methods are
actually providing default behavior for instances with metaclass
STANDARD-CLASS, so they should be on the class STANDARD-OBJECT, that's
what it's there for.

It's also likely that an implementation would want STANDARD-OBJECT
methods for PRINT-OBJECT and DESCRIBE, in addition to the T methods.
However, for these generic functions we say only that there is always
an applicable method, we don't say how the system-supplied methods
are modularized.

(Above text was Bidenized from Gregor in part.)

    5. Are methods side-effected when method functions change?

One issue is, why should methods be different from classes and generic
functions?  Both defclass and defgeneric side-effect an existing object,
they don't create a new object if there already is one with the name.

Another issue seems to be what happens if an error is signalled, e.g. due
to noncongruent lambda lists?  Has the existing method object already
been side-effected when that happens?  In general, CLtL never says what
the state of the program is at the time an error is signalled, and would
allow an implementation to choose whether to do the error-checking first
or the side-effects first.  However, we might want CLOS to be a bit
cleaner.  I don't see any reason not to do the error checking before the
side-effects.

We should see if Gregor comes up with some reason at the meta-object level
why it's really important that methods be inconsistent with generic functions
and classes.  But in the absence of that, I think we ought to say that
defmethod side-effects an existing method object.  We haven't said what
the macro expansion of defmethod is exactly, and perhaps that's meta-object
level business, but I think it might involve a function similar to
ensure-generic-function that takes enough keyword arguments to describe
the method, and either makes a new object or updates an existing one.

    6. The reason that DEFMETHOD shouldn't specialize functions like CAR defaultly
    is that the compiler has already compiled a pile of stuff knowing what's up.
    WITH-ADDED-METHODS created lexical environment in which the compiler can know
    all about that gives. Therefore, I think that if WITH-ADDED-METHODS is asked
    to extend an ordinary function, it should make that function the method
    function for the default primary method.

Another point is that this effect can be simulated with GENERIC-FLET, by
writing a default method that calls the old definition of the function
name.  This suggest that there can't be much harm in allowing
WITH-ADDED-METHODS to do this.  My only problem in evaluating this is that
I can't figure out what WITH-ADDED-METHODS is good for in the first place.
However I agree that WITH-ADDED-METHODS should be allowed to extend an
ordinary function.

    7. What about the class named OBJECT? How does it fit in?

STANDARD-OBJECT, see above.

    8. If CALL-NEXT-METHOD is globally bound to a function that signals
    an error, the name CALL-NEXT-METHOD has indefinite scope. Is this
    what we mean to say? 

Well, the name has indefinite scope, but a particular binding of the name
to a useful value (not the global one that just signals an error) only has
lexical scope.  I don't know the proper way to say this, but I think it is
what we mean to say.

    By your comment about CALL-NEXT-METHOD's extent including the arguments
    to the method, do you mean this:

  (defmethod foo ((x1 c1) ... (xn Cn)
                  &optional (next-fun #'(lambda () (call-next-method))))...)

    I guess this is ok.

Yes, that's what I mean.  I guess the precise language would be that the
scope of the binding of CALL-NEXT-METHOD to a useful function includes
forms that appear in parameter specifiers of the defmethod, of course with
the exception of forms that appear in parameter specializer names of the
defmethod, since those latter forms are evaluated at definition time, not
at generic function call time.

Implementing this is epsilonically harder than allowing call-next-method
only in the body, but it seems to be what people will expect, and I'm
sure it's not enough harder to implement for implementation difficulty
to be a consideration.

By the way, this could also be written more simply

  (defmethod foo ((x1 c1) ... (xn Cn)
                  &optional (next-fun #'call-next-method))...)

which is why we changed call-next-method from a macro to a function.

Another open issue is that initargs you have to pass to make-instance
when making a standard-method.  I think the best bet is that in addition
to the initargs already documented, you must pass :lambda-list, which
is a lambda-list from which the implementation is permitted to derive
the number of required and optional arguments, the presence of rest/key
arguments, the keyword names, for lambda-list congruence computation.
An implementation is also permitted to ignore the :lambda-list argument
and get this information from the :function argument, if in that
implementation a lambda-list can be retrieved from a function (CLtL
doesn't require that).  I think the lambda-list can be anything that
meets these requirements, so it can be the specialized-lambda-list that
appears in the defmethod form, or it can be an ordinary function lambda
list, or it can be the type of stripped down lambda-list that can
be used with defgeneric.  I think it's better for the derivation of
lambda-list congruence information from lambda-lists to be in the system,
rather than being reimplemented by every user, hence I suggest a lot
of freedom in what kind of lambda-list the initarg can accept.  The
alternative would be to require the user to pass two numbers, a boolean,
and a list of keyword names, but I'd rather not do it that way.