fixing our problems with setf

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

The goal is to unify the handling of "setf functions" with the handling
of regular functions, so we don't need a proliferation of -setf versions
of defmethod, defgeneric, ensure-generic-function, fboundp, generic-labels,
with-added-methods, etc.  The major issue is that Common Lisp has not seen
a need to do this already, so the onus falls on CLOS.  In addition, we
ran into difficulty with confusion between the idea of associating a
function name with a function object (in our case, a generic function
object), and the idea of associating a way to setf a function with that
function.  This proposal unifies those ideas.

I believe the following proposal neatly solves the problem.

Add to Common Lisp the same concept of "setf functions" that we are
already introducing in CLOS.  Right now, Common Lisp only has "setf
macros", which are defined by define-setf-method and both forms of
defsetf.  I draw the distinction because a "setf macro" is something
that produces code (or other specifications, as in define-setf-method)
which, when evaluated, will perform the effect of an invocation of setf,
while a "setf function" is something that is called to perform directly
the effect of an invocation of setf.

As with regular functions, associated with any given name you can have a
setf function or a setf macro, but not both.  This means that one does
not define a setf function (with defmethod or defgeneric) and also call
defsetf.  The mere act of defining the setf function is enough to tell
setf what to do.

Since setf functions are in a separate, but parallel, namespace from
regular functions, we need a way to name them.  The simplest way is to
allow a list (setf -name-) to be used as the name of the setf function
that is called to perform the effect of (setf (-name- ...) ...).  The
following functions, macros, and special forms defined in CLtL need to
be enhanced to accept such lists where they now accept symbols as
function names:
  compile
  defun
  disassemble
  documentation
  fboundp
  flet
  fmakunbound
  function
  labels
  symbol-function and setf of symbol-function
  trace
  untrace

The following functions, macros, and special forms defined in CLOS need
to be enhanced in the same way:
  defgeneric
  defmethod
  ensure-generic-function
  generic-flet
  generic-labels
  with-added-methods
defmethod-setf and defgeneric-setf need to be removed.

Note that in Common Lisp, setf macroexpansion is an operation on
function names, not on functions.  It differs from some dialects of
Scheme, such as T, in this respect.  This proposal does not attempt to
change that.

Note that I do not propose to introduce lexically local setf macros,
that is, a cross between defsetf and macrolet.  This does not appear to
be logically necessary.  If someone else wants this, it would certainly
not be hard to do.  The main issue is whether all three ways of defining
lexically global setf macros need local counterparts.  I also do not now
propose to codify the definition of global setf macros, for example to
say that (macro-function '(setf foo)) returns an expander function that
takes two arguments and returns five values.

The next issue is what to do about the lambda-list of a setf-function.
We already agreed on what is the lambda-list in the expansion of
(defun (setf foo) ...) into 
(setf (symbol-function '(setf foo)) #'(lambda ...)).  Specifically,
#'(setf foo) takes one more required argument than #'foo, and the last
required argument is the new value to be stored.

The issue is whether certain function-defining macros should have a
special syntax with two lambda-lists when defining a setf function, or
should use their normal syntax.  The macros in question are defun, flet,
labels, defgeneric, defmethod, and the :method option to defgeneric,
generic-flet, generic-labels, and with-added-methods.  In favor of two
lambda-lists is that it's easier for the programmer to see which
parameter is bound to the new value to be stored.  In favor of one
lambda-list is syntactic consistency between setf functions and regular
functions.  At the meeting two weeks ago, we favored two lambda-lists,
but I now believe that that was a mistake.  The rule that the new value
is the last required argument should not be at all difficult for
programmers to understand.  Syntactic consistency is important.

The remaining issue is a scoping issue.  We have introduced lexically
local setf functions, where before Common Lisp only had lexically global
setf macros.  Thus the namespace of setf operators has been extended to
have a lexical component, just like the namespace of regular operators.
(Recall that "operator" means the union of functions, macros, and
special forms).  Regular functions and setf functions naturally come in
pairs, but since they are defined separately we have to specify what
happens in various cases where only one is defined at a given lexical
contour.

The simplest solution would be to say that (setf (-foo- ...) ...) ignores
the scope of the name -foo- when considering how to setf it.  However, this
solution has already been ruled out by the Cleanup committee's resolution
of issue GET-SETF-METHOD-ENVIRONMENT.  Specifically inside the scope of
a macrolet of foo, the expansion of (setf (foo ...) ...) proceeds by
calling the macrolet expander, regardless of whether there is a globally
defined setf macro for foo.  Similarly, inside the scope of a flet of foo,
(setf (foo ...) ...) is invalid.

To be consistent with this, I propose a suite of three rules for the
behavior of local function defining forms (flet, generic-flet,
generic-labels, labels, macrolet, and with-added-methods):

(1) A local function definition of the name foo implicitly shadows the
scope of a global setf macro definition for foo.

(2) A local function definition of the name foo implicitly shadows the
scope of any enclosing function definition of (setf foo).  When the same
function defining form defines both foo and (setf foo), the scopes of
the two names are equal regardless of their order of appearance in the
function defining form.

(3) A local function definition of the name (setf foo) does not change
the scope of any enclosing function definition of foo.

with-added-methods may require a special rule, because of its unusual
half-shadowing semantics.  I'm inclined to ignore this and say that
(with-added-methods ((foo ...)) ...) shadows any enclosing definition
of (setf foo), just like generic-labels.

Example (for Patrick):
(defmethod (setf subseq)
	   ((sequence vector) start (new-value vector) &optional end)
  (unless end (setq end (length sequence)))
  (setq end (min end (+ start (length new-value))))
  (do ((i start (1+ i))
       (j 0 (1+ j)))
      ((= i end) new-value)
    (setf (aref sequence i) (aref new-value j))))

If this meets with general approval I will recast this for the Cleanup
committee.