fixing our problems with setf

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

Here is a revision of what I mailed out yesterday, revised to reflect
Gregor's comments.  If this differs from the CL-cleanup draft proposal
I mailed out a few minutes ago, that proposal supersedes this one.

The goal is to unify the handling of "setf functions" with the handling
of regular functions, so we don't need a proliferation of -setf versions
of defmethod, defgeneric, ensure-generic-function, fboundp, generic-labels,
with-added-methods, etc.  The major issue is that Common Lisp has not seen
a need to do this already, so the onus falls on CLOS.  In addition, we
ran into difficulty with confusion between the idea of associating a
function name with a function object (in our case, a generic function
object), and the idea of associating a way to setf a function with that
function.  This proposal clarifies the situation.

Add to Common Lisp the same concept of "setf functions" that we are
already introducing in CLOS.  Right now, Common Lisp only has "setf
macros", which are defined by define-setf-method and both forms of
defsetf.  I draw the distinction because a "setf macro" is something
that produces code (or other specifications, as in define-setf-method)
which, when evaluated, will perform the effect of an invocation of setf,
while a "setf function" is something that is called to perform directly
the effect of an invocation of setf.

Since setf functions are in a separate, but parallel, namespace from
regular functions, we need a way to name them.  The simplest way is to
allow a list (setf -name-) to be used as the name of the setf function
that is called to perform the effect of (setf (-name- ...) ...).  The
following functions, macros, and special forms defined in CLtL need to
be enhanced to accept such lists where they now accept symbols as
function names:
  compile
  defun
  disassemble
  documentation
  fboundp
  flet
  fmakunbound
  function
  labels
  symbol-function and setf of symbol-function
  trace
  untrace
  and the declarations ftype, function, inline, and notinline

This makes the name of symbol-function a bit obsolete, but I do not
propose to introduce a new function to replace it.  The discrepancy is
not that important.

The following functions, macros, and special forms defined in CLOS need
to be enhanced in the same way:
  defgeneric
  defmethod
  ensure-generic-function
  generic-flet
  generic-labels
  with-added-methods
defmethod-setf and defgeneric-setf need to be removed.

A setf function receives the new value to be stored as its first
argument.  Thus, #'(setf foo) should have one more required parameter
than #'foo, the first required parameter is the new value to be stored,
and the remaining parameters should be the same as #'foo's parameters.

A setf function must return its first argument, since setf is defined
to return the new value.

Normally one does not define both a setf function and a setf macro
for the same reading function.

A definition of a setf function can be lexically local, like a
definition of a reading function.

Normally one defines a local reading function and a local setf function
together in a single FLET, LABELS, GENERIC-FLET, GENERIC-LABELS, or
WITH-ADDED-METHODS.

In the absence of any setf macro definition, SETF of a function expands
into a call to the setf function.  This means that the setf function
only needs to be defined at run time, not compile time.

Note that in Common Lisp, setf macroexpansion is an operation on
function names, not on functions.  It differs from some dialects of
Scheme, such as T, in this respect.  This proposal does not attempt to
change that.

Note that I do not propose to introduce lexically local setf macros,
that is, a cross between defsetf and macrolet.  This does not appear to
be logically necessary.  If someone else wants this, it would certainly
not be hard to do.  The main issue is whether all three ways of defining
lexically global setf macros need local counterparts.  A secondary issue
is whether to define the meaning of defmacro or macrolet of (setf foo).
I also do not now propose to enhance the definition of global setf
macros, for example to say that (macro-function '(setf foo)) returns an
expander function that takes two arguments and returns five values.
These issues logically belong to Common Lisp, not to CLOS.

Contrary to what we already decided about the lambda-list of a
setf-function, the new value to be stored will be passed as the first
argument.

The function-defining macros defun, flet, labels, defgeneric, defmethod,
and the :method option to defgeneric, generic-flet, generic-labels, and
with-added-methods will not have a special syntax with two lambda-lists
when defining a setf function, contrary to what we decided before.  The
programmer writing a setf-function must know to insert the new-value
parameter at the front of the lambda-list.

The remaining issue is a scoping issue.  We have introduced lexically
local setf functions, where before Common Lisp only had lexically global
setf macros.  Thus the namespace of setf operators has been extended to
have a lexical component, just like the namespace of regular operators.
(Recall that "operator" means the union of functions, macros, and
special forms).  Regular functions and setf functions naturally come in
pairs, but since they are defined separately we have to specify what
happens in various cases where only one is defined at a given lexical
contour.  The following rules for the behavior of SETF suffice; note
that these rules are ordered and the first rule to apply supersedes any
later rules.  These rules are a consistent extension of the current
behavior of Common Lisp and the Cleanup committee's resolution of issue
GET-SETF-METHOD-ENVIRONMENT.  Only rule 4 is new with this proposal.

Rules for the macroexpansion of (setf (foo x) y):

(1) If the function-name foo refers to the global function definition,
rather than a locally defined function or macro, and if there is a
setf-macro defined for foo, use the setf-macro to compute the expansion.

(2) If the function-name foo is defined as a macro in the current scope,
use macroexpand-1 to expand (foo x) and try again.

(3) If the function-name foo is defined as a special form in the current
scope, signal an error.

(4) Expand into the equivalent of
    (let ((#:temp-1 x)
	  (#:temp-2 y))
      (funcall #'(setf foo) #:temp-2 #:temp-1))

Note that rule 4 is independent of the scope of the function name
(setf foo) and does not care if that scope is different from the scope
of the function name foo.  This allows some nonsensical programs to
be written, but does not seem harmful enough to justify making the
rules more complicated.

Example (for Patrick):
(defmethod (setf subseq)
	   ((new-value vector) (sequence vector) start &optional end)
  (unless end (setq end (length sequence)))
  (setq end (min end (+ start (length new-value))))
  (do ((i start (1+ i))
       (j 0 (1+ j)))
      ((= i end) new-value)
    (setf (aref sequence i) (aref new-value j))))