Not about Issue: LET-TOP-LEVEL (version 1)

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

    Date: Wed, 2 Mar 88 21:19:16 gmt
    From: Jeff Dalton <jeff%aiva.edinburgh.ac.uk@NSS.Cs.Ucl.AC.UK>

I've extracted from this message a side-discussion, about the meaning
of Common Lisp's specification that except for a few primitive special
forms, all special forms are really macros that expand into something
other than special forms.

    Some implementations do not expand DEFUN into anything so reasonable, or
    else define DEFUN as a special form that is not equivalent to the macro
    expansion.  They may thus end up with DEFUN semantics that are different
    from the semantics of functions produced by the appropriate SETF of
    SYMBOL-FUNCTION.

    I would say these implementations are incorrect, because of the
    description of DEFUN on page 67 of CLtL, where a LAMBDA expression for
    the function is explicitly given and where it is specified that that
    function is "defined in the lexical environment in which the DEFUN form
    was executed", and because the macro definition of a macro implemented
    as a special form is supposed to be equivalent to the special form
    definition (CLtL, p. 57)....
    ....In addition there is a more general problem
    about macros that often occurs with defun -- nothing seems to rule out
    expansions to expresions like (SYSTEM::DO-A-DEFUN '(DEFUN ...)).

Right.  I've long felt that what CLtL says on page 57 is bankrupt.  More
politely, it sounded like a nice idea, but when it comes to implement
it, you realize that the language does not contain all the primitives
needed to implement these macro expansions in a way that can truly be
understood by a portable program-understanding program.  Furthermore,
as a practical matter it's unlikely that the implementations can all
agree with each other on the semantics of things like DEFUN at this
level of detail, because this impinges on the program development
environment; it's outside the domain of the language proper.

I think it would have been smarter to have more special forms.  It's
easier to write a portable program-understanding program that recognizes
DEFUN, than to write one that recognizes the zillions of different
things that DEFUN might macroexpand into.  Keeping the interface in
terms defined by CLtL ensures that the semantics that the
program-understanding program needs to be aware of are visible, and the
program development environment semantics that it's not interested in
are kept out of the way, rather than become visible as function calls to
implementation dependent functions.

Alternatively, we could have made a clean separation between the language
and the program-development environment, and defined as part of the
language all the hooks for the program-development environment.  Then
we could have precisely specified the expansion of all these macros; not
just said "implementors are encouraged", but specified precisely what
the macros do.  For example, we could have required DEFUN to be
implemented exactly as
  (defmacro defun (name lambda-list &body declarations-and-forms
		   &environment env)
    (multiple-value-bind (declarations documentation body)
	(parse-declarations declarations-and-forms env)
      `(progn
	 (program-development-environment:begin-definition
	    ',name 'function)	 
	 (setf (symbol-function ',name)
	       (function (lambda ,lambda-list
			   (declare (name ,name))
			   (declare (documentation ,documentation))
			   ,@declarations
			   ,@body)))
	 (setf (documentation ',name 'function) ,documentation)
	 (compiler:notice-function-definition
	   ,name ,lambda-list)
	 (program-development-environment:end-definition
	   ',name 'function)
	 ',name)))
Note all the things here that aren't in Common Lisp currently.  I
don't guarantee that deeper thought wouldn't change the above expansion.

I think the above alternative approach would produce, in the end, a
better language.  However, I suspect the amount of work required to get
there would be impractically large for either the present organizational
structure or the organizational structure that originally defined
Common Lisp.

A place where we went wrong in designing Common Lisp was to think that
the problem in writing a program-understanding program was in making it
understand all the special forms, and in thinking that that was a
syntactic problem, because special forms have idiosyncratic syntax.
It's true that it takes some code to understand all the special forms.
But that's just syntax, and it's easy.  The real issue is idiosyncratic
semantics, and minimizing idiosyncratic syntax does nothing to help
with that problem, and in fact makes it worse by obscuring it and by
leading people to think at first that the problem has been solved
already.

    Specifically, consider the implementation note on page 58.  The
    restriction on what may appear in macro expansions is insufficient to
    "ensure that the expansion can be processed ... in an implementation-
    independent manner".  But what would be sufficient?  A necessary
    condition is that all implementation-specific special forms be
    accompanied by equivalent (modulo efficiency and what else?) macro
    definitions so that any expression can be "fully expanded" so as to
    contain no implementation-specific special forms.  CLtL does require
    equivalent macros for special form implementations of operations
    defined as macros in Common Lisp, but does not say anything about
    other special forms an implementation may use.  Such a requirement
    should be added.  However, for sufficiency something must rule out
    functions of the DO-A-DEFUN sort (special forms in disguise).

I agree with you.  My comments above are a restatement of the same
concerns, I think.

    Perhaps this is being addressed as a compiler issue?

Not as far as I am aware.  It's really not a compiler issue anyway:
if we only had an interpreter the issue would be exactly the same.
The meta-issue is whether to the goals on pp.1-3 of CLtL the community
wishes to add a goal of precisely defined, portable semantics for
things like DEFUN.  If not, the half baked attempt on pp.57-8 should
be removed.  If so, a lot needs to be added.  I'm not sure any of
the existing committees are chartered to work on this.