issue DEFINING-MACROS-NON-TOP-LEVEL, version 8

[sandra%defun@cs.utah.edu (Sandra J Loosemore)]

The definition of "top-level" has been moved to issue 
EVAL-WHEN-NON-TOP-LEVEL.  Otherwise this writeup is mostly 
unchanged.

Forum:		Compiler
Issue:		DEFINING-MACROS-NON-TOP-LEVEL
References:	CLtL p. 66-70, 143
		Issue EVAL-WHEN-NON-TOP-LEVEL
		Issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS
		Issue COMPILER-LET-CONFUSION
Category:	CLARIFICATION, ENHANCEMENT
Edit History:   6-May-88, V1 by Sandra Loosemore
		9-Jun-88, V2 by Sandra Loosemore
		12-Sep-88, V3 by Sandra Loosemore (fix garbled section 4)
                21-Sep-88, V4 by Sandra Loosemore (clarify section 5)
		16-Dec-88, V5 by Sandra Loosemore (major restructuring)
		31-Dec-88, V6 by Sandra Loosemore (wording clarifications)
		07-Jan-89, V7 by Sandra Loosemore (add example)
		09-Mar-89, V8 by Sandra Loosemore (more restructuring)
Status:		Ready for release


Problem Description:

CLtL leaves the interpretation of defining forms such as DEFMACRO and
DEFVAR that appear in other than top-level locations unclear.

On page 66, it is stated: "It is not illegal to use these forms at
other than top level, but whether it is meaningful to do so depends on
context.  Compilers, for example, may not recognize these forms
properly in other than top-level contexts".  At least one implementation 
has interpreted this to mean that it is permissible to simply refuse
to compile defining macros that do not appear at top-level.


Proposal: DEFINING-MACROS-NON-TOP-LEVEL:ALLOW

(1) Remove the language from p. 66 of CLtL quoted above.  Clarify that
while defining macros normally appear at top level, it is meaningful
to place them in non-top-level contexts and that the compiler must
handle them properly in all situations.  However, the compile-time side
effects described in issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS 
only take place when the defining macros appear at top-level.

(2) Remove the language on p. 145 of CLtL, which states that macro
functions are always defined in the null lexical environment.  Clarify
that all defining macros which create functional objects (including
DEFMACRO, DEFTYPE, DEFINE-SETF-METHOD, and the complex form of
DEFSETF, as well as DEFUN) must ensure that those functions are
defined in the lexical environment in which the defining form is
evaluated.

(3) Specify that top-level forms in a file being compiled are
guaranteed to be processed sequentially.  The order in which
non-top-level subforms of a top-level form are processed by the
compiler is explicitly left unspecified.


Rationale:

This proposal makes the rules for when defining macros cause
compile-time side effects to be exactly the same as the rules for when
(EVAL-WHEN (COMPILE) ...) causes compile-time evaluation.  This
provides a simple implementation technique.

Item (3) serves two purposes.  First, it guarantees users that
compile-time side-effects from top-level EVAL-WHEN forms or defining
macros will happen in the correct order; programmers can depend upon
the compile-time side-effects of a top-level form being visible during
the compilation of subsequent forms.  Second, it allows compilers to
perform certain kinds of source-to-source transformations that change
the order of subforms.

For instance, the following example from CLtL

  (let ((old-count *access-count*))
      (unwind-protect
          (progn 
	      (incf *access-count*)
	      (perform-access))
	  (setq *access-count* old-count)))

is entirely equivalent to:

  (let ((old-count *access-count*))
      (let ((thunk  #'(lambda () (setq *access-count* old-count))))
          (unwind-protect
	      (progn
	          (incf *access-count*)
		  (perform-access))
	      (funcall thunk))))

(This is a real example from the A-Lisp compiler, which implements
UNWIND-PROTECT by having it push a "thunk" to perform the cleanup
actions onto the catch stack before executing the protected form.)


Current Practice:

Most implementations do allow defining macros in non-top-level places.
However, the rules for when they cause compile-time side-effects are
not always the same as those for EVAL-WHEN.  This is the case in
Lucid Common Lisp, for example.


Cost to implementors:

Implementations that currently don't compile defining macros correctly
when they appear at non-top-level will have to be changed.


Cost to users:

None.  This is a compatible extension.


Benefits:

The notion of defining macros as being somehow special when they
appear at top-level is removed, since their behavior can be explained
using EVAL-WHEN as a primitive.  Allowing defining macros to appear
anywhere instead of restricting them to certain positions results in a
cleaner language design.


Discussion:

This proposal is consistent with the behavior specified in proposal
EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL.  In particular, if the compile
time side-effects for defining macros specified in proposal
COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS:CLARIFY are implemented using
EVAL-WHEN, the "right" compiler behavior for defining macros at
non-top-level will happen automatically.
-------