issue DEFINING-MACROS-NON-TOP-LEVEL, version 3

[sandra%defun@cs.utah.edu (Sandra J Loosemore)]

David Gray has pointed out that the first paragraph of section (4) in
the version of this proposal that was distributed at the last meeting
was badly garbled.  Here is a corrected version.

-Sandra


Issue:		DEFINING-MACROS-NON-TOP-LEVEL
References:	CLtL p. 66-70, 143
		Issue EVAL-WHEN-NON-TOP-LEVEL
		Issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS
Category:	CLARIFICATION, ENHANCEMENT
Edit History:   6-May-88, V1 by Sandra Loosemore
		9-Jun-88, V2 by Sandra Loosemore
		12-Sep-88, V3 by Sandra Loosemore (fix garbled section 4)


Problem Description:

CLtL leaves the interpretation of defining forms such as DEFMACRO and
DEFVAR that appear in other than top-level locations unspecified.
Resolution of other issues (EVAL-WHEN-NON-TOP-LEVEL and
COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS) now allows reasonable
semantics to be assigned to defining forms which appear at
non-top-level.


Proposal: DEFINING-MACROS-NON-TOP-LEVEL:ALLOW

(1) Clarify that while defining macros normally appear at top level,
it is meaningful to place them in non-top-level contexts and that the
compiler must handle them properly in all situations.  Remove the
language on p. 66 of CLtL which states that the compiler is not
required to recognize defining macros at other than top-level.

(2) The proposal COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS:CLARIFY
defines a model for specifying how defining macros work.  To
summarize, the expansion of the macro (rather than its expander
function) is responsible for storing information about the definition.
Compile-time side effects are typically handled by including one or
more EVAL-WHEN forms in the expansion.  A compiler may choose
some other implementation, such as treating defining macros as
implementation-specific special forms, provided that the semantics
are compatible.

(3) Defining macros which define functional objects (such as DEFUN and
DEFMACRO) must ensure that the functions are defined in the lexical
environment in which the defining macro appears.  In the model
referred to above, this would normally be implemented by producing a
FUNCTION special form in the macro expansion.  For example, the
following code causes the function BAR to be closed over the variable
X:

    (let ((x  (some-hairy-computation)))
        (defun bar (y) (+ x y)))

(4) The language on p. 145 of CLtL, which states that macro functions
are always defined in the null lexical environment, should be removed.
Instead, the defining forms DEFMACRO, DEFTYPE, DEFINE-SETF-METHOD, and
the complex form of DEFSETF, which make functional definitions
available at compile time, use the environment which is apparent at
the time of evaluation.  When calls to these macros appear in a
non-null lexical environment, an explicit (EVAL-WHEN (COMPILE) ...)
must normally be wrapped around the entire containing top-level form
to ensure that the correct lexical environment is seen at both compile
time and load time.

An example may help clarify why this is necessary.  The code fragment

    (let ((x  (some-hairy-computation)))
        (defmacro bar-macro (y) `(+ ,x ,y)))

would macroexpand into something similar to

    (let ((x  (some-hairy-computation)))
        (eval-when (eval compile load)
            (setf (macro-function 'bar-macro) 
	          #'(lambda (form env)
		        (let ((y  (second form)))
			    `(+ ,x ,y))))
            'bar-macro))

Since the rules for (EVAL-WHEN (COMPILE) ...) state that evaluation
takes place in the null lexical environment, in this situation X would
be treated as a special variable within the macro function.  However,
in the EVAL or LOAD situations, the lexical value of X would be used.
To ensure consistency, the correct definition would be:

    (eval-when (eval compile load)
        (let ((x  (some-hairy-computation)))
            (defmacro bar (y) `(+ ,x ,y))))


(5) Clarify that ``top-level forms'' are evaluable data objects read
in from an input source (such as a keyboard or disk file) by
successive calls to the function READ.  As a special case, forms
within a top-level PROGN are also considered to be top-level forms.
Specify that top-level forms in a file being compiled are guaranteed
to be processed sequentially, but the order in which subforms of a
top-level form are processed by the compiler is explicitly left
unspecified.  It is an error for user code to depend upon the
compile-time side-effects of a defining macro within the same
top-level form in which the defining macro appears.


Rationale:

The notion of a ``top-level form'' is rather confused, since the term
is used in CLtL to refer both to a place where a form may appear (what
this proposal continues to call ``top-level''), and to instances of
forms which traditionally appear there (what this proposal calls
``defining macros'').  

There has been a suggestion that the notion of a top-level form should
be extended to include forms in the body of a top-level LET, to allow
forms such as DEFUN to be meaningful there.  However, we feel that a
cleaner solution is to remove the restrictions on the placement of
defining macros altogether. 


Current Practice:


Cost to implementors:


Cost to users:

None.  This is a compatible extension.


Benefits:

The notion of defining macros as being somehow special is removed from
the language.  Allowing defining macros to appear anywhere instead of
restricting them to certain positions results in a cleaner language
design.


Discussion:
-------